However obvious the matters feel now, I became confused enough over certain points about how Neocities works that I wrote Q's: so it stands that I should publicize the workings-out of these points to help prevent them from becoming FAQ's.
Some Reddit users have written about disabling their profiles in order to keep newly-created pages hidden.
The ones who mention it believe that this works, but as an experiment,
I disabled my profile and created a page with no links leading to it (are-you-there.html),
and thus ostensibly deep—and one day later it was indexed by DuckDuckGo.
It seems like the search engines stopped exhaustively indexing my website after either it became too old or I added too many pages,
so I cannot repeat the experiment.
Can you confirm the result, basically, that there is no such thing as a hidden page here?
It is true that pages can be indexed without any internal links or profile activity to lead there—Neocities'
global site map is what gives this away. No matter, as long as homesteaders know how the system works.
So, to users trying to manage the appearance in search results of cruft, or pages which are worse off being presented out of context:
this folk remedy does not work. Instead, make judicious use of the
noindex
and nofollow meta elements.
Wikipedia even says that these tags only work if the pages are not excluded by
robots.txt .
There should be a dry run option for the pull command, it would help diagnose situations like the following:
Pulling causes a carriage return to be added before every line feed in every file (\n -> \r\n, \r\n -> \r\r\n, ad infinitum).
I did get a "cool" corrupted favicon as a side effect but the i n c r e a s i n g l y l o n g s p a c e s in the remaining files are too much to put up with.
This bug is real, and not the result of my text editor attempting to reformat a non-text file: the file changes as soon as it is pulled. If you know how many times a file was effected, this bug is annoying but not dangerous. Run a find-and-replace on all CR/LF sequences using your own fancy text editor.