Hi, I'm configurating a site to be as SEO friendly as possible. Which folder is the root, where to place the "robots.txt" file? Do "bot's" have "access" to the folders one step up from the default web folder? /cgi-bin, /private etc? Thanks in advance
The root folder of the site is the web folder, so the robots.txt file has to be placed into the web folder.
"web" is the root! Hi Till, Thanks for the quick answer The root folder of the site is the web folder, so the robots.txt file has to be placed into the web folder. Which means, the directories e.g. "/cgi-bin" is not accessable for the "bots"
Not unless you enable CGI for a web site. CGI/Perl scripts have to be placed in the cgi-bin directory.