[LLVMdev] llvm.org robots.txt prevents crawling by Google code search?
Nick Lewycky
nicholas at mxc.ca
Wed Oct 13 23:18:36 PDT 2010
Anton Korobeynikov wrote:
>> indexing the llvm.org svn archive. This means that when you search for an
>> LLVM-related symbol in code search, you get one of the many (possibly
>> out-of-date) mirrors, rather than the up-to-date llvm.org version. This is
>> sad.
> This is intentional. The workload of the server was pretty huge w/o this.
That was the old server though, wasn't it? Would we actually have any
problems if we reenabled this?
Nick
More information about the llvm-dev
mailing list