[LLVMdev] llvm.org robots.txt prevents crawling by Google code search?

Nick Lewycky nicholas at mxc.ca
Wed Oct 13 23:18:36 PDT 2010


Anton Korobeynikov wrote:
>> indexing the llvm.org svn archive. This means that when you search for an
>> LLVM-related symbol in code search, you get one of the many (possibly
>> out-of-date) mirrors, rather than the up-to-date llvm.org version. This is
>> sad.
> This is intentional. The workload of the server was pretty huge w/o this.

That was the old server though, wasn't it? Would we actually have any 
problems if we reenabled this?

Nick



More information about the llvm-dev mailing list