[llvm-dev] llvm bugzilla not indexable
Jacob Lifshay via llvm-dev
llvm-dev at lists.llvm.org
Sun Oct 7 13:07:28 PDT 2018
It probably could be setup to whitelist some of the well-behaved search
engine bots. If it's set to disallow by default then the broken bots should
not try to index it.
On Sun, Oct 7, 2018, 12:57 Anton Korobeynikov via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> We explicitly disallow indexing of bugzilla due to lots of bogus
> requests from broken bots which create major workload on llvm.org.
> On Sun, Oct 7, 2018 at 7:21 PM Ties Stuij via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > Hi all,
> > As I was perusing LLVM Bugzilla, I noticed that it's not indexed by
> search engines, probably thanks to the robots.txt file* which inhibits all
> > I think that it's extremely useful for these issues to show up in say
> google search queries, so I wonder if it's just an oversight or if there's
> another reason for this.
> > Thanks,
> > /Ties
> > * http://bugs.llvm.org/robots.txt:
> > User-agent: *
> > Allow: /index.cgi
> > Disallow: /
> > IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sender immediately and do not disclose the
> contents to any other person, use it for any purpose, or store or copy the
> information in any medium. Thank you.
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> With best regards, Anton Korobeynikov
> Department of Statistical Modelling, Saint Petersburg State University
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev