[llvm-dev] LLVM.org/viewvc down?

[Joerg Sonnenberger via llvm-dev]

On Thu, Mar 17, 2016 at 01:56:18PM +0300, Anton Korobeynikov wrote:
> > How much of that are stupid (chinese) spiders?
> You're right, the majority of them were from Chinese IPs.
> 
> > I had a lot of load
> > issues for the NetBSD fossil repositories until I added the following to
> > my nginx configuration:
> >
> >   if ($http_user_agent ~* (360Spider|80legs|App3leWebKit|Baiduspider|EasouSpider)) {
> >     return 403;
> >   }
> I saw some other user agents as well. And they ignored robots.txt, yes.

Could you start reenabling the service with such an exclude list and
maybe slowly extend it to cover UAs from the same IP ranges produces more
than 100MB/day or so over a few days?

Joerg