Re: Scroogle and Tor

[scroogle@xxxxxxxxxxx]

> Gregory Maxwell wrote:
>
> As far as performance goes, you can download a list of nodes which can
> reach a particular address at
> https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.2.3.4
> but, these results have the same problem with omitted nodes that I
> mentioned.

That's the torproject.org bulk list I've been using, alternating with
the blutmagie.de list. When I download the torproject.org list I ask
for exit nodes that can reach one of my servers. I alternate between
asking for port 443 and port 80 on that server.

> Someone else emailed me directly:
>
> Seems like you could get a lot smarter about this and block successive
> queries from the same IP that happen less than a few seconds from each
> other.

Difficult, because blutmagie.de and another high-traffic site account for
about 20 percent of my total Tor requests. I have to exempt them from some
of my screening if there's a chance of false positives. I'm already doing
something like what you suggest, after exempting these two sites. It's
normally turned off, but I try this first when I have a problem. I try
other things too before blocking all exit nodes.

Another problem is that search-engine use presents a special challenge.
Often legitimate searchers fire off a few searches in quick succession.
The input box is right there, and they may modify it just slightly and
fire off another search.

An extreme example of this is something I see several times a week
outside of Tor (which is too slow to do this). Someone has a Scroogle
search plugin out there that mimics an instant-search feature for every
keystroke as you key in your search term. This is something Google
introduced last year. But trying to do this on Scroogle is insane.
Even if it works to the user's satisfaction, I consider this extremely
abusive, and I block these IPs for a week as soon as I see it happening.
The reason it's insane is that Scroogle has six servers, while Google
has several hundred thousand servers. I wish these script kiddies would
do the math first!

-- Daniel Brandt



***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxxx with
unsubscribe or-talk    in the body. http://archives.seul.org/or/talk/