On Wed, Nov 25, 2009 at 12:15 PM, Paul Poulain <paul.poulain@biblibre.com> wrote:
Clay Fouts a écrit :
> I've discovered that a web crawler was opening up large numbers of
> simultaneous requests for the RSS/Atom feed.
<snip>
>  If anyone can suggest a solution that can keep
> the system stable while not removing functionality, please let me
> know.
>
couldn't a robots.txt do the job ?


Perhaps combined with targeted ipfilter if the crawler disrespects robots.txt.