https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- (In reply to Kyle M Hall (khall) from comment #0)

Botnets and web spiders can easily bring a Koha instance down. We should implement optional rate limiting in Koha. It appears there are two plack middlewares we could use: https://metacpan.org/pod/Plack::Middleware::Throttle or https://metacpan.org/pod/Plack::Middleware::Throttle::Lite

Throttle supports memcached as a backend, but is more complex. Lite uses an in-memory backend, but has fewer dependencies.

I was inspired by Plack::Middleware::Throttle to write my own Koha::Middleware::Throttle. It uses memcached as the storage backend, and its expiring keys mechanism means it is self-managing, which is great. My version does use some local custom code for managing configuration in a more flexible way, so I'll need to strip that out, but it's still useful without it. Happy to share my work here. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion Assignee|koha-bugs@lists.koha-commun |dcook@prosentient.com.au |ity.org | --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Test plan: 1. vi /etc/koha/sites/kohadev/plack.psgi 2. Add the following to the /opac mount point (higher than Koha::Middlware::CSRF but lower than HTTPExceptions) : enable "+Koha::Middleware::Throttle", debug => 1, interface => 'opac', paths => [ '^/opac-search', ], allow_list => [ '192.168.1.0/24', ], request_threshold => 5, #Number of requests to trigger throttle last_n_minutes => 1, #Number of minutes to consider when aggregating request counts cidr_block => 24, #CIDR mask to use for determing range IP address belongs to (may only be 8, 16, 24, 32) ; 3. sudo koha-plack --restart kohadev 4. Do 6 requests to opac-search.pl within 1 minute and see the pretty "You've been throttled" page. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- I was inspired by Plack::Middleware::Throttle, but I found it to be too limited for our purposes. You'll likely notice limited use of Koha::Cache, and that's because we don't implement the "incr" method in Koha::Cache, which is vital for multi-process safety in this feature. -- I think there's lots of room for improvement with this feature, although it is limited in what it can do against distributed botnets and web spiders. That said, I just looked at some logs, and I saw it throttled a couple of spiders, so every little bit helps. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Thanks for sharing this. We are currently under fire too :) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #4)

I have other work to present an interactive challenge for users, but it's not currently compatible with this feature, nor ready to be shared just yet.

I think a challenge page is better than a blocked page, as it gives legitimate human users an opportunity to get back to searching. But that's a whole other thing.

If you submit, please keep me in cc. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39109 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |lukasz.koszyk@kit.edu, | |michaela.sieber@kit.edu, | |raphael.straub@kit.edu -- You are receiving this mail because: You are watching all bug changes.