[Koha-bugs] [Bug 31242] Add rate-limiting to the REST API

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31242

--- Comment #4 from David Cook <dcook at prosentient.com.au> ---
(In reply to Kyle M Hall from comment #3)
> (In reply to David Cook from comment #2)
> > Maybe it does make sense to put the rate limiting in Starman just to keep
> > things as simple as possible. 
> 
> Yes, that was the idea. If it's baked in, it's less onerous for libraries. 

While technically it might not be ideal, I think it would be practical for a
FOSS project like Koha for sure. Having something out of the box makes sense. 

I often think about how X, Y, Z technologies would be great for Koha, but how
they're not practical because they are too complex for non-technical Koha
adopters...

I answered a query about horizontally scaling Koha, and I suppose if you were
to horizontally scale Koha, you'd want your rate limiting to be done by your
load balancer. But we can cross that bridge when we come to it too.

> > We could just put it in the builder for "/api/v1/app.pl", so it wouldn't
> > affect the performance of the /opac and /intranet apps. 
> 
> Is there a benefit to rate limiting everything? It hasn't been an issue
> lately but back in the day I know we had an issue with "thrashing" where
> some partner's browsers would DOS a server with requests for reasons we
> never understood.

If we rate limited everything (without IP checking), I think they'd still DOS
Koha just by hitting the global rate limit. 

We still occasionally have issues where out of control bots (usually benign
crawlers) will DOS a Koha instance (not the whole server fortunately) by
keeping Starman busy, but we have additional software that finds and blocks
those. 

In theory, we could install fail2ban alongside Koha and point it at Plack
logs... (I suppose that is a different kind of rate limiting)

> > The middlewares Kyle linked do look ancient though.
> 
> True, but if it works, it doesn't need updated!

Haha. That's what I was thinking as well. 

> > https://www.krakend.io/docs/endpoints/rate-limit/ poses some interesting
> > points about rate limiting methods, especially whether to rate limit by
> > endpoint or rate limit by endpoint * client IP address. 
> 
> That *is* interesting! I seems more "fair" to do it by endpoint+ip, but
> really, a system can only handle some much traffic no matter how fair it is.
> Now I'm going to say it would be nice to have both of those, and for them to
> each be configurable ;)

I don't know if you followed the link, but that's what they recommend! "Both
and" rather than "Either or". You determine how many simultaneous clients you
expect to handle, give each client up to X requests per Y and then have a Z
global limit that should be the sum of X for all clients. Probably would need
to tweak it a bit from there but not a bad guideline.

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.