[Bug 41223] New: SUSHI harvest should respect rate limits
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Bug ID: 41223 Summary: SUSHI harvest should respect rate limits Initiative type: --- Sponsorship --- status: Product: Koha Version: 25.05 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: ERM Assignee: jonathan.druart@gmail.com Reporter: jvinke@austinseminary.edu QA Contact: testopia@bugs.koha-community.org CC: jonathan.druart@gmail.com, jonathan.field@openfifth.co.uk, martin.renvoize@openfifth.co.uk, matt.blenkinsop@openfifth.co.uk, pedro.amorim@openfifth.co.uk Many SUSHI providers have rate limits that will prevent successful harvest of usage statistics if they are exceeded. If a harvest job results in a "Fatal - Client has made too many requests 1020" response, there is currently no recourse because rerunning the harvest will result in the issue. If there were either a built-in rate limit or a setting that could be applied to specific Data Providers it would resolve this. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Mathieu Saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mathsabypro@gmail.com --- Comment #1 from Mathieu Saby <mathsabypro@gmail.com> --- +1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 --- Comment #2 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- KIT are actually working on this and I believe they are going to show us their work at Hackfest this year.. I'd like to help them upstream that. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|jonathan.druart@gmail.com |martin.renvoize@openfifth.c | |o.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |Hackfest -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 --- Comment #3 from Michaela Sieber <michaela.sieber@kit.edu> --- Thanks Martin for adding us to CC. Yes, we have developed a plugin that retries background jobs which ended with error code 1011 or 1020. (see also SQL Report on Mattermost Channel ERM : https://chat.koha-community.org/koha-community/pl/egfbk37pjfgy3p6ppbznm99mfr ) Since we needed a quick solution, the plugin is currently compatible with the Koha version we are using in our production environment, which is 24.11.10 at the moment. It has not been tested with newer Koha versions yet, but I expect it should work with them as well. Our plugin is not yet publicly available, but I will inform you as soon as it is ready and published on GitLab. And yes, we will also present the plugin and a case study on COUNTER harvesting in Marseille this year, and probably at KohaCon as well. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38804 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com --- Comment #4 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- What's the state of this bug post-Hackfest? Ideally we would solve this rate limit issue in core Koha rather than relying on a plugin. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|martin.renvoize@openfifth.c |matt.blenkinsop@openfifth.c |o.uk |o.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- I'm assigning this to Matt.. lets use the plugin as an example and submit a full fix to core. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 --- Comment #6 from Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk> --- Is the plugin now ready to be made public? If so I can take a look and work on fitting it into core -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41305 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41307 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41223 --- Comment #7 from Michaela Sieber <michaela.sieber@kit.edu> --- (In reply to Matt Blenkinsop from comment #6)
Is the plugin now ready to be made public? If so I can take a look and work on fitting it into core
https://gitlab.com/kit-library/koha-plugin-erm-harvesting-management -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org