https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #168746|0 |1 is obsolete| | --- Comment #2 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Created attachment 168750 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168750&action=edit Bug 37308: Add user-agent for SUSHI outgoing requests Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #168750|0 |1 is obsolete| | --- Comment #3 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Created attachment 168751 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168751&action=edit Bug 37308: Add user-agent for SUSHI outgoing requests Signed-off-by: Belal Ahmadi <belal.ahmadi@uwl.ac.uk> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 --- Comment #5 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- (In reply to Katrin Fischer from comment #4)

Can you explain a bit more about this bug and who might be affected?

Hi Katrin: On one of our customers, for a specific SUSHI provider, whenever a 'test' or 'harvest' request was performed, that specific SUSHI provider returned the following HTTP::Response message (shortened for brevity): '_content' => 'error code: 1010', '_request' => bless( { ... '_method' => 'GET', ... }, 'HTTP::Request' ), '_protocol' => 'HTTP/1.1', '_msg' => 'Forbidden', '_headers' => bless( { ... 'server' => 'cloudflare', ... }, 'HTTP::Headers' ), '_rc' => '403', '_max_body_size' => undef }, 'HTTP::Response' ); Investigating error CloudFlare's code 1010, we find the following: "Error 1010: The owner of this website has banned your access based on your browser's signature" is often encountered when web scraping using browser automation tools like Puppetter, Playwright or Selenium." A curl to the endpoint directly from the server worked, and accessing the URL directly from the browser also worked. Only the request happening directly from Koha's LWP::UserAgent returned Forbidden 403 with the above HTTP::Response. It looked like that provider was considering a request from Koha to be a bot and blocking it. Everything works fine after applying the current patch. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |nick@bywatersolutions.com |y.org | --- Comment #7 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- This is quite hard to come up with a test plan for as credentials are required for a specific provider, but I'm available for a quick call to demonstrate the issue if required. Adding Nick here as QA contact because I stole this solution from his patch: https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=32995&attachment=151124 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |major -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #168751|0 |1 is obsolete| | --- Comment #8 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 170377 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170377&action=edit Bug 37308: Add user-agent for SUSHI outgoing requests Signed-off-by: Belal Ahmadi <belal.ahmadi@uwl.ac.uk> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 --- Comment #10 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 170379 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170379&action=edit Bug 37308: (QA follow-up) Tidy code Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.11! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|37526 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37526 [Bug 37526] Handle redirects in SUSHI requests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37308 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #14 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Not backporting to 23.05.x unless requested -- You are receiving this mail because: You are watching all bug changes.