[Bug 31104] New: Pagination generates HTTP "Link:" header which is over 8291 bytes apache's limit
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Bug ID: 31104 Summary: Pagination generates HTTP "Link:" header which is over 8291 bytes apache's limit Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: minor Priority: P5 - low Component: REST API Assignee: tomascohen@gmail.com Reporter: nugged@gmail.com CC: tomascohen@gmail.com to repeat: 1. Go to /cgi-bin/koha/members/member.pl 2. put this test into the "Search for patron" "Search:" field: "some long test to search in patron fields" 3. Observe "nothing found" but in the server log, it will say (28)No space left on device: [client 127.0.0.1:60330] AH10124: header size is over the limit allowed by ResponseFieldSize (8192 bytes). Bad response header: 'Link: <https://... this of course happens when you searching for something long but seems we came to so long URL which fails to fit into the "forward" (Link: ... header) field, if to debug, it creates a "search" API request URL that is already 5219 bytes long, then this URL is re-encoded and put two times inside the "Link:" header, so it quickly jumps over the 8192 bytes header limit. Note: as we found together that solution in Koha/REST/Plugin/Pagination.pm to split @links in separate "Link:" headers. SIDENOTE: considering that we already have URL 5129 long in this example, we might anyway even without header doubling bump into the 8192 limits (!). We might need re-think later how in general this formed up. Not related to the current shorter fix though. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Andrew Nugged <nugged@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Pagination generates HTTP |Pagination generates HTTP |"Link:" header which is |"Link:" header which is |over 8291 bytes apache's |over 8192 bytes apache's |limit |limit -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 --- Comment #1 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 137240 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137240&action=edit Bug 31104: Put each link on a separate header This patch makes the Pagination helper put each Link entry on the response headers on a separate header. In practice, this leads to shorter headers and will avoid apache default/reasonable limits. To test: 1. Open the patron search page 2. Open the koha logs: $ tail -f /var/log/koha/kohadev/*.log 3. On the 'Search for patron' input, use a really long string, like 'superlongstringthatistoolongforapacheyeahsuperlong' => SUCCESS: There's no user with an attribute with that content :-D => FAIL: You get an error like (28)No space left on device: [client 127.0.0.1:60330] AH10124: header size is over the limit allowed by ResponseFieldSize (8192 bytes). Bad response header: 'Link:...' 4. Apply this patch 5. Restart all: $ restart_all 6. Repeat 2 and 3 => SUCCESS: No results => SUCCESS: No error in the logs 7. Run: $ kshell k$ prove t/Koha/REST/Plugin/Pagination.t \ t/db_dependent/api/v1/*.t => SUCCESS: Tests pass! Note 1: The Link header is expected to be in CSV format, which is usually the way libraries treat repeated headers anyway. Note 2: Apache is not rejecting the response, just stripping out the 'bad header'. So this is not a critical issue. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com, | |kyle@bywatersolutions.com, | |lari.taskula@hypernova.fi, | |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #2 from David Nind <david@davidnind.com> --- I'm not seeing the error described in step 3. Are there some other settings required to generate the error? Testing on master with koha-testing-docker. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to David Nind from comment #2)
I'm not seeing the error described in step 3.
I had to put a longer and longer string to reach the limit. Just add some random chars. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Michal Urban <michalurban177@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Michal Urban <michalurban177@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137240|0 |1 is obsolete| | --- Comment #4 from Michal Urban <michalurban177@gmail.com> --- Created attachment 137433 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137433&action=edit Bug 31104: Put each link on a separate header This patch makes the Pagination helper put each Link entry on the response headers on a separate header. In practice, this leads to shorter headers and will avoid apache default/reasonable limits. To test: 1. Open the patron search page 2. Open the koha logs: $ tail -f /var/log/koha/kohadev/*.log 3. On the 'Search for patron' input, use a really long string, like 'superlongstringthatistoolongforapacheyeahsuperlong' => SUCCESS: There's no user with an attribute with that content :-D => FAIL: You get an error like (28)No space left on device: [client 127.0.0.1:60330] AH10124: header size is over the limit allowed by ResponseFieldSize (8192 bytes). Bad response header: 'Link:...' 4. Apply this patch 5. Restart all: $ restart_all 6. Repeat 2 and 3 => SUCCESS: No results => SUCCESS: No error in the logs 7. Run: $ kshell k$ prove t/Koha/REST/Plugin/Pagination.t \ t/db_dependent/api/v1/*.t => SUCCESS: Tests pass! Note 1: The Link header is expected to be in CSV format, which is usually the way libraries treat repeated headers anyway. Note 2: Apache is not rejecting the response, just stripping out the 'bad header'. So this is not a critical issue. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Michal Urban <michalurban177@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137433|0 |1 is obsolete| | --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 137517 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137517&action=edit Bug 31104: Put each link on a separate header This patch makes the Pagination helper put each Link entry on the response headers on a separate header. In practice, this leads to shorter headers and will avoid apache default/reasonable limits. To test: 1. Open the patron search page 2. Open the koha logs: $ tail -f /var/log/koha/kohadev/*.log 3. On the 'Search for patron' input, use a really long string, like 'superlongstringthatistoolongforapacheyeahsuperlong' => SUCCESS: There's no user with an attribute with that content :-D => FAIL: You get an error like (28)No space left on device: [client 127.0.0.1:60330] AH10124: header size is over the limit allowed by ResponseFieldSize (8192 bytes). Bad response header: 'Link:...' 4. Apply this patch 5. Restart all: $ restart_all 6. Repeat 2 and 3 => SUCCESS: No results => SUCCESS: No error in the logs 7. Run: $ kshell k$ prove t/Koha/REST/Plugin/Pagination.t \ t/db_dependent/api/v1/*.t => SUCCESS: Tests pass! Note 1: The Link header is expected to be in CSV format, which is usually the way libraries treat repeated headers anyway. Note 2: Apache is not rejecting the response, just stripping out the 'bad header'. So this is not a critical issue. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Michal Urban <michalurban177@gmail.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice clear fix Tomas, thanks. All works as expected.. perhaps it might have been nice to see a unit test for it, but otherwise, I'm happy here. Passing QA. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |22.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 --- Comment #7 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 --- Comment #8 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 137610 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137610&action=edit Bug 31104: Add a selenium test -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |additional_work_needed --- Comment #9 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Will still pass after bug 31138. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|additional_work_needed | --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Thanks, Jonathan! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137610|0 |1 is obsolete| | --- Comment #11 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 137628 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137628&action=edit Bug 31104: Add a selenium test -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |additional_work_needed --- Comment #12 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- No sure what's going on here, but the selenium tests are failing on jenkins. They are passing with and without commit "Bug 31104: Put each link on a separate header" They are failing after bug 31138. And actually this patch does not fix the problem, I am still getting "414 Request-URI Too Long". -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|additional_work_needed | --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Tests reverted. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Keywords| |rel_22_05_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 George Williams (NEKLS) <george@nekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |george@nekls.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31104 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39637 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org