[Bug 24756] New: Occasional D8 and U18 failures for Koha/XSLT/Security.t
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Bug ID: 24756 Summary: Occasional D8 and U18 failures for Koha/XSLT/Security.t Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Test Suite Assignee: chris@bigballofwax.co.nz Reporter: m.de.rooy@rijksmuseum.nl QA Contact: testopia@bugs.koha-community.org not ok 7 - Remote import does not fail on read_net This occurs on D8 and U18 but not consistently? For instance, at this very moment all Jenkins tests passed on D8 but master and 18.11 fail here for U18. Unfortunately, I cant even reproduce the error on D8 and D9 in my test containers. Trying to resolve it here by catching the warnings 'manually'. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|chris@bigballofwax.co.nz |m.de.rooy@rijksmuseum.nl Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 99746 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99746&action=edit Bug 24756: Fix D8 and U18 failures for Koha/XSLT/Security.t Instead of warning_like, we eventually catch multiple warnings and look if we catch one specific warn and not catch another specific one. Test plan: Run t/db_dependent/Koha/XSLT/Security.t on D8, D9, D10 or U18. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Passes on D8 and D9 for me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |23290 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23290 [Bug 23290] XSLT system preferences allow administrators to exploit XML and XSLT vulnerabilities -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org, | |martin.renvoize@ptfs-europe | |.com, tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Another option is to just remove this test. In the script we test the read_net case already. This is just something additional that we could miss too.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #3 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Skipping QA so we can see how Jenkins responds. Certainly, no harm will be done by this change :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- U18 on master: not ok 7 - Expected import error ok 8 - No read_net warn for remote import -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 99753 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99753&action=edit Bug 24756: Skip last test for U18 Unless someone comes up with the real warnings, skip this test. Tried to build an U18 container, but didnt work out. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Any feedback on this "last resort" follow-up ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Yeah, skip that skip. D9 failed for MDB? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #99753|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 99754 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99754&action=edit Bug 24756: Fix failing tests by removing temporary filenames In order to resolve the random failure, that might be caused by reusing filenames in File::Temp and caching xslt code, this patch replaces the temporary file by using the code parameter. Note: Since XSLT uses a digest based on the passed code fragment, we will exclude collisions here. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Martin, Assuming that the failure might be in a combination of not unique file names and caching somehow, I added a second patch and tested it on D8, D9 and D10. Hopefully catching it now.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #10)
Martin, Assuming that the failure might be in a combination of not unique file names and caching somehow, I added a second patch and tested it on D8, D9 and D10. Hopefully catching it now..
Maybe I added the literal files once just for that reason. Who knows? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Occasional D8 and U18 |Occasional failures for |failures for |Koha/XSLT/Security.t |Koha/XSLT/Security.t | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |BLOCKED --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Master D10 #100 29-02-2020 19:45 not ok 7 - Expected import error ok 8 - No read_net warn for remote import Master D9 My8 #179 28-02-2020 15:57 idem not ok 7, ok 8 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 99858 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99858&action=edit Bug 24756: Show the warnings on Jenkins Temporary measure to see which warnings Jenkins got while we cannot reproduce them ourselves on D8, D9 and D10. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- This produces output (in my D9 testing environment) like: ok 7 - Expected import error. Additional info: $VAR1 = 'XSLT::Base: No such file or directoryI/O warning : failed to load external entity "http://notexpected.koha-community.org/noxsl/nothing.xsl" # compilation error: file unknown-558d811413c0 element import # xsl:import : unable to load http://notexpected.koha-community.org/noxsl/nothing.xsl # at /usr/share/koha/Koha/XSLT/Base.pm line 343. # '; # ok 8 - No read_net warn for remote import Hopefully we'll know more when we run this on Jenkins? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #14)
Hopefully we'll know more when we run this on Jenkins?
=== https://jenkins.koha-community.org/job/Koha_Master_D8/725/ not ok 7 - Expected import error. Additional info: $VAR1 = 'XSLT::Base: I/O warning : failed to load HTTP resource # compilation error: file unknown-3d22ae0 element import # xsl:import : unable to load http://notexpected.koha-community.org/noxsl/nothing.xsl # at /kohadevbox/koha/Koha/XSLT/Base.pm line 343. # '; # ok 8 - No read_net warn for remote import Here the wording "failed to load HTTP resource" is used instead of "failed to load external entity". It appears that both can come up. libxml2 contains both strings failed to load external entity "%s" failed to load HTTP resource "%s" and even: Failed to load externalRef %s Failed to load include %s (But we did not see the last two.) So we probably just need: is( ( grep { /failed to load (external entity|HTTP resource)/ } @warn ), 1, 'Expected import error' ); I will wait a few days now before replacing the additional Dumper info. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #16 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 99892 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99892&action=edit Bug 24756: Catch two failed to load-variations in Security.t Test 7 in Security.t failed when 'failed to load HTTP resource' was returned by libxml2 instead of 'failed to load external entity'. We now remove the debugging with Dumper again and adjust the regex to include both variations. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com --- Comment #18 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- It seems that we need it for 19.11 as well 19.11_D9 (79) koha_1 | # Failed test 'Remote import does not fail on read_net' koha_1 | # at t/db_dependent/Koha/XSLT/Security.t line 124. koha_1 | # found warning: XSLT_Handler: I/O warning : failed to load HTTP resource koha_1 | # expected to find warning: (?^u:I/O warning : failed to load external entity) koha_1 | # Looks like you failed 1 test of 7. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #19 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #18)
It seems that we need it for 19.11 as well
19.11_D9 (79)
koha_1 | # Failed test 'Remote import does not fail on read_net' koha_1 | # at t/db_dependent/Koha/XSLT/Security.t line 124. koha_1 | # found warning: XSLT_Handler: I/O warning : failed to load HTTP resource koha_1 | # expected to find warning: (?^u:I/O warning : failed to load external entity) koha_1 | # Looks like you failed 1 test of 7.
This should be backported, yes. The RMaints look in this queue, right? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #20 from Joy Nelson <joy@bywatersolutions.com> --- this does not apply cleanly to 19.11.x branch. Please rebase. Thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #21 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 102270 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102270&action=edit Bug 24756: [19.11.X] Catch two failed to load-variations in Security.t Test 7 in Security.t failed when 'failed to load HTTP resource' was returned by libxml2 instead of 'failed to load external entity'. We now remove the debugging with Dumper again and adjust the regex to include both variations. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 --- Comment #22 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Joy Nelson from comment #20)
this does not apply cleanly to 19.11.x branch. Please rebase. Thanks!
Hi Joy. Actually you only need the last patch marked 19.11.X. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #102270|0 |1 is obsolete| | --- Comment #23 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 102271 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102271&action=edit Bug 24756: [19.11.X] Catch two failed to load-variations in Security.t Test 7 in Security.t failed when 'failed to load HTTP resource' was returned by libxml2 instead of 'failed to load external entity'. We now remove the debugging with Dumper again and adjust the regex to include both variations. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|20.05.00 |20.05.00, 19.11.05 released in| | --- Comment #24 from Joy Nelson <joy@bywatersolutions.com> --- backported 19.11 patch to 19.11.x branch for 19.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24756 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.05.00, 19.11.05 |20.05.00, 19.11.05, released in| |19.05.10 Status|Pushed to stable |Pushed to oldstable CC| |lucas@bywatersolutions.com --- Comment #25 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 19.05.x for 19.05.10 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org