[Bug 18979] New: Speed up 'valid-templates.t' tests
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Bug ID: 18979 Summary: Speed up 'valid-templates.t' tests Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: minor Priority: P5 - low Component: Test Suite Assignee: jonathan.druart@bugs.koha-community.org Reporter: jonathan.druart@bugs.koha-community.org QA Contact: testopia@bugs.koha-community.org Depends on: 18055 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18055 [Bug 18055] Speed up '00-strict.t' test -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 65208 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65208&action=edit Bug 18979: Speed up 'valid-templates.t' tests Test plan: Confirm than prove xt/author/valid-templates.t is quicker with this patch. Before I got: Files=1, Tests=840, 21 wallclock secs ( 0.18 usr 0.02 sys + 20.28 cusr 0.32 csys = 20.80 CPU) After: Files=1, Tests=840, 17 wallclock secs ( 0.06 usr 0.01 sys + 21.56 cusr 0.48 csys = 22.11 CPU) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marc Véron <veron@veron.ch> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |veron@veron.ch --- Comment #2 from Marc Véron <veron@veron.ch> --- Testing on top of Bug 18055 Testing without patch: Files=1, Tests=838, 40 wallclock secs ( 0.17 usr 0.01 sys + 28.71 cusr 0.73 csys = 29.62 CPU) With patch: Can't locate Parallel/ForkManager.pm (...) you may need to install the Parallel::ForkManager module -> Installing Parallel::ForkManager Can't locate Sys/CPU.pm ->Installing Sys::CPU Re-testing without patch (4 times): Files=1, Tests=838, 33 wallclock secs ( 0.15 usr 0.02 sys + 28.68 cusr 0.38 csys = 29.23 CPU) Files=1, Tests=838, 30 wallclock secs ( 0.14 usr 0.01 sys + 28.08 cusr 0.32 csys = 28.55 CPU) Files=1, Tests=838, 30 wallclock secs ( 0.14 usr 0.01 sys + 28.61 cusr 0.22 csys = 28.98 CPU) Files=1, Tests=838, 31 wallclock secs ( 0.15 usr 0.01 sys + 28.34 cusr 0.27 csys = 28.77 CPU) Testing with patch (4 times): Files=1, Tests=838, 36 wallclock secs ( 0.06 usr 0.00 sys + 31.04 cusr 0.60 csys = 31.70 CPU) Files=1, Tests=838, 34 wallclock secs ( 0.07 usr 0.00 sys + 29.98 cusr 0.53 csys = 30.58 CPU) Files=1, Tests=838, 32 wallclock secs ( 0.06 usr 0.00 sys + 29.53 cusr 0.65 csys = 30.24 CPU) Files=1, Tests=838, 32 wallclock secs ( 0.05 usr 0.01 sys + 29.74 cusr 0.53 csys = 30.33 CPU) Result on my vm: It seems that installing Parallel::ForkManager and Sys::CPU speeded up xt/author/valid-templates.t (31 - 33 wallclock sec instead of 40 sec, without patch) With patch, xt/author/valid-templates.t was comparable or slower (32 - 36 wallclock sec, with patch) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |18993 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18993 [Bug 18993] Bump libtest-simple-perl to 1.302075 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 David Bourgault <david.bourgault@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65208|0 |1 is obsolete| | --- Comment #3 from David Bourgault <david.bourgault@inlibro.com> --- Created attachment 67484 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=67484&action=edit Bug 18979: Speed up 'valid-templates.t' tests Test plan: Confirm than prove xt/author/valid-templates.t is quicker with this patch. Before I got: Files=1, Tests=840, 21 wallclock secs ( 0.18 usr 0.02 sys + 20.28 cusr 0.32 csys = 20.80 CPU) After: Files=1, Tests=840, 17 wallclock secs ( 0.06 usr 0.01 sys + 21.56 cusr 0.48 csys = 22.11 CPU) Signed-off by: David Bourgault <david.bourgault@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 David Bourgault <david.bourgault@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david.bourgault@inlibro.com Status|Needs Signoff |Signed Off --- Comment #4 from David Bourgault <david.bourgault@inlibro.com> --- I tested 10 times, before and after. Before it averaged 18.4 wallclock seconds per run After it average dropped slightly to 16.6 wallclock seconds It's a very minor improvement, but there seems to be one. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Files=1, Tests=842, 42 wallclock secs ( 0.20 usr 0.02 sys + 41.23 cusr 0.40 csys = 41.85 CPU) Files=1, Tests=842, 37 wallclock secs ( 0.06 usr 0.01 sys + 42.91 cusr 0.84 csys = 43.82 CPU) Does this justify another dependency ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Marcel de Rooy from comment #5)
Files=1, Tests=842, 42 wallclock secs ( 0.20 usr 0.02 sys + 41.23 cusr 0.40 csys = 41.85 CPU)
Files=1, Tests=842, 37 wallclock secs ( 0.06 usr 0.01 sys + 42.91 cusr 0.84 csys = 43.82 CPU)
Does this justify another dependency ?
The dependency is used by bug 18055 as well, and we gain 10min -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- About the "use threads;" line:
From the perl doc:
The "interpreter-based threads" provided by Perl are not the fast, lightweight system for multitasking that one might expect or hope for. Threads are implemented in a way that make them easy to misuse. Few people know how to use them correctly or will be able to provide help. The use of interpreter-based threads in perl is officially discouraged. Do we really need it ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Moving to Discussion for feedback. Feel free to move back later.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com --- Comment #9 from Mason James <mtj@kohaaloha.com> --- (In reply to Marcel de Rooy from comment #7)
About the "use threads;" line:
Do we really need it ?
hi Marcel, i think it's not needed :) i must have forgotten to remove the line before sending, i'll fix that now... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #10 from Mason James <mtj@kohaaloha.com> --- a little update here... it turns out that the 'use threads' line was needed for patch to succeed, after all the 'use threads' line seems to handle a known bug where Test::Simple generates the TAP tests out of order, and causes the tests to fail the good news is that there are some options... 1/ use the Test2::Suite module, which handles parallel tests well... https://metacpan.org/pod/Test2::Suite the only catch here is that it's new-ish, so we'll need to backport the module to deb8/jessie too 2/ just allow the 'use threads' line. Because no threads are ever actually created, it should be pretty safe fyi: i did try using forks.pm too, but sadly got lots of random fails -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #11 from Mason James <mtj@kohaaloha.com> --- Created attachment 69150 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69150&action=edit Bug 18979: Swap threads for Test2::Suite this patch swaps threads for the Test2::Suite module to test... 1/ apply patch 2/ install Test2::Suite $ sudo apt-get install libtest2-suite-perl , or... $ cpanm Test2::Suite 3/ run test $ sudo koha-shell -c " export PERL5LIB=/home/mason/git/koha/master; prove -v xt/author/valid-templates.t" kmaster ok 1 - Testing opac bootstrap templates { ... } ok 2 - Testing staff prog templates { ... } 1..2 ok All tests successful. Result: PASS 4. note test passes Result: PASS -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Mason James from comment #10)
a little update here... it turns out that the 'use threads' line was needed for patch to succeed, after all
the 'use threads' line seems to handle a known bug where Test::Simple generates the TAP tests out of order, and causes the tests to fail
the good news is that there are some options...
1/ use the Test2::Suite module, which handles parallel tests well... https://metacpan.org/pod/Test2::Suite
the only catch here is that it's new-ish, so we'll need to backport the module to deb8/jessie too
2/ just allow the 'use threads' line. Because no threads are ever actually created, it should be pretty safe
fyi: i did try using forks.pm too, but sadly got lots of random fails
Mason, Which option has your preference ? Marcel -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #13 from Mason James <mtj@kohaaloha.com> --- (In reply to Marcel de Rooy from comment #12)
(In reply to Mason James from comment #10)
a little update here... it turns out that the 'use threads' line was needed for patch to succeed, after all
the 'use threads' line seems to handle a known bug where Test::Simple generates the TAP tests out of order, and causes the tests to fail
the good news is that there are some options...
1/ use the Test2::Suite module, which handles parallel tests well... https://metacpan.org/pod/Test2::Suite
the only catch here is that it's new-ish, so we'll need to backport the module to deb8/jessie too
2/ just allow the 'use threads' line. Because no threads are ever actually created, it should be pretty safe
fyi: i did try using forks.pm too, but sadly got lots of random fails
Mason, Which option has your preference ?
Marcel
hiya, i'd vote for option 1/ as quick fix, and then a separate bug for a leisurely switch to Test2::Suite anyone else have an opinion? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #14 from Mason James <mtj@kohaaloha.com> ---
hiya, i'd vote for option 1/ as quick fix, and then a separate bug for a leisurely switch to Test2::Suite
oops, i meant option 2, then a follow-up for option 1 :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Bug 18979 depends on bug 18993, which changed state. Bug 18993 Summary: Bump libtest-simple-perl to 1.302073 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18993 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #15 from Mason James <mtj@kohaaloha.com> --- (In reply to Mason James from comment #14)
hiya, i'd vote for option 1/ as quick fix, and then a separate bug for a leisurely switch to Test2::Suite
oops, i meant option 2, then a follow-up for option 1 :)
Marcel, lets go for option 2 here... i'll obsolete the newer Test2::Suite patch, for now ..and set the status back to SIGNED-OFF -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #69150|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Marcel, could you have another look here? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #67484|0 |1 is obsolete| | --- Comment #17 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 71552 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71552&action=edit Bug 18979: Speed up 'valid-templates.t' tests Test plan: Confirm than prove xt/author/valid-templates.t is quicker with this patch. Before I got: Files=1, Tests=840, 21 wallclock secs ( 0.18 usr 0.02 sys + 20.28 cusr 0.32 csys = 20.80 CPU) After: Files=1, Tests=840, 17 wallclock secs ( 0.06 usr 0.01 sys + 21.56 cusr 0.48 csys = 22.11 CPU) Signed-off-by: David Bourgault <david.bourgault@inlibro.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Katrin Fischer from comment #16)
Marcel, could you have another look here?
I added a forgotten signoff line from David. I have no objection to this patch, but also note that forking makes us unaware of the warnings raised by the tests like: # excluding template /usr/share/koha/devclone/koha-tmpl/opac-tmpl/bootstrap/en/includes/opac-bottom.inc because it cannot stand on its own etc. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #19 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 18.05, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18979 Bug 18979 depends on bug 18055, which changed state. Bug 18055 Summary: Speed up '00-strict.t' test https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18055 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org