[Bug 41273] New: xt/perltidy.t could use the same parallel handling as other tests
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Bug ID: 41273 Summary: xt/perltidy.t could use the same parallel handling as other tests Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Test Suite Assignee: chris@bigballofwax.co.nz Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org Some of our tests look for the number of CPU's in the environment they're running on and then split the tests to run in parallel. We could do that for xt/perltidy.t too and save time when running those tests locally. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|chris@bigballofwax.co.nz |martin.renvoize@openfifth.c | |o.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 189714 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=189714&action=edit Bug 41273: Parallelize test execution for improved performance This patch adds parallel processing to xt/perltidy.t using Parallel::ForkManager, following the same pattern as xt/pl_valid.t and xt/tt_tidy.t. Changes: - Uses Parallel::ForkManager to process files concurrently - Auto-detects CPU count via Sys::CPU (respects KOHA_PROVE_CPUS env var) - Preserves working directory context in forked child processes - Maintains correct failure tracking in %results hash Performance improvement: ~N× speedup on N-core systems (e.g., 8× faster on 8-core, 16× faster on 16-core machines) Test plan: 1. Clear incremental test cache (if needed): docker exec kohadev-koha bash -c 'rm -rf /tmp/koha-ci-results/perltidy*' 2. Run test: docker exec kohadev-koha bash -c 'KOHA_PROVE_CPUS=8 prove xt/perltidy.t' 3. Verify all 3138 tests pass 4. Verify test completes significantly faster than before -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 --- Comment #2 from Jonathan Druart <jonathan.druart@gmail.com> --- +# Capture the current working directory before forking +my $cwd = getcwd(); + # Ensure we're in the correct directory in the forked process + chdir($cwd); Do we really need that? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Jonathan Druart from comment #2)
+# Capture the current working directory before forking +my $cwd = getcwd();
+ # Ensure we're in the correct directory in the forked process + chdir($cwd);
Do we really need that?
@Martin: can you have a look at Joubu's question please? Patch still applies and passes QA tests. Setting FQA to raise attention. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #189714|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 209363 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=209363&action=edit Bug 41273: Parallelize test execution for improved performance This patch adds parallel processing to xt/perltidy.t using Parallel::ForkManager, following the same pattern as xt/pl_valid.t and xt/tt_tidy.t. Changes: - Uses Parallel::ForkManager to process files concurrently - Auto-detects CPU count via Sys::CPU (respects KOHA_PROVE_CPUS env var) - Preserves working directory context in forked child processes - Maintains correct failure tracking in %results hash Performance improvement: ~N× speedup on N-core systems (e.g., 8× faster on 8-core, 16× faster on 16-core machines) Test plan: 1. Clear incremental test cache (if needed): docker exec kohadev-koha bash -c 'rm -rf /tmp/koha-ci-results/perltidy*' 2. Run test: docker exec kohadev-koha bash -c 'KOHA_PROVE_CPUS=8 prove xt/perltidy.t' 3. Verify all 3138 tests pass 4. Verify test completes significantly faster than before -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Thanks Katrin, and Joubu for the review. You are right, we don't need it: a forked child inherits the working directory of its parent, so there is nothing to restore, and the other parallel tests (xt/tt_tidy.t, xt/pl_valid.t, t/db_dependent/00-strict.t) don't do it either. I have removed the getcwd/chdir lines and the Cwd import (attachment replaced). To check, I ran the whole of xt/perltidy.t in KTD with KOHA_PROVE_CPUS=6 without them: all 3373 files resolve their relative paths correctly in the workers. The one failure is t/db_dependent/api/v1/messages.t, which is not tidy on main (the file is not touched by this patch, it comes from bug 23998), so the full run currently fails regardless of this patch until that file is tidied. Setting back to Needs Signoff. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41273 --- Comment #6 from Jonathan Druart <jonathan.druart@gmail.com> --- Please don't forget we have bug 43505 in the queue... Also if you really want to speed it up you should simply use KOHA_CI_INCREMENTAL_RUNS=1 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org