[Bug 28278] New: Improve $KOHA_CONF parsing speed by using XML::LibXML
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Bug ID: 28278 Summary: Improve $KOHA_CONF parsing speed by using XML::LibXML Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: julian.maurice@biblibre.com Reporter: julian.maurice@biblibre.com QA Contact: testopia@bugs.koha-community.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #1 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 120374 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120374&action=edit Bug 28278: Add unit test for Koha::Config::read_from_file -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #2 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 120375 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120375&action=edit Bug 28278: Improve $KOHA_CONF parsing speed by using XML::LibXML - Without the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 3.585 s ± 0.018 s [User: 3.531 s, System: 0.049 s] Range (min … max): 3.547 s … 3.612 s 10 runs - With the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 1.122 s ± 0.028 s [User: 1.104 s, System: 0.014 s] Range (min … max): 1.095 s … 1.189 s 10 runs Test plan: 1. Apply the first patch (the one with the unit tests) and make sure tests pass: `prove t/Koha/Config.t` 2. Apply the rest of the patches and verify that tests still pass: `prove t/Koha/Config.t`. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com Blocks| |7170 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7170 [Bug 7170] Remove use of XML::Simple -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #120374|0 |1 is obsolete| | --- Comment #3 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 120414 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120414&action=edit Bug 28278: Add unit test for Koha::Config::read_from_file Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #120375|0 |1 is obsolete| | --- Comment #4 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 120415 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120415&action=edit Bug 28278: Improve $KOHA_CONF parsing speed by using XML::LibXML - Without the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 3.585 s ± 0.018 s [User: 3.531 s, System: 0.049 s] Range (min … max): 3.547 s … 3.612 s 10 runs - With the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 1.122 s ± 0.028 s [User: 1.104 s, System: 0.014 s] Range (min … max): 1.095 s … 1.189 s 10 runs Test plan: 1. Apply the first patch (the one with the unit tests) and make sure tests pass: `prove t/Koha/Config.t` 2. Apply the rest of the patches and verify that tests still pass: `prove t/Koha/Config.t`. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net Status|Needs Signoff |Signed Off --- Comment #5 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Works :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- How much is the gain then? 2.5s for 10000 runs? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #7 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Jonathan Druart from comment #6)
How much is the gain then? 2.5s for 10000 runs?
It's for 1000 runs, so in the end I should gain something like 2.5ms at startup. It's not much, but still better, right ? :) Note that this was tested on my laptop with a i5-6440HQ CPU @ 2.60GHz The gain should be greater on slower hardware (just a guess; please test it on different hardware) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #8 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Right, the "10 runs" confused me, but ofc it's an average. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #9 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Without patch: Time (mean ± σ): 6.293 s ± 0.243 s [User: 6.181 s, System: 0.090 s] Range (min … max): 6.012 s … 6.852 s 10 runs With patch: Time (mean ± σ): 2.006 s ± 0.050 s [User: 1.937 s, System: 0.058 s] Range (min … max): 1.939 s … 2.092 s 10 runs i5-6200U CPU @ 2.30GHz -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=26790 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #10 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Without patch Time (mean ± σ): 7.349 s ± 0.268 s [User: 6.451 s, System: 0.348 s] Range (min … max): 6.968 s … 7.775 s 10 runs With patch Time (mean ± σ): 2.377 s ± 0.104 s [User: 1.993 s, System: 0.148 s] Range (min … max): 2.266 s … 2.551 s 10 runs Run on a production virtual server. I'm happy with the result and the test is thorough... no regressions found. I'd love to see the subsequent bugs to split out the zebra config and possibly move away from XML.. but for now, even putting the performance improvement aside, this is a great step in the right direction for dropping our reliance on XML::Simple. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #120414|0 |1 is obsolete| | --- Comment #11 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 120455 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120455&action=edit Bug 28278: Add unit test for Koha::Config::read_from_file Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #120415|0 |1 is obsolete| | --- Comment #12 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 120456 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120456&action=edit Bug 28278: Improve $KOHA_CONF parsing speed by using XML::LibXML - Without the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 3.585 s ± 0.018 s [User: 3.531 s, System: 0.049 s] Range (min … max): 3.547 s … 3.612 s 10 runs - With the patch: $ hyperfine --warmup 1 \ 'perl -MKoha::Config -e "Koha::Config->read_from_file(\$ENV{KOHA_CONF}) for (1..1000)"' Time (mean ± σ): 1.122 s ± 0.028 s [User: 1.104 s, System: 0.014 s] Range (min … max): 1.095 s … 1.189 s 10 runs Test plan: 1. Apply the first patch (the one with the unit tests) and make sure tests pass: `prove t/Koha/Config.t` 2. Apply the rest of the patches and verify that tests still pass: `prove t/Koha/Config.t`. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #13 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 120666 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120666&action=edit Bug 28278: Uncomment and test access_dirs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |21.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 --- Comment #14 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Pushed to master for 21.05, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |28305 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28305 [Bug 28305] Remove doc reference to XML::Simple in C4::Config -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #15 from Fridolin Somers <fridolin.somers@biblibre.com> --- Enhancement not pushed to 20.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28385 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dpavlin@rot13.org --- Comment #16 from Dobrica Pavlinusic <dpavlin@rot13.org> --- This patch broke ldap configuration - fix is submitted under Bug 28385 - ldap server configuration broken since migration from XML::Simple -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |28385 See Also|https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28385 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28385 [Bug 28385] ldap server configuration broken since migration from XML::Simple -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |31393 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31393 [Bug 31393] Shibboleth broken since migration from XML::Simple -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28278 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #17 from David Cook <dcook@prosentient.com.au> --- Looks like this introduced a problem for Shibboleth config too but I have a patch on bug 31393 for that. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org