[Bug 20377] New: Various speed tweaks to 'remove_unused_authorities.pl' script
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Bug ID: 20377 Summary: Various speed tweaks to 'remove_unused_authorities.pl' script Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: mtj@kohaaloha.com QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz Here is a patch to make 'remove_unused_authorities.pl' a bit faster i get a 5x speedup after this patch fyi: i pinched most of this patch from bulkmarcimport.pl 1/ import a bunch of authority record into a Koha, and index 2/ run script, note execution time $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m50s 3/ apply patch 4/ run script again, and notice it's a bit faster $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m10s -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 --- Comment #1 from Mason James <mtj@kohaaloha.com> --- Created attachment 72727 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=72727&action=edit Bug 20377: Various speed tweaks to 'remove_unused_authorities.pl' script Here is a patch to make 'remove_unused_authorities.pl' a bit faster i get a 5x speedup after this patch fyi: i pinched most of this patch from bulkmarcimport.pl 1/ import a bunch of authority record into a Koha, and index 2/ run script, note execution time $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m50s 3/ apply patch 4/ run script again, and notice it's a bit faster $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m10s -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Brendan Gallagher <brendan@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Brendan Gallagher <brendan@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #72727|0 |1 is obsolete| | --- Comment #2 from Brendan Gallagher <brendan@bywatersolutions.com> --- Created attachment 72775 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=72775&action=edit Bug 20377: Various speed tweaks to 'remove_unused_authorities.pl' script Here is a patch to make 'remove_unused_authorities.pl' a bit faster i get a 5x speedup after this patch fyi: i pinched most of this patch from bulkmarcimport.pl 1/ import a bunch of authority record into a Koha, and index 2/ run script, note execution time $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m50s 3/ apply patch 4/ run script again, and notice it's a bit faster $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m10s Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | Patch complexity|--- |Small patch Assignee|koha-bugs@lists.koha-commun |mtj@kohaaloha.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Note: My first impression about your use of $commit was that we should not use it that way (I would interpret it as a confirm flag like in other cron jobs), but bulkmarcimport is using it like you do here.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #72775|0 |1 is obsolete| | --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 73166 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=73166&action=edit Bug 20377: Various speed tweaks to 'remove_unused_authorities.pl' script Here is a patch to make 'remove_unused_authorities.pl' a bit faster i get a 5x speedup after this patch fyi: i pinched most of this patch from bulkmarcimport.pl 1/ import a bunch of authority record into a Koha, and index 2/ run script, note execution time $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m50s 3/ apply patch 4/ run script again, and notice it's a bit faster $ time perl ./misc/migration_tools/remove_unused_authorities.pl real 0m10s Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Combining -test and -commit is kind of confusing but could be considered outside the scope? Would welcome a confirm flag instead of having to add -test. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA CC| |jonathan.druart@bugs.koha-c | |ommunity.org --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- - Please improve bug title and commit messages (there are no various tweaks, just using a transaction) - specify default value in POD - maybe we should remove 'c' to avoid the ambiguity with confirm? - transaction must be done using $schema->txn_do -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 --- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- + DelAuthority({ authid => $data->{'authid'}, skip_merge => 1 }) unless $test; Checking $test must be done on the commit statements -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Various speed tweaks to |Add transaction and |'remove_unused_authorities. |skip_merge to |pl' script |'remove_unused_authorities. | |pl' script -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bob@calyx.net.au, | |dilan@calyx.net.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20377 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=21865 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org