[Bug 21865] New: improve remove_unused_authorities.pl script
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bug ID: 21865 Summary: improve remove_unused_authorities.pl script Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: fridolin.somers@biblibre.com QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz remove_unused_authorities.pl script could be improved. At least : test mode more verbose, ES compatibility, add confirm arg -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=20377 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |fridolin.somers@biblibre.co |ity.org |m Status|NEW |ASSIGNED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #1 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 82558 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82558&action=edit Bug 21865: improve remove_unused_authorities.pl script remove_unused_authorities.pl script can be improved. This patch changes changes verbosity so than test mode can be used to know the autorities that are used and those that can be deleted. It also writes a line in output if limited authority type(s). This patch also removes the unused vars $thresholdmin and $thresholdmax. It also changes the query to use SQL with parameters for authority types. Test plan : 1) On a catalog create a new authority 2) Be sure catalog is well indexed 3) Run misc/migration_tools/remove_unused_authorities.pl -t 4) You will see the line : *** Testing only, authorities will not be deleted. *** 5) You will see lines of : authid=x type=y : used X time(s) 6) You will see the line for the authority created in 1) : authid=x type=y : can be deleted 7) You will see at the end : x authorities parsed y can be deleted because unused z unchanged because used 8) Run misc/migration_tools/remove_unused_authorities.pl 9) You don't see the line : *** Testing only, authorities will not be deleted. *** 10) You will see lines of : authid=x type=y : used X time(s) 11) You will see the line for the authority created in 1) : authid=x type=y : deleted 12) You will see at the end : x authorities parsed y deleted because unused z unchanged because used 13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO 14) You see the line : Restricted to authority type(s) : NP,CO. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #2 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 82559 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82559&action=edit Bug 21865: ES compatiblity of remove_unused_authorities.pl The script misc/migration_tools/remove_unused_authorities.pl directly checks if Zebra search is OK. This patch changes so that this test is only if Zebra is the search engine. It also adds a test on the search off any authority number indexed (index 'an'). With Zebra its : an,alwaysmatches='' With ES its : an:* This test ensure that biblios records are indexed and that not all autorities will be deleted. Test plan: 1) On a catalog create a new authority 2) Use Zebra in systempreference SearchEngine 3) Stop Zebra server 4) Run misc/migration_tools/remove_unused_authorities.pl -c 5) The script does nothing and says : Zebra server seems not to be available. This script needs Zebra runs. 6) Restart Zebra server 7) Delete biblio index base 8) Run misc/migration_tools/remove_unused_authorities.pl -c 9) The script does nothing and says : Searching authority number in biblio records seems not to be available : an,alwaysmatches='' 10) Use ElasticSearch in systempreference SearchEngine 11) Delete biblio index base 12) Run misc/migration_tools/remove_unused_authorities.pl -c 13) The script does nothing and says : Searching authority number in biblio records seems not to be available : an:* -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #3 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 82560 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82560&action=edit Bug 21865: add confirm to remove_unused_authorities.pl script Like in most scripts in misc, add confirm argument to ensure script is not run without knowing what it does. Test plan: 1) Run misc/migration_tools/remove_unused_authorities.pl -h 2) You see help line for confirm 3) Run misc/migration_tools/remove_unused_authorities.pl 4) You see help and script does nothing 5) Run misc/migration_tools/remove_unused_authorities.pl -c 6) Script runs like wanted -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bgkriegel@gmail.com --- Comment #4 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Hi, with this code 40 if ( $want_help || !$confirm ) { you need to confirm even to do a test. Is that the intended behavior? May be "if ( $want_help || !($test || $confirm) ) {" Otherwise the script works well (tested using zebra) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #5 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #4)
Hi, with this code
40 if ( $want_help || !$confirm ) {
you need to confirm even to do a test. Is that the intended behavior?
May be "if ( $want_help || !($test || $confirm) ) {"
Otherwise the script works well (tested using zebra)
Indeed, good idea -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #6 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 100577 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100577&action=edit Bug 21865: allow test arg alone -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82558|0 |1 is obsolete| | --- Comment #7 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 100918 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100918&action=edit Bug 21865: improve remove_unused_authorities.pl script remove_unused_authorities.pl script can be improved. This patch changes changes verbosity so than test mode can be used to know the autorities that are used and those that can be deleted. It also writes a line in output if limited authority type(s). This patch also removes the unused vars $thresholdmin and $thresholdmax. It also changes the query to use SQL with parameters for authority types. Test plan : 1) On a catalog create a new authority 2) Be sure catalog is well indexed 3) Run misc/migration_tools/remove_unused_authorities.pl -t 4) You will see the line : *** Testing only, authorities will not be deleted. *** 5) You will see lines of : authid=x type=y : used X time(s) 6) You will see the line for the authority created in 1) : authid=x type=y : can be deleted 7) You will see at the end : x authorities parsed y can be deleted because unused z unchanged because used 8) Run misc/migration_tools/remove_unused_authorities.pl 9) You don't see the line : *** Testing only, authorities will not be deleted. *** 10) You will see lines of : authid=x type=y : used X time(s) 11) You will see the line for the authority created in 1) : authid=x type=y : deleted 12) You will see at the end : x authorities parsed y deleted because unused z unchanged because used 13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO 14) You see the line : Restricted to authority type(s) : NP,CO. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82559|0 |1 is obsolete| | --- Comment #8 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 100919 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100919&action=edit Bug 21865: ES compatiblity of remove_unused_authorities.pl The script misc/migration_tools/remove_unused_authorities.pl directly checks if Zebra search is OK. This patch changes so that this test is only if Zebra is the search engine. It also adds a test on the search off any authority number indexed (index 'an'). With Zebra its : an,alwaysmatches='' With ES its : an:* This test ensure that biblios records are indexed and that not all autorities will be deleted. Test plan: 1) On a catalog create a new authority 2) Use Zebra in systempreference SearchEngine 3) Stop Zebra server 4) Run misc/migration_tools/remove_unused_authorities.pl -c 5) The script does nothing and says : Zebra server seems not to be available. This script needs Zebra runs. 6) Restart Zebra server 7) Delete biblio index base 8) Run misc/migration_tools/remove_unused_authorities.pl -c 9) The script does nothing and says : Searching authority number in biblio records seems not to be available : an,alwaysmatches='' 10) Use ElasticSearch in systempreference SearchEngine 11) Delete biblio index base 12) Run misc/migration_tools/remove_unused_authorities.pl -c 13) The script does nothing and says : Searching authority number in biblio records seems not to be available : an:* Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82560|0 |1 is obsolete| | --- Comment #9 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 100920 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100920&action=edit Bug 21865: add confirm to remove_unused_authorities.pl script Like in most scripts in misc, add confirm argument to ensure script is not run without knowing what it does. Test plan: 1) Run misc/migration_tools/remove_unused_authorities.pl -h 2) You see help line for confirm 3) Run misc/migration_tools/remove_unused_authorities.pl 4) You see help and script does nothing 5) Run misc/migration_tools/remove_unused_authorities.pl -c 6) Script runs like wanted Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #100577|0 |1 is obsolete| | --- Comment #10 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 100921 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100921&action=edit Bug 21865: allow test arg alone Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #11 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Tested with zebra, no errors. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #12 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #11)
Tested with zebra, no errors.
And tested with Elasticsearch, no errors -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #100918|0 |1 is obsolete| | Attachment #100919|0 |1 is obsolete| | Attachment #100920|0 |1 is obsolete| | Attachment #100921|0 |1 is obsolete| | --- Comment #13 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102836 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102836&action=edit Bug 21865: improve remove_unused_authorities.pl script remove_unused_authorities.pl script can be improved. This patch changes changes verbosity so than test mode can be used to know the autorities that are used and those that can be deleted. It also writes a line in output if limited authority type(s). This patch also removes the unused vars $thresholdmin and $thresholdmax. It also changes the query to use SQL with parameters for authority types. Test plan : 1) On a catalog create a new authority 2) Be sure catalog is well indexed 3) Run misc/migration_tools/remove_unused_authorities.pl -t 4) You will see the line : *** Testing only, authorities will not be deleted. *** 5) You will see lines of : authid=x type=y : used X time(s) 6) You will see the line for the authority created in 1) : authid=x type=y : can be deleted 7) You will see at the end : x authorities parsed y can be deleted because unused z unchanged because used 8) Run misc/migration_tools/remove_unused_authorities.pl 9) You don't see the line : *** Testing only, authorities will not be deleted. *** 10) You will see lines of : authid=x type=y : used X time(s) 11) You will see the line for the authority created in 1) : authid=x type=y : deleted 12) You will see at the end : x authorities parsed y deleted because unused z unchanged because used 13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO 14) You see the line : Restricted to authority type(s) : NP,CO. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #14 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102837 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102837&action=edit Bug 21865: ES compatiblity of remove_unused_authorities.pl The script misc/migration_tools/remove_unused_authorities.pl directly checks if Zebra search is OK. This patch changes so that this test is only if Zebra is the search engine. It also adds a test on the search off any authority number indexed (index 'an'). With Zebra its : an,alwaysmatches='' With ES its : an:* This test ensure that biblios records are indexed and that not all autorities will be deleted. Test plan: 1) On a catalog create a new authority 2) Use Zebra in systempreference SearchEngine 3) Stop Zebra server 4) Run misc/migration_tools/remove_unused_authorities.pl -c 5) The script does nothing and says : Zebra server seems not to be available. This script needs Zebra runs. 6) Restart Zebra server 7) Delete biblio index base 8) Run misc/migration_tools/remove_unused_authorities.pl -c 9) The script does nothing and says : Searching authority number in biblio records seems not to be available : an,alwaysmatches='' 10) Use ElasticSearch in systempreference SearchEngine 11) Delete biblio index base 12) Run misc/migration_tools/remove_unused_authorities.pl -c 13) The script does nothing and says : Searching authority number in biblio records seems not to be available : an:* Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #15 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102838 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102838&action=edit Bug 21865: add confirm to remove_unused_authorities.pl script Like in most scripts in misc, add confirm argument to ensure script is not run without knowing what it does. Test plan: 1) Run misc/migration_tools/remove_unused_authorities.pl -h 2) You see help line for confirm 3) Run misc/migration_tools/remove_unused_authorities.pl 4) You see help and script does nothing 5) Run misc/migration_tools/remove_unused_authorities.pl -c 6) Script runs like wanted Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #16 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102839 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102839&action=edit Bug 21865: allow test arg alone Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|improve |Improve |remove_unused_authorities.p |remove_unused_authorities.p |l script |l script -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.05.00 |20.05.00, 19.11.06 released in| | CC| |joy@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #18 from Joy Nelson <joy@bywatersolutions.com> --- backported to 19.11.x for 19.11.06 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Summary|Improve |Add elasticesarch support |remove_unused_authorities.p |to, and improve verbose |l script |output of, | |`remove_unused_authorities. | |pl` -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add elasticesarch support |Add Elasticsarch support |to, and improve verbose |to, and improve verbose |output of, |output of, |`remove_unused_authorities. |`remove_unused_authorities. |pl` |pl` -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21865 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add Elasticsarch support |Add Elasticsearch support |to, and improve verbose |to, and improve verbose |output of, |output of, |`remove_unused_authorities. |`remove_unused_authorities. |pl` |pl` -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org