[Bug 20384] New: Elastic rebuild script improvements - options for indexing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Bug ID: 20384 Summary: Elastic rebuild script improvements - options for indexing Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: claire.hernandez@biblibre.com I would be glad to use the rebuild script with some more options like: -n 100 index 100 first records -n "100,2" index 2 records after 100th (101 and 102) --from XXX index biblio with biblionumber >= XXX --to YYY index biblio with biblionumber <= YYY bonus: -i gives elastic install information: syspref, numbers of biblios/auth indexed, name of indice used etc. There is maybe an other way to name the options. Advice or opinion or objection on how to do this ? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 dominique.rouger@univ-st-etienne.fr changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dominique.rouger@univ-st-et | |ienne.fr --- Comment #1 from dominique.rouger@univ-st-etienne.fr --- It could also be useful to have an option to launch an indexing by decrementing the record number -w "100,2" index 2 biblio with biblionumber after 100th (101 and 102) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #2 from dominique.rouger@univ-st-etienne.fr --- It could also be useful to have an option to launch an indexing by decrementing the record number -w "100,2" index 2 biblio with biblionumber after 100th (101 and 102) and -w "100,-2" index 2 biblio with biblionumber before 100th (99 and 98) This allows the highest biblionumbers, i.e. the most recent records, to be indexed first. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #3 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- I'd say we should use same args as misc/migration_tools/rebuild_zebra.pl : range of records : --length 1234 how many biblio you want to export --offset 1243 offset you want to start to a where limit : --where let you specify a WHERE query, like itemtype='BOOK' or something like that --table specify a table (can be items, biblioitems, biblio, biblio_metadata) to retrieve biblionumber to index. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #4 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- something whould be awesome : limit on biblionumbers from a file. allow to rebuild from a report results. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |axel.amghar@biblibre.com |ity.org | CC| |axel.amghar@biblibre.com Status|NEW |ASSIGNED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #5 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89425 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89425&action=edit Bug 20384 - Elastic rebuild script improvements - options --offset & --length This patch add the options --offset & --length to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89425|0 |1 is obsolete| | --- Comment #6 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89559 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89559&action=edit Bug 20384 - Elastic rebuild script improvements - options --offset & --length This patch add the options --offset & --length to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #7 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89560 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89560&action=edit Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #8 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89727 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89727&action=edit Bug 20384 - Elastic rebuild script improvements : option --file To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #9 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89886 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89886&action=edit Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89560|0 |1 is obsolete| | --- Comment #10 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89887 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89887&action=edit Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89727|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89559|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89886|0 |1 is obsolete| | --- Comment #11 from axel Amghar <axel.amghar@biblibre.com> --- Created attachment 89888 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=89888&action=edit Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 axel Amghar <axel.amghar@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elastic rebuild script |Elasticsearch rebuild |improvements - options for |script improvements - |indexing |options for indexing -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=22826 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply CC| |bgkriegel@gmail.com --- Comment #12 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Please rebase, does not apply. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #89887|0 |1 is obsolete| | Attachment #89888|0 |1 is obsolete| | --- Comment #13 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 106587 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106587&action=edit Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 --- Comment #14 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 106588 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106588&action=edit Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20384 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Status|Needs Signoff |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org