[Bug 16506] New: rebuild_zebra.pl still using USMARC as default
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Bug ID: 16506 Summary: rebuild_zebra.pl still using USMARC as default Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: gmcharlt@gmail.com Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org The current rebuild_zebra.pl still uses usmarc as serialization format. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |16505 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16505 [Bug 16505] rebuild_zebra.pl skips updates if -x is passed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506
--- Comment #1 from Tomás Cohen Arazi <tomascohen@gmail.com> ---
Created attachment 51537
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51537&action=edit
Bug 16506: Make rebuild_zebra.pl use XML as default
This patch deprecates the -x switch, making XML the default serialization
format
used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a
warning
for the end user about the deprecation so they fix their cronjobs. Later we
could remove it.
To test:
- Disable all indexing (daemon/cronjob)
- Create 2 records
- Edit one of them, delete the other one
- Verify they are queued for updates in zebraqueue
- sudo koha-mysql kohadev
> SELECT * FROM zebraqueue WHERE done=0
...
| 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13
14:23:45 |
| 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16
14:14:33 |
| 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16
14:15:06 |
+-----+--------------------+---------------+--------------+------+---------------------+
- Now go to koha-shell
$ sudo koha-shell kohadev ; cd kohaclone
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> FAIL: They contain the records you added/modified/deleted but they are in
USMARC format
- Apply the patch
- Mark your records for indexing (in koha-mysql kohadev)
> UPDATE zebraqueue SET done=0 WHERE id > 264
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in XML format
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in USMARC format
- Sign off :-D
[1] Warnings covered by a followup
--
You are receiving this mail because:
You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 51538 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51538&action=edit Bug 16506: (Followup) remove warnings -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |abl@biblos.pk.edu.pl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 51552 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51552&action=edit Bug 16506: (followup) Fix wrong option switch warning message -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506
Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #51537|0 |1
is obsolete| |
--- Comment #4 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> ---
Created attachment 51566
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51566&action=edit
[SIGNED-OFF] Bug 16506: Make rebuild_zebra.pl use XML as default
This patch deprecates the -x switch, making XML the default serialization
format
used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a
warning
for the end user about the deprecation so they fix their cronjobs. Later we
could remove it.
To test:
- Disable all indexing (daemon/cronjob)
- Create 2 records
- Edit one of them, delete the other one
- Verify they are queued for updates in zebraqueue
- sudo koha-mysql kohadev
> SELECT * FROM zebraqueue WHERE done=0
...
| 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13
14:23:45 |
| 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16
14:14:33 |
| 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16
14:15:06 |
+-----+--------------------+---------------+--------------+------+---------------------+
- Now go to koha-shell
$ sudo koha-shell kohadev ; cd kohaclone
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> FAIL: They contain the records you added/modified/deleted but they are in
USMARC format
- Apply the patch
- Mark your records for indexing (in koha-mysql kohadev)
> UPDATE zebraqueue SET done=0 WHERE id > 264
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in XML format
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in USMARC format
- Sign off :-D
[1] Warnings covered by a followup
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
On top of Bug 16505
Work as described following test plan, usmarc default pre patch,
post patch xml default and usmarc on request.
No errors (all patchset)
--
You are receiving this mail because:
You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #51538|0 |1 is obsolete| | --- Comment #5 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 51567 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51567&action=edit [SIGNED-OFF] Bug 16506: (Followup) remove warnings Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #51552|0 |1 is obsolete| | --- Comment #6 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 51568 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51568&action=edit [SIGNED-OFF] Bug 16506: (followup) Fix wrong option switch warning message Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bgkriegel@gmail.com Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |12641 CC| |jonathan.druart@bugs.koha-c | |ommunity.org Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12641 [Bug 12641] Formally deprecate the GRS-1 indexing mode -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506
Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #51566|0 |1
is obsolete| |
Attachment #51567|0 |1
is obsolete| |
Attachment #51568|0 |1
is obsolete| |
--- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> ---
Created attachment 51675
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51675&action=edit
Bug 16506: Make rebuild_zebra.pl use XML as default
This patch deprecates the -x switch, making XML the default serialization
format
used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a
warning
for the end user about the deprecation so they fix their cronjobs. Later we
could remove it.
To test:
- Disable all indexing (daemon/cronjob)
- Create 2 records
- Edit one of them, delete the other one
- Verify they are queued for updates in zebraqueue
- sudo koha-mysql kohadev
> SELECT * FROM zebraqueue WHERE done=0
...
| 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13
14:23:45 |
| 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16
14:14:33 |
| 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16
14:15:06 |
+-----+--------------------+---------------+--------------+------+---------------------+
- Now go to koha-shell
$ sudo koha-shell kohadev ; cd kohaclone
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> FAIL: They contain the records you added/modified/deleted but they are in
USMARC format
- Apply the patch
- Mark your records for indexing (in koha-mysql kohadev)
> UPDATE zebraqueue SET done=0 WHERE id > 264
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in XML format
- Run:
$ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml
You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
* less /tmp/jI0OeHy6Tn/del_biblio/exported_records
* less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in USMARC format
- Sign off :-D
[1] Warnings covered by a followup
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
On top of Bug 16505
Work as described following test plan, usmarc default pre patch,
post patch xml default and usmarc on request.
No errors (all patchset)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
--
You are receiving this mail because:
You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #8 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 51676 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51676&action=edit Bug 16506: (Followup) remove warnings Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 51677 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51677&action=edit Bug 16506: (followup) Fix wrong option switch warning message Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 51678 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51678&action=edit Bug 16506: Remove warning for UNIMARC installs Use of uninitialized value in numeric eq (==) Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |kyle@bywatersolutions.com --- Comment #11 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to master for Koha 16.05. Thanks Tomas, Jonathan! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com --- Comment #12 from Julian Maurice <julian.maurice@biblibre.com> --- It doesn't look like a bugfix (severity = normal), can someone confirm this is an enhancement ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Julian Maurice from comment #12)
It doesn't look like a bugfix (severity = normal), can someone confirm this is an enhancement ?
Well, it might be. Because bug 11096 introduced XML as exchange format for DOM setups, to allow bigger records to get indexed, and we then deprecated GRS-1. To me this is something we just forgot to tweak after 11096. But it is really up to you to include it in 3.22 as it supposes a behaviour change for people using the cronjob to update their indexes. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16506 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |16830 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16830 [Bug 16830] koha-indexer still uses the deprecated -x option switch -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org