[Bug 27784] New: Unknown authority types break elasticsearch authorities indexing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Bug ID: 27784 Summary: Unknown authority types break elasticsearch authorities indexing Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com We are seeing more libraries ending up with 'SUBDIV' type authority records in their systems. Koha has no support for these. They can be added to the authority types in the staff interface, however, values are hardcoded in various modules and Koha has no concept of how to link a subfield heading to a record, we only deal in whole fields The larger problem is something to think about, but more directly we should not allow indexing to die when an unknown type is encountered To recreate: 1 - Import a subdivision authority record via Z39 or use the one attached to this bug 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -d 3 - Authority indexing dies: Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202. Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206. Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207. Can't call method "tag" on an undefined value at /kohadevbox/koha/C4/Heading.pm line 71. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 117378 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117378&action=edit Bug 27784: Unit tests -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 117379 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117379&action=edit Bug 27784: Don't parse subdivision authorities as headings Subdivision authorities are not used for linking, however, they are recognized by C4::AuthoritiesMarc While these records are not used for linking, they could provide reference and should be allowed to exist in the catalog without breaking ES indexing THis patch simply skips the step of parsing the authorities into the linking form if the type contains '_SUBD' To test: 1 - Import a subdivision authority record via Z39 or use the one attached to this bug 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -d 3 - Authority indexing dies: Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202. Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206. Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207. Can't call method "tag" on an undefined value at /kohadevbox/koha/C4/Heading.pm line 71. 4 - Apply patches 5 - reindex 6 - Success! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 117401 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117401&action=edit Sample record -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #117378|0 |1 is obsolete| | --- Comment #4 from David Nind <david@davidnind.com> --- Created attachment 117459 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117459&action=edit Bug 27784: Unit tests Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #117379|0 |1 is obsolete| | --- Comment #5 from David Nind <david@davidnind.com> --- Created attachment 117460 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117460&action=edit Bug 27784: Don't parse subdivision authorities as headings Subdivision authorities are not used for linking, however, they are recognized by C4::AuthoritiesMarc While these records are not used for linking, they could provide reference and should be allowed to exist in the catalog without breaking ES indexing THis patch simply skips the step of parsing the authorities into the linking form if the type contains '_SUBD' To test: 1 - Import a subdivision authority record via Z39 or use the one attached to this bug 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -d 3 - Authority indexing dies: Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202. Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206. Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207. Can't call method "tag" on an undefined value at /kohadevbox/koha/C4/Heading.pm line 71. 4 - Apply patches 5 - reindex 6 - Success! Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #6 from David Nind <david@davidnind.com> --- Thanks Nick for providing a sample record! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl, | |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #117459|0 |1 is obsolete| | --- Comment #7 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 117563 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117563&action=edit Bug 27784: Unit tests Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #117460|0 |1 is obsolete| | --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 117564 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117564&action=edit Bug 27784: Don't parse subdivision authorities as headings Subdivision authorities are not used for linking, however, they are recognized by C4::AuthoritiesMarc While these records are not used for linking, they could provide reference and should be allowed to exist in the catalog without breaking ES indexing THis patch simply skips the step of parsing the authorities into the linking form if the type contains '_SUBD' To test: 1 - Import a subdivision authority record via Z39 or use the one attached to this bug 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -d 3 - Authority indexing dies: Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202. Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206. Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207. Can't call method "tag" on an undefined value at /kohadevbox/koha/C4/Heading.pm line 71. 4 - Apply patches 5 - reindex 6 - Success! Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- A reasonable and pragmatic solution to the issue. Whilst it would be nice to add support for subdivision linking, this patch is a pragmatic solution to prevent indexing explosions. It works as expected, has a unit regression test and passes the QA script. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |21.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 21.05, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|21.05.00 |21.05.00,20.11.04 released in| | CC| |fridolin.somers@biblibre.co | |m --- Comment #11 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 20.11.x for 20.11.04 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|21.05.00,20.11.04 |21.05.00,20.11.04,20.05.10 released in| | Status|Pushed to stable |Pushed to oldstable --- Comment #12 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Pushed to 20.05.x for 20.05.10 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|21.05.00,20.11.04,20.05.10 |21.05.00,20.11.04,20.05.10, released in| |19.11.16 Status|Pushed to oldstable |Pushed to oldoldstable CC| |victor@tuxayo.net --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported: Pushed to 19.11.x branch for 19.11.16 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This patch fixes release notes| |Elasticsearch indexing | |failures caused by 'SUBDIV' | |type authority records in | |Koha. It skips the step of | |parsing the authorities | |into the linking form if | |the type contains '_SUBD'. | | | |Notes: | |- Koha currently | |doesn't have support for | |'SUBDIV' type authority | |records. | |- They can be | |added to the authority | |types in the staff | |interface, however, values | |are hard coded in various | |modules and Koha has no | |concept of how to link a | |subfield heading to a | |record, as we only deal in | |whole fields. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27784 jmbroust <jean-manuel.broust@univ-lyon2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jean-manuel.broust@univ-lyo | |n2.fr -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org