[Bug 24269] New: Authority matching in Elasticsearch is broken when authority has subdivisions
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Bug ID: 24269 Summary: Authority matching in Elasticsearch is broken when authority has subdivisions Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com In C4/Heading/MARC21.pm when we call _get_search_heading we look at the heading and turn some subfields into terms so: 150 $aChess$xOpenings becomes 'Chess generalsubdiv openings' 126 my %subdivisions = ( 127 'v' => 'formsubdiv', 128 'x' => 'generalsubdiv', 129 'y' => 'chronologicalsubdiv', 130 'z' => 'geographicsubdiv', 131 ); 267 $heading .= " $subdivisions{$code} $value"; We don't do this when indexing into ES so we will never find matches for these records -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alex.arnaud@biblibre.com, | |ere.maijala@helsinki.fi, | |glasklas@gmail.com, | |jonathan.druart@bugs.koha-c | |ommunity.org, | |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Myka Kennedy Stephens <mkstephens@lancasterseminary.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mkstephens@lancasterseminar | |y.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 97357 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97357&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #97357|0 |1 is obsolete| | --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 97981 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97981&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags To test: 1 - Be using Elasticsearch 2 - Check that field 150 is mapped to 'Match-heading' or add it (the subfields don't matter) 3 - Add a topic term authority record like "150 $aCats$vFiction" 4 - Add a 650 with $aCats and $vFiction and $e depicted to a bibliographic record 5 - Run the linker for the bib perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v 6 - Confirm the record is not correctly linked to the record 7 - Apply patch 8 - Reindex authorities for ES perl misc/search_tools/rebuild_elasticsearch.pl -v -d -a 9 - Run the linker and confirm record is correctly linked perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |24421 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24421 [Bug 24421] Generated authorities are missing subfields -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #97981|0 |1 is obsolete| | --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 97982 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97982&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags To test: 1 - Be using Elasticsearch 2 - Check that field 150 is mapped to 'Match-heading' or add it (the subfields don't matter) 3 - Add a topic term authority record like "150 $aCats$vFiction" 4 - Add a 650 with $aCats and $vFiction and $e depicted to a bibliographic record 5 - Run the linker for the bib perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v 6 - Confirm the record is not correctly linked to the record 7 - Apply patch 8 - Reindex authorities for ES perl misc/search_tools/rebuild_elasticsearch.pl -v -d -a 9 - Run the linker and confirm record is correctly linked perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Comment on attachment 97982 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97982 Bug 24269: Adjust C4::Heading to generate headings from auth tags Review of attachment 97982: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=24269&attachment=97982) ----------------------------------------------------------------- ::: C4/Heading.pm @@ +52,5 @@
Given a C<MARC::Field> object containing a heading from a bib record, create a C<C4::Heading> object.
+The optional third parameter is 'auth' - it is handled as boolean. If supplied we treat the field as an auth record field. Otherwise if it is a bib field. The fields checked are the same in a UNIMARC system and this parameter is ignored
I'm not sure I understand this rewording.. you've removed the '$marc_flavour' param right, and instead, always use the application context? I think this could be more clearly worded. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #97982|0 |1 is obsolete| | --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 97996 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97996&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags To test: 1 - Be using Elasticsearch 2 - Check that field 150 is mapped to 'Match-heading' or add it (the subfields don't matter) 3 - Add a topic term authority record like "150 $aCats$vFiction" 4 - Add a 650 with $aCats and $vFiction and $e depicted to a bibliographic record 5 - Run the linker for the bib perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v 6 - Confirm the record is not correctly linked to the record 7 - Apply patch 8 - Reindex authorities for ES perl misc/search_tools/rebuild_elasticsearch.pl -v -d -a 9 - Run the linker and confirm record is correctly linked perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Signing off as this logically all makes sense, the tests pass and it passes it's test plan. Minor comment about rewording the updated POD slightly as it's not entirely clear without reading the code. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Martin Renvoize from comment #4)
Comment on attachment 97982 [details] [review] Bug 24269: Adjust C4::Heading to generate headings from auth tags
Review of attachment 97982 [details] [review]: -----------------------------------------------------------------
::: C4/Heading.pm @@ +52,5 @@
Given a C<MARC::Field> object containing a heading from a bib record, create a C<C4::Heading> object.
+The optional third parameter is 'auth' - it is handled as boolean. If supplied we treat the field as an auth record field. Otherwise if it is a bib field. The fields checked are the same in a UNIMARC system and this parameter is ignored
I'm not sure I understand this rewording.. you've removed the '$marc_flavour' param right, and instead, always use the application context? I think this could be more clearly worded.
I did, I didn't mention it in the POD, meant to mention in the commit message - we had that option, but it was never used and I didn't see a reason to preserve it -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Bouzid Fergani <bouzid.fergani@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #97996|0 |1 is obsolete| | --- Comment #8 from Bouzid Fergani <bouzid.fergani@inlibro.com> --- Created attachment 99039 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99039&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags To test: 1 - Be using Elasticsearch 2 - Check that field 150 is mapped to 'Match-heading' or add it (the subfields don't matter) 3 - Add a topic term authority record like "150 $aCats$vFiction" 4 - Add a 650 with $aCats and $vFiction and $e depicted to a bibliographic record 5 - Run the linker for the bib perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v 6 - Confirm the record is not correctly linked to the record 7 - Apply patch 8 - Reindex authorities for ES perl misc/search_tools/rebuild_elasticsearch.pl -v -d -a 9 - Run the linker and confirm record is correctly linked perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Bouzid Fergani <bouzid.fergani@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bouzid.fergani@inlibro.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- We now have a double signoff on this including my own.. I'll do a quick final check before pushing but I think we can safely say this is PQA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=24094 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #99039|0 |1 is obsolete| | --- Comment #10 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 99246 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99246&action=edit Bug 24269: Adjust C4::Heading to generate headings from auth tags To test: 1 - Be using Elasticsearch 2 - Check that field 150 is mapped to 'Match-heading' or add it (the subfields don't matter) 3 - Add a topic term authority record like "150 $aCats$vFiction" 4 - Add a 650 with $aCats and $vFiction and $e depicted to a bibliographic record 5 - Run the linker for the bib perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v 6 - Confirm the record is not correctly linked to the record 7 - Apply patch 8 - Reindex authorities for ES perl misc/search_tools/rebuild_elasticsearch.pl -v -d -a 9 - Run the linker and confirm record is correctly linked perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=89" -v Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 --- Comment #11 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 AspenCat Team <aspencatteam@clicweb.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aspencatteam@clicweb.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Heather <heather_hernandez@nps.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |heather_hernandez@nps.gov -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|20.05.00 |20.05.00, 19.11.04 released in| | CC| |joy@bywatersolutions.com --- Comment #12 from Joy Nelson <joy@bywatersolutions.com> --- Pushed to 19.11.x for 19.11.04 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24269 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |25273 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25273 [Bug 25273] Elasticsearch Authority matching is returning too many results -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org