[Bug 25050] New: Elasticsearch - authority indexing depends on mapping order
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Bug ID: 25050 Summary: Elasticsearch - authority indexing depends on mapping order Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com There is an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 102346 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102346&action=edit Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 102347 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102347&action=edit Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 102348 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102348&action=edit Mappings that uncovered the bug You can test with this mapping file. Donwload and save the file. Alter your koha-conf.xml to point to this file. Reload your koha mappings. Reset all the things Reindex ES Check an auth record: curl -XGET 'es:9200/koha_kohadev_authorities/data/568?pretty' | grep -A 5 match-heading Note match heading contains the terms, but not a formatted heading string Apply patch Reindex ES Check an auth record You should now have a string with 'subdiv' terms e.g.: Authors, Irish chronologicalsubdiv 20th century formsubdiv Biography -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com | |, ere.maijala@helsinki.fi, | |martin.renvoize@ptfs-europe | |.com, | |mkstephens@lancasterseminar | |y.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #102346|0 |1 is obsolete| | --- Comment #4 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 102356 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102356&action=edit Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #102347|0 |1 is obsolete| | --- Comment #5 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 102357 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102357&action=edit Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #102356|0 |1 is obsolete| | Attachment #102357|0 |1 is obsolete| | --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102683 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102683&action=edit Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 --- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 102684 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=102684&action=edit Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com Version(s)|20.05.00 |20.05.00, 19.11.06 released in| | Status|Pushed to master |Pushed to stable --- Comment #9 from Joy Nelson <joy@bywatersolutions.com> --- backported to 19.11.x for 19.11.06 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25050 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #10 from Lucas Gass <lucas@bywatersolutions.com> --- doesnt apply clean to 19.05.x, if needed please rebase -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org