[Bug 23630] New: Elasticsearch indexing is removing field 999
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Bug ID: 23630 Summary: Elasticsearch indexing is removing field 999 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: fridolin.somers@biblibre.com Elasticsearch indexing uses 999$c to store record id by deleting the all field first ! So you can not store anything in field 999, even in UNIMARC and even in authorities records. Looks like it is quick fix code added to start Elasticsearch use. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |fridolin.somers@biblibre.co |ity.org |m -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 --- Comment #1 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 92971 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92971&action=edit Bug 23630: Do not remove field 999 in Elasticsearch indexing Elasticsearch indexing uses 999$c to store record id by deleting the all field first ! So you can not store anything in field 999, even in UNIMARC and even in authorities records. Looks like it is quick fix code added to start Elasticsearch use. This behavior is disturbing and very strange for UNIMARC flavour. This patch corrects by defining record ids mandatory in Koha::SearchEngine::Elasticsearch::Indexer::update_index(). This ids array is actually always given (except in UT). I think it is useless to allow adding a record without its id. Test plan : 1) Use Elasticsearch as SearchEngine 2) Create a subfield 999$z in default framework 3) Create a record with default framework 4) Enter a random string (never used in catalog) like "tototata" in 999$z 5) In Search engine configuration, define search field "subject" for 999$z 6) Rebuild record : misc/search_tools/rebuild_elasticsearch.pl -b -bn <biblionumber> -v 7) Search for the random string => You get a result 8) Optionnaly look at records in ES : <es server>:9200/<es index name>/data/<biblionumber> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |matthias.meusburger@biblibr | |e.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |In release notes| |Koha::SearchEngine::Elastic | |search::Indexer::update_ind | |ex() first arg record ids | |is now mandatory -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ere.maijala@helsinki.fi, | |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |major -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #92971|0 |1 is obsolete| | --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 93242 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93242&action=edit Bug 23630: Do not remove field 999 in Elasticsearch indexing Elasticsearch indexing uses 999$c to store record id by deleting the all field first ! So you can not store anything in field 999, even in UNIMARC and even in authorities records. Looks like it is quick fix code added to start Elasticsearch use. This behavior is disturbing and very strange for UNIMARC flavour. This patch corrects by defining record ids mandatory in Koha::SearchEngine::Elasticsearch::Indexer::update_index(). This ids array is actually always given (except in UT). I think it is useless to allow adding a record without its id. Test plan : 1) Use Elasticsearch as SearchEngine 2) Create a subfield 999$z in default framework 3) Create a record with default framework 4) Enter a random string (never used in catalog) like "tototata" in 999$z 5) In Search engine configuration, define search field "subject" for 999$z 6) Rebuild record : misc/search_tools/rebuild_elasticsearch.pl -b -bn <biblionumber> -v 7) Search for the random string => You get a result 8) Optionnaly look at records in ES : <es server>:9200/<es index name>/data/<biblionumber> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Following the test plan worked, additionally I forced the record to have a 999c and 999d that did not match the biblionumber - the only issue I found was that the links were odd in the opac, however, they still worked, If someone has this issue with their records, it is something they should fix, and we shouldn't clear the entire field for a rare occurrence. Editing the record in Koha basic editor strips the 999c and 999d as it is. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93242|0 |1 is obsolete| | --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 94823 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94823&action=edit Bug 23630: Do not remove field 999 in Elasticsearch indexing Elasticsearch indexing uses 999$c to store record id by deleting the all field first ! So you can not store anything in field 999, even in UNIMARC and even in authorities records. Looks like it is quick fix code added to start Elasticsearch use. This behavior is disturbing and very strange for UNIMARC flavour. This patch corrects by defining record ids mandatory in Koha::SearchEngine::Elasticsearch::Indexer::update_index(). This ids array is actually always given (except in UT). I think it is useless to allow adding a record without its id. Test plan : 1) Use Elasticsearch as SearchEngine 2) Create a subfield 999$z in default framework 3) Create a record with default framework 4) Enter a random string (never used in catalog) like "tototata" in 999$z 5) In Search engine configuration, define search field "subject" for 999$z 6) Rebuild record : misc/search_tools/rebuild_elasticsearch.pl -b -bn <biblionumber> -v 7) Search for the random string => You get a result 8) Optionnaly look at records in ES : <es server>:9200/<es index name>/data/<biblionumber> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA CC| |martin.renvoize@ptfs-europe | |.com --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Seems like a sane enough change, works as expected and passes qa scripts. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |19.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work! Pushed to master for 19.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 94912 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94912&action=edit Bug 23630: (QA follow-up) Fix tests marc_records_to_documents is now an arrayref of hashes, not an arrayref of arrays _sanitise_records has been removed, we don't need those tests anymore -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work! Pushed to master for 19.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23630 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|19.11.00 |19.11.00,19.05.05 released in| | Status|Pushed to master |Pushed to stable --- Comment #9 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Pushed to 19.05.x for 19.05.05 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org