[Bug 25873] New: Elasticsearch - Records with malformed data are not indexed
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Bug ID: 25873 Summary: Elasticsearch - Records with malformed data are not indexed Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: julian.maurice@biblibre.com Reporter: julian.maurice@biblibre.com If a record contains malformed data that is supposed to be indexed in an integer field, the document is rejected by elasticsearch. Elasticsearch should ignore malformed data. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 --- Comment #1 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 106276 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106276&action=edit Bug 25873: Ignore malformed data for Elasticsearch integer fields If we try to put malformed data into an integer field, Elasticsearch rejects the whole document. Setting 'ignore_malformed' to true allows to ignore malformed data and process the other fields of the document normally https://www.elastic.co/guide/en/elasticsearch/reference/7.8/ignore-malformed... Test plan: * Without the patch 1. In search engine configuration, change the type of a text field to 'Number' (for instance 'title') 2. misc/search_tools/rebuild_elasticsearch.pl -d -b 3. See that the index is empty (unless you have titles consisting only of digits) * With the patch 1. misc/search_tools/rebuild_elasticsearch.pl -d -b 2. Now records are correctly indexed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106276|0 |1 is obsolete| | --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 106703 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106703&action=edit Bug 25873: Ignore malformed data for Elasticsearch integer fields If we try to put malformed data into an integer field, Elasticsearch rejects the whole document. Setting 'ignore_malformed' to true allows to ignore malformed data and process the other fields of the document normally https://www.elastic.co/guide/en/elasticsearch/reference/7.8/ignore-malformed... Test plan: * Without the patch 1. In search engine configuration, change the type of a text field to 'Number' (for instance 'title') 2. misc/search_tools/rebuild_elasticsearch.pl -d -b 3. See that the index is empty (unless you have titles consisting only of digits) * With the patch 1. misc/search_tools/rebuild_elasticsearch.pl -d -b 2. Now records are correctly indexed Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA CC| |nick@bywatersolutions.com --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Trivial config change that can be overridden if desired, passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 20.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Version(s)|20.11.00 |20.11.00, 20.05.03 released in| | --- Comment #5 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 20.05.x for 20.05.03 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aleisha@catalyst.net.nz Version(s)|20.11.00, 20.05.03 |20.11.00, 20.05.03, released in| |19.11.09 Status|Pushed to master |Pushed to stable --- Comment #6 from Aleisha Amohia <aleisha@catalyst.net.nz> --- backported to 19.11.x for 19.11.09 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net Version(s)|20.11.00, 20.05.03, |20.11.00, 20.05.03, released in|19.11.09 |19.11.09, 19.05.15 Status|Pushed to stable |Pushed to oldstable --- Comment #7 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported to 19.05.x branch for 19.05.15 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25873 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |Pushed to oldoldstable -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org