[Bug 30142] New: ElasticSearch MARC mappings should not accept whitespaces
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Bug ID: 30142 Summary: ElasticSearch MARC mappings should not accept whitespaces Change sponsored?: --- Product: Koha Version: 21.05 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: domm@plix.at We had a "funny" bug today, because I updated some ElasticSearch Biblio MARC mappings, and entered "245bn " instead of "245bn" (notice the trailing space?) The Form accepted the invalid value, but after any edit to a biblio we'd get a 500 Error, with this (not very helpful) error message in plack-error.log "Invalid MARC field expression: 245bn " After spotting this, it was easy to fix the typo, but I think that the form should either be stricter when validating input or at least remove whitespace from the MARC input (AFAIK whitesapce is not allowed here). There is a TODO marker in admin/searchengine/elasticsearch/mappings.pl#161 which is probably related :-) I will submit a patch in a minute that will at least remove all whitespaces. Actually checking the format should not be too hard with some regex, but I'm not sure if the format is specified anywhere, and in the DB I see quite different values ("440a", "111(abcd)", "008_/14", "leader_/0-4", ...) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 --- Comment #1 from Thomas Klausner <domm@plix.at> --- Created attachment 130948 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130948&action=edit replace whitespace in marc mapping A simple fix to prevent whitespaces in marc field mappings -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 --- Comment #2 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Don't forget to set to "Needs Sign-off" so your patches don't get lost! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130948|0 |1 is obsolete| | --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 130997 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130997&action=edit Bug 30142: Remove spaces from ElasticSearch mapping MARC fields A first step to "validate" the MARC mappings: Remove all whitespace, so if a user enters "245a " (with a trailing whitespace, which can easily happen when copy/pasting) we only store "245a" in the DB. This is neccessary, because the ES indexer will throw an exception in an invalid MARC mapping. Test Plan: * Go to /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl * Go to the Bibliographic Records Tab * Enter "100 a b c " (notice the whitespaces!) in the first "mapping" field * Scroll down and save * Go back to the Bibliographic Records Tab * The spaces are still there Now apply the patch * Repeat the above steps * After saving you should see "100abc" without any spaces in the "mapping" field Sponsored-by: Steiermärkische Landesbibliothek Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130997|0 |1 is obsolete| | --- Comment #4 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 132411 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132411&action=edit Bug 30142: Remove spaces from ElasticSearch mapping MARC fields A first step to "validate" the MARC mappings: Remove all whitespace, so if a user enters "245a " (with a trailing whitespace, which can easily happen when copy/pasting) we only store "245a" in the DB. This is neccessary, because the ES indexer will throw an exception in an invalid MARC mapping. Test Plan: * Go to /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl * Go to the Bibliographic Records Tab * Enter "100 a b c " (notice the whitespaces!) in the first "mapping" field * Scroll down and save * Go back to the Bibliographic Records Tab * The spaces are still there Now apply the patch * Repeat the above steps * After saving you should see "100abc" without any spaces in the "mapping" field Sponsored-by: Steiermärkische Landesbibliothek Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |nick@bywatersolutions.com Assignee|koha-bugs@lists.koha-commun |domm@plix.at |ity.org | CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Thanks Nick! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|21.05 |unspecified Severity|enhancement |normal CC| |jonathan.druart+koha@gmail. | |com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 --- Comment #6 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to master for 22.05, thanks to everybody involved 🦄 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com Status|Pushed to master |Pushed to stable Version(s)|22.05.00 |22.05.00,21.11.05 released in| | --- Comment #7 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to 21.11.x for 21.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com Version(s)|22.05.00,21.11.05 |22.05.00,21.11.05,21.05.14 released in| | Status|Pushed to stable |Pushed to oldstable --- Comment #8 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Pushed to 21.05.x for 21.05.14 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30142 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED CC| |victor@tuxayo.net --- Comment #9 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Not backported to oldoldstable (20.11.x). Feel free to ask if it's needed. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org