[Bug 21716] New: Item Search hangs when \ exists in MARC fields
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Bug ID: 21716 Summary: Item Search hangs when \ exists in MARC fields Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Staff Client Assignee: koha-bugs@lists.koha-community.org Reporter: claire_gravely@hotmail.com QA Contact: testopia@bugs.koha-community.org CC: gmcharlt@gmail.com When there are \ (backslashes) in MARC records the item search hangs when trying to load the page the record should appear on. It gets stuck on processing. My specific example has a Publisher written as Narr\Francke\Attempto in 260 $b The error is: SyntaxError: JSON.parse: bad escaped character at line 8 column 9 of the JSON data Tested on 16.11 17.11 and master. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com --- Comment #1 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- OMG, we're using a TT template to convert Perl to JSON.. no wonder this doesn't work properly :'( -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #2 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- JSON.Escape and the `| json` filter is probably your friend here... or go for the bigger job and convert it to using a json parser properly. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Claire Gravely <claire_gravely@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #3 from Claire Gravely <claire_gravely@hotmail.com> --- Created attachment 82776 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82776&action=edit Bug 21716: Item Search hangs when \ exists in MARC fields When there are \ (backslashes) in MARC records the item search hangs when trying to load the page the record should appear on. It gets stuck on processing. This patch fixes this issue. To Test: 1. Create or edit a biblio and add in \ to the publisher field. eg: Narr\Francke\Attempto 2. Make sure this biblio has items attached that you can search for 3. Do an item search which would include the edited biblio in the results - a general search for a home library or shelving location will work 4. Try to load All results (or the page with the edited biblio) The page will hang :-( 5. Apply patch 6. Perform the item search again 7. Try to load All results The page will load :-) Mentored by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82776|0 |1 is obsolete| | --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 82779 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82779&action=edit Bug 21716: Item Search hangs when \ exists in MARC fields When there are \ (backslashes) in MARC records the item search hangs when trying to load the page the record should appear on. It gets stuck on processing. This patch fixes this issue. To Test: 1. Create or edit a biblio and add in \ to the publisher field. eg: Narr\Francke\Attempto 2. Make sure this biblio has items attached that you can search for 3. Do an item search which would include the edited biblio in the results - a general search for a home library or shelving location will work 4. Try to load All results (or the page with the edited biblio) The page will hang :-( 5. Apply patch 6. Perform the item search again 7. Try to load All results The page will load :-) Mentored-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This works as expected and is a helpful addition... I've tested it and am signing off. QA Note.. this is sort of horrible, that's we're generating JSON by passing it through a template.. but that's how it currently is and it would be a fairly major refactor to undertake for such a minor fix. We could apply the 'json' filter further too, but I believe that should be as a followup somewhere myself, rather than confusing the issue here. Might be nice to add regression tests too.. wonder if Jonathan or someone could mentor writing those (I must admit, to an extent the Selenium tests are still a mystery to me). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Claire Gravely <claire_gravely@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |claire_gravely@hotmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com Status|Signed Off |Failed QA --- Comment #6 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Martin Renvoize from comment #5)
This works as expected and is a helpful addition... I've tested it and am signing off. It doesn't work for me. The page loads, yes. But publisher and publication year have disappeared.
QA Note.. this is sort of horrible, that's we're generating JSON by passing it through a template.. but that's how it currently is and it would be a fairly major refactor to undertake for such a minor fix. I agree this is horrible, and I'm the one responsible for it :) When I wrote the patch, I believed it was the only way to have translatable strings in a server-side Datatable setup. Fortunately I was wrong, and we can make use of the datatable's 'mRender' option to separate the data (what should be returned in the JSON response) from the rendered html (which should contain translated strings) It does not require a big refactor. I will submit a patch soon.
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82779|0 |1 is obsolete| | --- Comment #7 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 82980 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82980&action=edit Bug 21716: Do not use TT to produce JSON for item search results It was initially done this way to have translatable strings in the results table, but this method has flaws and it can be done in a better way. This patch moves the rendering code to client-side javascript, so the JSON returned by the server now contains only unprocessed data. Also, this patch makes use of the new i18n macros for translation and fixes a whitespace bug in i18n.inc Reported-by: Claire Gravely <claire.gravely@bsz-bw.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Oh nice :) Fancy testing Claire and I'll grab it for QA? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #9 from Owen Leonard <oleonard@myacpl.org> --- The QA tool says there are missing filters, for example: missing_filter at line 383 ( output += ' ' + [% t('by').json %]; ) it should be: [% t('by').json | $raw %] -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #10 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 83147 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83147&action=edit Bug 21716: Add missing $raw filters -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wizzyrea@gmail.com Status|Needs Signoff |Failed QA --- Comment #11 from Liz Rea <wizzyrea@gmail.com> --- Hi, I've just checked this patch, and it seems that the hang doesn't occur on current master, but it is broken again without the patch (infinite processing). Is this still needed? It appears to me that it isn't. Cheers, Liz -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=20891 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #12 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Maybe the fix from bug 20891 would be helpful here. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82980|0 |1 is obsolete| | Attachment #83147|0 |1 is obsolete| | --- Comment #13 from Liz Rea <wizzyrea@gmail.com> --- Created attachment 88790 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=88790&action=edit Bug 21716: Escape publisher in item search To prevent endless processing errors. To test: * create a biblio with a publisher with a backslash - i.e. 260$b with graham\lineham * do an item search that will include that title in the results * wait forever, because it won't work. * apply this patch * repeat steps, notice that you a. get results b. your result is listed with it's correct publisher * rejoice -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #14 from Claire Gravely <claire_gravely@hotmail.com> --- Hi, I tried testing but with the patch applied I get no results at all (even the pages without my dodgy biblio). The item search results page hangs on processing and in the console I get a response 'internal server error' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Hayley Mapley <hayleymapley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #88790|0 |1 is obsolete| | --- Comment #15 from Hayley Mapley <hayleymapley@catalyst.net.nz> --- Created attachment 91780 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=91780&action=edit Bug 21716: Escape publisher in item search To prevent endless processing errors. To test: * create a biblio with a publisher with a backslash - i.e. 260$b with graham\lineham * do an item search that will include that title in the results * wait forever, because it won't work. * apply this patch * repeat steps, notice that you a. get results b. your result is listed with it's correct publisher * rejoice Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Hayley Mapley <hayleymapley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hayleymapley@catalyst.net.n | |z Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #91780|0 |1 is obsolete| | --- Comment #16 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 91807 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=91807&action=edit Bug 21716: Escape publisher in item search To prevent endless processing errors. To test: * create a biblio with a publisher with a backslash - i.e. 260$b with graham\lineham * do an item search that will include that title in the results * wait forever, because it won't work. * apply this patch * repeat steps, notice that you a. get results b. your result is listed with it's correct publisher * rejoice Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |19.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work! Pushed to master for 19.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable CC| |lucas@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 --- Comment #18 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 19.05.x for 19.05.03 backported to 18.11.x for 18.11.10 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org