[Bug 25342] New: Scripts not running under plack can cause duplication of ES records
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Bug ID: 25342 Summary: Scripts not running under plack can cause duplication of ES records Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com We received reports of duplicated records after batch modifications. It seems that when passing the ID to elasticsearch it is parsed as a string under Plack, but is parsed as a number under CGI -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 104108 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104108&action=edit Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | CC| |andrew@bywatersolutions.com | |, glasklas@gmail.com, | |jonathan.druart@bugs.koha-c | |ommunity.org, | |martin.renvoize@ptfs-europe | |.com, | |severine.queune@bulac.fr -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Eric Phetteplace <ephetteplace@cca.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ephetteplace@cca.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104108|0 |1 is obsolete| | --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 104175 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104175&action=edit Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #3 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Nick, this sounds familiar to me, did not we already get that kind of error with ES (int that needed to be stringified). Still no chance to get tests? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=23986 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #4 from David Gustafsson <glasklas@gmail.com> --- Good catch. I guess this fix can be reverted once https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24823 is merged since this seams to be a catmandu-issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #5 from David Gustafsson <glasklas@gmail.com> --- Wrong bug, sorry. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #6 from David Gustafsson <glasklas@gmail.com> --- Meant to comment on bug 23986, but applies here as well. I will fix the issue with the patch not applying correctly in bug 24823. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=24823 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 AspenCat Team <aspencatteam@clicweb.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aspencatteam@clicweb.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 104509 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104509&action=edit Bug 25342: Unit test -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #8 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #3)
Nick, this sounds familiar to me, did not we already get that kind of error with ES (int that needed to be stringified).
Still no chance to get tests?
The test is likely to pass before this patch as well, the issue is in the sending of the request somewhere, but this proves that at least stringifying it within Koha still gives what we expect -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Nick Clemens from comment #0)
We received reports of duplicated records after batch modifications.
It seems that when passing the ID to elasticsearch it is parsed as a string under Plack, but is parsed as a number under CGI
Reading the bug title I undersetood command line script, but it will actually fix script called in CGI mode, right? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #10 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #9)
(In reply to Nick Clemens from comment #0)
We received reports of duplicated records after batch modifications.
It seems that when passing the ID to elasticsearch it is parsed as a string under Plack, but is parsed as a number under CGI
Reading the bug title I undersetood command line script, but it will actually fix script called in CGI mode, right?
Yes, scripts that are excluded from plack like batch modification of bibs and imports are affected and fixed by this -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104175|0 |1 is obsolete| | Attachment #104509|0 |1 is obsolete| | --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 104685 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104685&action=edit Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 --- Comment #12 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 104686 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104686&action=edit Bug 25342: Unit test Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.05.00 released in| | --- Comment #13 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|20.05.00 |20.05.00, 19.11.06 released in| | CC| |joy@bywatersolutions.com --- Comment #14 from Joy Nelson <joy@bywatersolutions.com> --- Pushed to 19.11.x for 19.11.06 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25342 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.05.00, 19.11.06 |20.05.00, 19.11.06, released in| |19.05.11 Status|Pushed to stable |Pushed to oldstable CC| |lucas@bywatersolutions.com --- Comment #15 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 19.05.x for 19.05.11 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org