[Bug 24809] New: OAI PMH can fail on fetching deleted records
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Bug ID: 24809 Summary: OAI PMH can fail on fetching deleted records Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Authentication Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: dpavlin@rot13.org When calling GetRecord if we pass a deleted biblionumber that has a deleted item we can end up thinking a record that is deleted is not deleted and so fail on getting the record The problem is here: 48 UNION 49 SELECT timestamp from deleteditems 50 WHERE biblionumber=? If you delete an item from a record, then delete the record, the first query gets a timestamp and then attempts to get the record as if it is not deleted To recreate: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml 2 - Set sysprefs: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 100178 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100178&action=edit Bug 24809: Get only timestamp of deleted items attached to existing bibs when check if bib deleted To test: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml 2 - Set sysprefs: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 7 - Apply patch 8 - restart_all 9 - Repeat 10 - Reload URL from step 3 11 - Success -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Authentication |Architecture, internals, | |and plumbing Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #100178|0 |1 is obsolete| | --- Comment #2 from Michal Denar <black23@gmail.com> --- Created attachment 100317 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100317&action=edit Bug 24809: Get only timestamp of deleted items attached to existing bibs when check if bib deleted To test: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml 2 - Set sysprefs: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 7 - Apply patch 8 - restart_all 9 - Repeat 10 - Reload URL from step 3 11 - Success Signed-off-by: Michal Denar <black23@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com, | |josef.moravec@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl Status|Signed Off |Failed QA --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I am not able to reproduce this problem. Probably a config issue. The commit message is incomplete? Where do I insert that items should be included? I now only see the biblio record and have the impression that the yaml file is ignored. Should I add the created yaml file in the prefs? etc. If I add include_items: 1 and add the yaml file in the pref, I am seeing error: Dissemination as 'marcxml' is not supported. Please provide more info. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- '/var/lib/koha/master/OAI.yaml' is empty or non-existent at /usr/lib/x86_64-linux-gnu/perl5/5.24/YAML/Syck.pm line 129. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- ok, copied something from bug 12252 (create a yaml file is not very helpful in this message here) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- ah reproduced finally: Empty String at /usr/share/koha/Koha/OAI/Server/Record.pm line 58. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Still not getting the point of the change completely: $sql .= " UNION SELECT deleteditems.timestamp FROM deleteditems JOIN biblio USING (biblionumber) WHERE biblionumber=? UNION SELECT timestamp from items WHERE biblionumber=? "; Ambiguous now for biblionumber when adding the left join? Why do you add the left join if you only want deleteditems ? Please provide feedback. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #100317|0 |1 is obsolete| | --- Comment #8 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 100687 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=100687&action=edit Bug 24809: Get only timestamp of deleted items attached to existing bibs when check if bib deleted To test: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml' with content like below: format: marcxml: metadataPrefix: marcxml metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim schema: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd include_items: 1 2 - Set the systempreferences below to indicated values: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 7 - Apply patch 8 - restart_all 9 - Repeat 10 - Reload URL from step 3 11 - Success Signed-off-by: Michal Denar <black23@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |12252 Severity|enhancement |normal Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12252 [Bug 12252] OAI-PMH GetRecord result doesn't include item data -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #9 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Marcel de Rooy from comment #7)
Still not getting the point of the change completely:
$sql .= " UNION SELECT deleteditems.timestamp FROM deleteditems JOIN biblio USING (biblionumber) WHERE biblionumber=? UNION SELECT timestamp from items WHERE biblionumber=? ";
The goal of the query is the get the latest change for a record and return undef if record is deleted. We need to check deleted items because deleting an item will not update the biblioitems timestamp. We need to limit to deleted items where the bib is not deleted.
Ambiguous now for biblionumber when adding the left join?1 Why do you add the left join if you only want deleteditems ?
Where is the LEFT JOIN? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |BLOCKED --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Revisiting -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #9)
Where is the LEFT JOIN?
Scratch that. It is an INNER join. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #100687|0 |1 is obsolete| | --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 101066 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=101066&action=edit Bug 24809: Get only timestamp of deleted items attached to existing bibs when check if bib deleted To test: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml' with content like below: format: marcxml: metadataPrefix: marcxml metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim schema: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd include_items: 1 2 - Set the systempreferences below to indicated values: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 7 - Apply patch 8 - restart_all 9 - Repeat 10 - Reload URL from step 3 11 - Success Signed-off-by: Michal Denar <black23@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 101067 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=101067&action=edit Bug 24809: (QA follow-up) Add prefix to prevent disambiguity This seems not to be needed. Just at the safe side. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #14 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|20.05.00 |20.05.00, 19.11.05 released in| | CC| |joy@bywatersolutions.com --- Comment #15 from Joy Nelson <joy@bywatersolutions.com> --- Backported to 19.11.x branch for 19.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable CC| |lucas@bywatersolutions.com --- Comment #16 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 19.05.x for 19.05.10 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 --- Comment #17 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 19.05.x for 19.05.10 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24809 Hayley Mapley <hayleymapley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |Pushed to oldoldstable CC| |hayleymapley@catalyst.net.n | |z Version(s)|20.05.00, 19.11.05 |20.05.00, 19.11.05, released in| |19.05.10, 18.11.16 --- Comment #18 from Hayley Mapley <hayleymapley@catalyst.net.nz> --- Backported to 18.11.x for 18.11.16 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org