[Bug 30308] New: bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData)
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Bug ID: 30308 Summary: bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData) Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: magnus@libriotech.no QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz I was trying to import records and items with bulkmarcimport.pl. The records were added, but no items were saved to the DB. Turning off OAI-PMH:AutoUpdateSets and OAI-PMH:AutoUpdateSetsEmbedItemData fixed the problem. I think this is what happens: 1. bulkmarcimport.pl calls C4::Biblio::AddBiblio on line 461: eval { ( $biblionumber, $biblioitemnumber ) = AddBiblio( $record, '', { defer_marc_save => 1 } ) }; https://git.koha-community.org/Koha-community/Koha/src/branch/master/misc/mi... 2. C4::Biblio::AddBiblio has this code: # update OAI-PMH sets if(C4::Context->preference("OAI-PMH:AutoUpdateSets")) { C4::OAI::Sets::UpdateOAISetsBiblio($biblionumber, $record); } https://git.koha-community.org/Koha-community/Koha/src/branch/master/C4/Bibl... 3. C4::OAI::Sets::UpdateOAISetsBiblio does this: if (C4::Context->preference('OAI-PMH:AutoUpdateSetsEmbedItemData')) { C4::Biblio::EmbedItemsInMarcBiblio({ marc_record => $record, biblionumber => $biblionumber }); } https://git.koha-community.org/Koha-community/Koha/src/branch/master/C4/OAI/... 4. C4::Biblio::EmbedItemsInMarcBiblio does: my ($params) = @_; $marc = $params->{marc_record}; ... _strip_item_fields($marc, $frameworkcode); # ... and embed the current items my $dbh = C4::Context->dbh; my $sth = $dbh->prepare("SELECT itemnumber FROM items WHERE biblionumber = ?"); https://git.koha-community.org/Koha-community/Koha/src/branch/master/C4/Bibl... The end result is that item data is stripped from the record that is about to be imported, and no item data is added from the DB, since they are not there yet. Not sure how to fix this. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal CC| |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- This stood out to me as I have some plugin code in C4::Biblio::AddBiblio which fails due to bulkmarcimport.pl using defer_marc_save. I imagine this wouldn't be an issue if we just got rid of the frustrating defer_marc_save via Bug 25539. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Oh actually on another review... it's totally unrelated. Don't mind me. I'm just really biased against defer_marc_save 😅 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |major -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #3 from Fridolin Somers <fridolin.somers@biblibre.com> --- We should clone MARC::Record in UpdateOAISetsBiblio ? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #4 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 137502 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137502&action=edit Bug 30308 - Fix bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |frederic@tamil.fr --- Comment #5 from Frédéric Demians <frederic@tamil.fr> --- (In reply to Fridolin Somers from comment #3)
We should clone MARC::Record in UpdateOAISetsBiblio ?
Yes! It solves the issue. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Shouldn't this be Needs Signoff, Frederic? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #7 from David Nind <david@davidnind.com> --- Could we get a test plan for this? Thanks! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137502|0 |1 is obsolete| | --- Comment #8 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 137630 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137630&action=edit Bug 30308 - Fix bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData) Test plan: (A) Reproduce the bug: A1 In Administration > System preferences > Web services: - Enable OAI-PMH - Enable OAI-PMH:AutoUpdateSets - Enable OAI-PMH:AutoUpdateSetsEmbedItemData A2 In Tools > Export data > Export bibliographic records, export 1 (one) biblio record. Be sure that this biblio record has at least one item. Don't tick "Don't export items". A3 Delete the exported biblio record, and its items. A4 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc A5 Retrieve the biblio record in Koha. Note the absence of the item(s). This is the bug. (B) Apply the patch B1 Delete the record (without item) loaded at A4. B2 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc B2 Retrieve the biblio record in Koha. Note the presence of the item(s). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137630|0 |1 is obsolete| | --- Comment #9 from David Nind <david@davidnind.com> --- Created attachment 137721 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137721&action=edit Bug 30308 - Fix bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData) Test plan: (A) Reproduce the bug: A1 In Administration > System preferences > Web services: - Enable OAI-PMH - Enable OAI-PMH:AutoUpdateSets - Enable OAI-PMH:AutoUpdateSetsEmbedItemData A2 In Tools > Export data > Export bibliographic records, export 1 (one) biblio record. Be sure that this biblio record has at least one item. Don't tick "Don't export items". A3 Delete the exported biblio record, and its items. A4 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc A5 Retrieve the biblio record in Koha. Note the absence of the item(s). This is the bug. (B) Apply the patch B1 Delete the record (without item) loaded at A4. B2 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc B2 Retrieve the biblio record in Koha. Note the presence of the item(s). Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #10 from David Nind <david@davidnind.com> --- Thanks Frédéric for the test plan! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |frederic@tamil.fr |ity.org | --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Frederic, please don't forget to assign yourself! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |kyle@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137721|0 |1 is obsolete| | --- Comment #12 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 138301 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=138301&action=edit Bug 30308: Fix bulkmarcimport.pl broken by OAI-PMH:AutoUpdateSets(EmbedItemData) Test plan: (A) Reproduce the bug: A1 In Administration > System preferences > Web services: - Enable OAI-PMH - Enable OAI-PMH:AutoUpdateSets - Enable OAI-PMH:AutoUpdateSetsEmbedItemData A2 In Tools > Export data > Export bibliographic records, export 1 (one) biblio record. Be sure that this biblio record has at least one item. Don't tick "Don't export items". A3 Delete the exported biblio record, and its items. A4 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc A5 Retrieve the biblio record in Koha. Note the absence of the item(s). This is the bug. (B) Apply the patch B1 Delete the record (without item) loaded at A4. B2 In command line, on the server load the exported file: ./bulkmarcimport.pl -b -v -file /path/to/koha.mrc B2 Retrieve the biblio record in Koha. Note the presence of the item(s). Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |22.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #14 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Don't forget to sign, Kyle :) Thx for QA! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #15 from Fridolin Somers <fridolin.somers@biblibre.com> --- Please backport to old branches -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable CC| |lucas@bywatersolutions.com Version(s)|22.11.00 |22.11.00, 22.05.05 released in| | --- Comment #16 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for 22.05.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.11.00, 22.05.05 |22.11.00,21.11, 22.05.05 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 --- Comment #17 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- Thanks! Pushed to 21.11 for 21.11.12 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.11.00,21.11, 22.05.05 |22.11.00,22.05.05,21.11.12 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30308 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED CC| |victor@tuxayo.net Version(s)|22.11.00,22.05.05,21.11.12 |22.11.00,22.05.05,21.11.12, released in| |21.05.19 --- Comment #18 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported: Pushed to 21.05.x branch for 21.05.19 nothing to document -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org