[Bug 26557] New: Batch import fails when incoming records contain itemnumber
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Bug ID: 26557 Summary: Batch import fails when incoming records contain itemnumber Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: andrew@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org To recreate: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" 7 - the Import button grays out, the job progress bar appears, but the import never progresses I have confirmed this happens in master and 20.05. On 19.11 -- and previous versions, as far as I recall -- Koha makes a new item with a new itemnumber. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |26543 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26543 [Bug 26543] Elasticsearch - Importing biblios with multiple items can cause incorrect index -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |jonathan.druart@bugs.koha-c |ity.org |ommunity.org Severity|normal |major CC| |jonathan.druart@bugs.koha-c | |ommunity.org --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pretty sure this is caused by bug 23463. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |23463 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23463 [Bug 23463] Move C4::Items CRUD subroutines to Koha::Item -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 --- Comment #2 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 113816 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113816&action=edit Bug 26557: (bug 23463 follow-up) Fix Batch import when incoming records contain itemnumber Prior to ug 23463 AddItemFromMarc where calling AddItem, that did not take into account the itemnumber field. Now that we are using Koha::Item, we need to remove the items.itemnumber field from the MARC record Test plan: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" => Without this patch you get (in the logs, or hidden) manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '23' for key 'PRIMARY' [for Statement "INSERT INTO `items` ( `barcode`, `biblioitemnumber`, `biblionumber`, `ccode`, `cn_sort`, `cn_source`, `damaged_on`, `dateaccessioned`, `datelastborrowed`, `datelastseen`, `holdingbranch`, `homebranch`, `itemcallnumber`, `itemlost_on`, `itemnumber`, `itype`, `location`, `more_subfields_xml`, `onloan`, `permanent_location`, `replacementpricedate`, `timestamp`, `withdrawn_on`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, current_timestamp, ? )" with ParamValues: 0="BC_23", 1=8, 2=8, 3="REF", 4='CN__23', 5=undef, 6=undef, 7="2014-09-04", 8=undef, 9="2014-09-04", 10="FPL", 11="FPL", 12="CN_23", 13=undef, 14="23", 15="BK", 16="GEN", 17=undef, 18=undef, 19="GEN", 20="2014-09-04", 21=undef] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836. manage-marc-import.pl: DBIx::Class::Storage::DBI::_dbh_execute(): Duplicate entry '23' for key 'PRIMARY' at /kohadevbox/koha/Koha/Object.pm line 169 manage-marc-import.pl: {UNKNOWN}: Transaction aborted: Duplicate ID. Rollback failed: DBIx::Class::Storage::txn_rollback(): Refusing to roll back without a started transaction at /kohadevbox/koha/tools/manage-marc-import.pl line 253 at /kohadevbox/koha/tools/manage-marc-import.pl line 253 => With this patch applied, the new item must be added to the existing bibliographic record -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_20_11_target -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |victor@tuxayo.net --- Comment #3 from Victor Grousset/tuxayo <victor@tuxayo.net> --- I have the error in sip.log o_o does that happen to someone else?
==> sip.log <== [Thu Nov 19 18:23:32 2020] manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '969' for key 'PRIMARY'
my setup: koha-testing-docker
6 - click "Import this batch into the catalog"
Also in my case, the import just stay frozen. Nonetheless, it works with the patch :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #113816|0 |1 is obsolete| | --- Comment #4 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 113844 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113844&action=edit Bug 26557: (bug 23463 follow-up) Fix Batch import when incoming records contain itemnumber Prior to ug 23463 AddItemFromMarc where calling AddItem, that did not take into account the itemnumber field. Now that we are using Koha::Item, we need to remove the items.itemnumber field from the MARC record Test plan: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" => Without this patch you get (in the logs, or hidden) manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '23' for key 'PRIMARY' [for Statement "INSERT INTO `items` ( `barcode`, `biblioitemnumber`, `biblionumber`, `ccode`, `cn_sort`, `cn_source`, `damaged_on`, `dateaccessioned`, `datelastborrowed`, `datelastseen`, `holdingbranch`, `homebranch`, `itemcallnumber`, `itemlost_on`, `itemnumber`, `itype`, `location`, `more_subfields_xml`, `onloan`, `permanent_location`, `replacementpricedate`, `timestamp`, `withdrawn_on`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, current_timestamp, ? )" with ParamValues: 0="BC_23", 1=8, 2=8, 3="REF", 4='CN__23', 5=undef, 6=undef, 7="2014-09-04", 8=undef, 9="2014-09-04", 10="FPL", 11="FPL", 12="CN_23", 13=undef, 14="23", 15="BK", 16="GEN", 17=undef, 18=undef, 19="GEN", 20="2014-09-04", 21=undef] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836. manage-marc-import.pl: DBIx::Class::Storage::DBI::_dbh_execute(): Duplicate entry '23' for key 'PRIMARY' at /kohadevbox/koha/Koha/Object.pm line 169 manage-marc-import.pl: {UNKNOWN}: Transaction aborted: Duplicate ID. Rollback failed: DBIx::Class::Storage::txn_rollback(): Refusing to roll back without a started transaction at /kohadevbox/koha/tools/manage-marc-import.pl line 253 at /kohadevbox/koha/tools/manage-marc-import.pl line 253 => With this patch applied, the new item must be added to the existing bibliographic record Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #113844|0 |1 is obsolete| | --- Comment #5 from David Nind <david@davidnind.com> --- Created attachment 113845 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113845&action=edit Bug 26557: (bug 23463 follow-up) Fix Batch import when incoming records contain itemnumber Prior to ug 23463 AddItemFromMarc where calling AddItem, that did not take into account the itemnumber field. Now that we are using Koha::Item, we need to remove the items.itemnumber field from the MARC record Test plan: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" => Without this patch you get (in the logs, or hidden) manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '23' for key 'PRIMARY' [for Statement "INSERT INTO `items` ( `barcode`, `biblioitemnumber`, `biblionumber`, `ccode`, `cn_sort`, `cn_source`, `damaged_on`, `dateaccessioned`, `datelastborrowed`, `datelastseen`, `holdingbranch`, `homebranch`, `itemcallnumber`, `itemlost_on`, `itemnumber`, `itype`, `location`, `more_subfields_xml`, `onloan`, `permanent_location`, `replacementpricedate`, `timestamp`, `withdrawn_on`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, current_timestamp, ? )" with ParamValues: 0="BC_23", 1=8, 2=8, 3="REF", 4='CN__23', 5=undef, 6=undef, 7="2014-09-04", 8=undef, 9="2014-09-04", 10="FPL", 11="FPL", 12="CN_23", 13=undef, 14="23", 15="BK", 16="GEN", 17=undef, 18=undef, 19="GEN", 20="2014-09-04", 21=undef] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836. manage-marc-import.pl: DBIx::Class::Storage::DBI::_dbh_execute(): Duplicate entry '23' for key 'PRIMARY' at /kohadevbox/koha/Koha/Object.pm line 169 manage-marc-import.pl: {UNKNOWN}: Transaction aborted: Duplicate ID. Rollback failed: DBIx::Class::Storage::txn_rollback(): Refusing to roll back without a started transaction at /kohadevbox/koha/tools/manage-marc-import.pl line 253 at /kohadevbox/koha/tools/manage-marc-import.pl line 253 => With this patch applied, the new item must be added to the existing bibliographic record Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 --- Comment #6 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Sound like we both signed this off around the same time ^^ I'm resending the patch with both signoffs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #113845|0 |1 is obsolete| | --- Comment #7 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 113848 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113848&action=edit Bug 26557: (bug 23463 follow-up) Fix Batch import when incoming records contain itemnumber Prior to ug 23463 AddItemFromMarc where calling AddItem, that did not take into account the itemnumber field. Now that we are using Koha::Item, we need to remove the items.itemnumber field from the MARC record Test plan: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" => Without this patch you get (in the logs, or hidden) manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '23' for key 'PRIMARY' [for Statement "INSERT INTO `items` ( `barcode`, `biblioitemnumber`, `biblionumber`, `ccode`, `cn_sort`, `cn_source`, `damaged_on`, `dateaccessioned`, `datelastborrowed`, `datelastseen`, `holdingbranch`, `homebranch`, `itemcallnumber`, `itemlost_on`, `itemnumber`, `itype`, `location`, `more_subfields_xml`, `onloan`, `permanent_location`, `replacementpricedate`, `timestamp`, `withdrawn_on`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, current_timestamp, ? )" with ParamValues: 0="BC_23", 1=8, 2=8, 3="REF", 4='CN__23', 5=undef, 6=undef, 7="2014-09-04", 8=undef, 9="2014-09-04", 10="FPL", 11="FPL", 12="CN_23", 13=undef, 14="23", 15="BK", 16="GEN", 17=undef, 18=undef, 19="GEN", 20="2014-09-04", 21=undef] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836. manage-marc-import.pl: DBIx::Class::Storage::DBI::_dbh_execute(): Duplicate entry '23' for key 'PRIMARY' at /kohadevbox/koha/Koha/Object.pm line 169 manage-marc-import.pl: {UNKNOWN}: Transaction aborted: Duplicate ID. Rollback failed: DBIx::Class::Storage::txn_rollback(): Refusing to roll back without a started transaction at /kohadevbox/koha/tools/manage-marc-import.pl line 253 at /kohadevbox/koha/tools/manage-marc-import.pl line 253 => With this patch applied, the new item must be added to the existing bibliographic record Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #113848|0 |1 is obsolete| | --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 113871 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113871&action=edit Bug 26557: (bug 23463 follow-up) Fix Batch import when incoming records contain itemnumber Prior to ug 23463 AddItemFromMarc where calling AddItem, that did not take into account the itemnumber field. Now that we are using Koha::Item, we need to remove the items.itemnumber field from the MARC record Test plan: 1 - find an existing bib in your system with just one item 2 - export that bib with the item attached 3 - delete the barcode from your item in Koha 4 - stage your exported marc file for reimport, match on biblionumber, set it to Always Add Items 5 - confirm that the bib matches and the incoming 952 is parsed 6 - click "Import this batch into the catalog" => Without this patch you get (in the logs, or hidden) manage-marc-import.pl: DBD::mysql::st execute failed: Duplicate entry '23' for key 'PRIMARY' [for Statement "INSERT INTO `items` ( `barcode`, `biblioitemnumber`, `biblionumber`, `ccode`, `cn_sort`, `cn_source`, `damaged_on`, `dateaccessioned`, `datelastborrowed`, `datelastseen`, `holdingbranch`, `homebranch`, `itemcallnumber`, `itemlost_on`, `itemnumber`, `itype`, `location`, `more_subfields_xml`, `onloan`, `permanent_location`, `replacementpricedate`, `timestamp`, `withdrawn_on`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, current_timestamp, ? )" with ParamValues: 0="BC_23", 1=8, 2=8, 3="REF", 4='CN__23', 5=undef, 6=undef, 7="2014-09-04", 8=undef, 9="2014-09-04", 10="FPL", 11="FPL", 12="CN_23", 13=undef, 14="23", 15="BK", 16="GEN", 17=undef, 18=undef, 19="GEN", 20="2014-09-04", 21=undef] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836. manage-marc-import.pl: DBIx::Class::Storage::DBI::_dbh_execute(): Duplicate entry '23' for key 'PRIMARY' at /kohadevbox/koha/Koha/Object.pm line 169 manage-marc-import.pl: {UNKNOWN}: Transaction aborted: Duplicate ID. Rollback failed: DBIx::Class::Storage::txn_rollback(): Refusing to roll back without a started transaction at /kohadevbox/koha/tools/manage-marc-import.pl line 253 at /kohadevbox/koha/tools/manage-marc-import.pl line 253 => With this patch applied, the new item must be added to the existing bibliographic record Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Signed Off |Passed QA --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This works as expected, and the fix is clear. I'm going to PQA, but would really like to see some Unit tests added in the future.. not blocking here in this case as tests need writing for nearly all the methods in this module so blocking this patch for lack of tests isn't pragmatic. :( -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 20.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Version(s)|20.11.00 |20.11.00, 20.05.06 released in| | Status|Pushed to master |Pushed to stable --- Comment #11 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 20.05.x for 20.05.06 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aleisha@catalyst.net.nz Status|Pushed to stable |RESOLVED Resolution|--- |FIXED --- Comment #12 from Aleisha Amohia <aleisha@catalyst.net.nz> --- missing dependencies, not backported to 19.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26557 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_20_11_target | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org