[Bug 41008] New: bulkmarcimport.pl -d broken for authorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Bug ID: 41008 Summary: bulkmarcimport.pl -d broken for authorities Change sponsored?: --- Product: Koha Version: 24.11 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: katrin.fischer@bsz-bw.de QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz In 24.11 the system complains about a FK constraint to the new oai tables when the script is run with -a -d. DBI Exception: DBD::mysql::db do failed: Cannot truncate a table referenced in a foreign key constraint (`koha_xxx`.`import_oai_authorities`, CONSTRAINT `FK_import_oai_authorities_1` FOREIGN KEY (`authid`) REFERENCES `koha_dlre1`.`auth_header` (`authid`)) at /usr/share/perl5/DBIx/Class/Schema.pm line -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |katrin.fischer@bsz-bw.de |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |35659 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35659 [Bug 35659] OAI harvester -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 187852 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187852&action=edit Bug 41008: Fix bulkmarimport.pl -a -d After bug 35659 that added the OAI harvester to Koha the -d switch for deleting authorities before importing with bulkmarcimport.pl was broken and resulted in a FK error. Switching from truncate to DELETE FROM like we already did for items, biblio etc. fixes the issue. To test: - Make sure you have some authorities in your database - Export some into a file for importing - Run bulkmarcimport -a -d --file <filename> - Verify you get the a DB FK constraint error. - Apply patch - Repeat steps, verify the script now works as expected again * Existing authorities are deleted * File is imported -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #2 from David Cook <dcook@prosentient.com.au> --- I'll take a look at this soon -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #3 from David Cook <dcook@prosentient.com.au> --- This does work... although technically do we want to add the following to match its biblio counterparts? "ALTER TABLE auth_header AUTO_INCREMENT = 1" -- Personally, I'd advise against doing that, because it could lead to duplicate IDs in the deletedauth_header table... the biblio* counterpart is problematic... I also note that the index isn't cleared with this delete either so a full re-index is necessary later... -- I'll attach a signed off patch but leaving "In Discussion" just to get some feedback on those points... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187852|0 |1 is obsolete| | --- Comment #4 from David Cook <dcook@prosentient.com.au> --- Created attachment 188458 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=188458&action=edit Bug 41008: Fix bulkmarimport.pl -a -d After bug 35659 that added the OAI harvester to Koha the -d switch for deleting authorities before importing with bulkmarcimport.pl was broken and resulted in a FK error. Switching from truncate to DELETE FROM like we already did for items, biblio etc. fixes the issue. To test: - Make sure you have some authorities in your database - Export some into a file for importing - Run bulkmarcimport -a -d --file <filename> - Verify you get the a DB FK constraint error. - Apply patch - Repeat steps, verify the script now works as expected again * Existing authorities are deleted * File is imported Signed-off-by: David Cook <dcook@prosentient.com.au> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- My feeling is that this fixes the immediate bug, while your suggestions would be worthwhile enhancements for another bug. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- In Discussion feels a little strong? Setting back to NSO to get more opinions. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to David Cook from comment #3)
This does work... although technically do we want to add the following to match its biblio counterparts?
"ALTER TABLE auth_header AUTO_INCREMENT = 1"
--
Personally, I'd advise against doing that, because it could lead to duplicate IDs in the deletedauth_header table... the biblio* counterpart is problematic...
I feel like this would be an addition beyond the current problem of the option throwing an error and as you said, might need more thought to be done properly. We might want to add an option to clean out deleted tables as well, but my feeling is that this is a migration script. Most people use it on an empty database.
I also note that the index isn't cleared with this delete either so a full re-index is necessary later...
I believe that is always the case when using bulkmarcimport and has always been the case. We could add a switch to properly reset the index, but that woudl be a separate bug in my opinion and also needed for bilbiographic.
I'll attach a signed off patch but leaving "In Discussion" just to get some feedback on those points...
In Discussion usually doesn't get any feedback because it falls out of the queues that are being looked at. Until we start argueing, I'll leave as NSO :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com, | |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #8 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #7)
In Discussion usually doesn't get any feedback because it falls out of the queues that are being looked at. Until we start argueing, I'll leave as NSO :)
Hehe fair enough. I don't know. Based on your responses, I think that I'm happy to put this to Signed Off. It works. Maybe a QAer will have a strong opinion one way or another. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Jan Kissig <bibliothek@th-wildau.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bibliothek@th-wildau.de See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=12488 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Jan Kissig <bibliothek@th-wildau.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #188458|0 |1 is obsolete| | --- Comment #9 from Jan Kissig <bibliothek@th-wildau.de> --- Created attachment 189916 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=189916&action=edit Bug 41008: Fix bulkmarimport.pl -a -d After bug 35659 that added the OAI harvester to Koha the -d switch for deleting authorities before importing with bulkmarcimport.pl was broken and resulted in a FK error. Switching from truncate to DELETE FROM like we already did for items, biblio etc. fixes the issue. To test: - Make sure you have some authorities in your database - Export some into a file for importing - Run bulkmarcimport -a -d --file <filename> - Verify you get the a DB FK constraint error. - Apply patch - Repeat steps, verify the script now works as expected again * Existing authorities are deleted * File is imported Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #10 from Jan Kissig <bibliothek@th-wildau.de> --- I linked #12488 where DELETE was introduced to bulkmarcimport for biblios. In my opinion this fix is sufficient to solve the issue and all other discussions should be put into a separate bug: - set auto_increment in auth_header to 1 - also delete/empty the deleted-(auth_header,biblio,biblioitems,items) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #189916|0 |1 is obsolete| | --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 190014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=190014&action=edit Bug 41008: Fix bulkmarimport.pl -a -d After bug 35659 that added the OAI harvester to Koha the -d switch for deleting authorities before importing with bulkmarcimport.pl was broken and resulted in a FK error. Switching from truncate to DELETE FROM like we already did for items, biblio etc. fixes the issue. To test: - Make sure you have some authorities in your database - Export some into a file for importing - Run bulkmarcimport -a -d --file <filename> - Verify you get the a DB FK constraint error. - Apply patch - Repeat steps, verify the script now works as expected again * Existing authorities are deleted * File is imported Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | Version|24.11 |Main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |25.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #12 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Pushed to stable Version(s)|25.11.00 |25.11.00,25.05.06 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #13 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Nice work everyone! Pushed to 25.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable CC| |fridolin.somers@biblibre.co | |m Version(s)|25.11.00,25.05.06 |25.11.00,25.05.06,24.11.11 released in| | --- Comment #14 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 24.11.x for 24.11.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41507 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41508 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 --- Comment #15 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Jan Kissig from comment #10)
I linked #12488 where DELETE was introduced to bulkmarcimport for biblios.
In my opinion this fix is sufficient to solve the issue and all other discussions should be put into a separate bug: - set auto_increment in auth_header to 1 - also delete/empty the deleted-(auth_header,biblio,biblioitems,items)
Thanks Jan and Marcel. I have filed the bug reports for these now: * Bug 41507 - bulkmarcimport.pl: When deleting authorities prior to import, reset auto_increment * Bug 41508 - bulkmarcimport.pl: Add option to clean deleted* tables when -d is used I am a bit doubtul about the second - if you are deleting all your records with -d, they don't go into the deleted tables. So for a migration use case, this might not be needed/useful. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |Needs documenting CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #16 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Depends on bug not in 22.11.x Not backporting to 22.11.x -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org