[Bug 9523] New: Trying to import staged marc records where a matched bib has been deleted caused the import to hang
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Bug ID: 9523 Summary: Trying to import staged marc records where a matched bib has been deleted caused the import to hang Classification: Unclassified Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Cataloging Assignee: gmcharlt@gmail.com Reporter: kyle@bywatersolutions.com CC: m.de.rooy@rijksmuseum.nl If a MARC file is staged and has found matching records already in Koha, but one of those records that was matched on is deleted before the staged records are imported, the import will stop cold at the missing record and just hang there indefinitely. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 --- Comment #1 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 14985 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=14985&action=edit Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |gmcharlt@gmail.com --- Comment #2 from Galen Charlton <gmcharlt@gmail.com> --- Your patch changes the query to SELECT candidate_match_id LEFT JOIN biblio ON ( candidate_match_id = biblionumber ) FROM import_record_matches WHERE import_record_id = ? ORDER BY score DESC, candidate_match_id DESC; This is invalid SQL ... JOIN clauses come after FROM clauses. Even were the SQL statement constructed correctly, the left join by itself would make no difference to the query results, as the candidate_match_ids would get returned whether or not the matching bib was still attached. The patch might *appear* to work, but would have the effect of unconditionally all bibs in the import batch, regardless of matches. As a note for testing, I suggest using the command-line staging import tools, as they reproduce the problem *and* give useful output in the case of a failure. For example, if you follow the reproduction steps but use the command-line tools to commit the batch, you get the following error: $ misc/commit_file.pl --batch-number 5 ... importing MARC records -- please wait Empty String at /usr/lib/perl5/XML/LibXML/SAX/Parser.pm line 42 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 --- Comment #3 from Kyle M Hall <kyle@bywatersolutions.com> --- You are correct. I'll post up a corrected patch when I can. Kyle (In reply to comment #2)
Your patch changes the query to
SELECT candidate_match_id LEFT JOIN biblio ON ( candidate_match_id = biblionumber ) FROM import_record_matches WHERE import_record_id = ? ORDER BY score DESC, candidate_match_id DESC;
This is invalid SQL ... JOIN clauses come after FROM clauses. Even were the SQL statement constructed correctly, the left join by itself would make no difference to the query results, as the candidate_match_ids would get returned whether or not the matching bib was still attached.
The patch might *appear* to work, but would have the effect of unconditionally all bibs in the import batch, regardless of matches.
As a note for testing, I suggest using the command-line staging import tools, as they reproduce the problem *and* give useful output in the case of a failure. For example, if you follow the reproduction steps but use the command-line tools to commit the batch, you get the following error:
$ misc/commit_file.pl --batch-number 5 ... importing MARC records -- please wait Empty String at /usr/lib/perl5/XML/LibXML/SAX/Parser.pm line 42
-- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #14985|0 |1 is obsolete| | --- Comment #4 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 15143 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=15143&action=edit Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #15143|0 |1 is obsolete| | --- Comment #5 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 15144 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=15144&action=edit Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #15144|0 |1 is obsolete| | --- Comment #6 from Galen Charlton <gmcharlt@gmail.com> --- Created attachment 16079 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16079&action=edit Bug 9523: importing staged bib records hangs if a matched bib has been deleted Test Plan: 1) Stage a MARC record file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. Signed-off-by: Galen Charlton <gmc@esilibrary.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m --- Comment #7 from Jonathan Druart <jonathan.druart@biblibre.com> --- Kyle, I cannot reproduce the issue. My test plan: 1/ Choose a biblio (biblionumber=XXX) 2/ Save it as MARC unicode/UTF-8 3/ Stage my record file with a matching rule (ISBN) and add the incoming record if matching. Screen is: 1 records in file 0 records not staged because of MARC error 1 records staged 1 records with at least one match in catalog per matching rule "ISBN" 1 item records found and staged 4/ Delete items and biblio XXX 5/ Import into the catalog Screen is: Number of records added 1 Number of records updated 0 Number of records ignored 0 Number of items added 1 Number of items ignored because of duplicate barcode 0 A new record with biblionumber=YYY is created. What I missed? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle.m.hall@gmail.com --- Comment #8 from Kyle M Hall <kyle.m.hall@gmail.com> --- Odd, I can still replicate this bug on master: http://screencast.com/t/M5qWorUpl Kyle (In reply to comment #7)
Kyle, I cannot reproduce the issue.
My test plan: 1/ Choose a biblio (biblionumber=XXX) 2/ Save it as MARC unicode/UTF-8 3/ Stage my record file with a matching rule (ISBN) and add the incoming record if matching. Screen is: 1 records in file 0 records not staged because of MARC error 1 records staged 1 records with at least one match in catalog per matching rule "ISBN" 1 item records found and staged
4/ Delete items and biblio XXX 5/ Import into the catalog Screen is: Number of records added 1 Number of records updated 0 Number of records ignored 0 Number of items added 1 Number of items ignored because of duplicate barcode 0
A new record with biblionumber=YYY is created.
What I missed?
-- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA QA Contact| |jonathan.druart@biblibre.co | |m --- Comment #9 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to comment #8)
Odd, I can still replicate this bug on master: http://screencast.com/t/M5qWorUpl
Kyle
Ok thanks. I can reproduce if I follow your way. I did not import twice the same record, but just one time an existing record. I thought it was the same thing. Marked as Passed QA. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16079|0 |1 is obsolete| | --- Comment #10 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 17344 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=17344&action=edit Bug 9523: importing staged bib records hangs if a matched bib has been deleted Test Plan: 1) Stage a MARC record file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. Signed-off-by: Galen Charlton <gmc@esilibrary.com> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 --- Comment #11 from Kyle M Hall <kyle.m.hall@gmail.com> --- (In reply to comment #9)
(In reply to comment #8)
Odd, I can still replicate this bug on master: http://screencast.com/t/M5qWorUpl
Kyle
Ok thanks. I can reproduce if I follow your way. I did not import twice the same record, but just one time an existing record. I thought it was the same thing.
Marked as Passed QA.
I should have worded it more explicitly, my bad! Kyle -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #12 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- This patch has been pushed to master and 3.12.x. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9523 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |chris@bigballofwax.co.nz --- Comment #13 from Chris Cormack <chris@bigballofwax.co.nz> --- Pushed to 3.10.x will be in 3.10.6 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org