[Bug 13531] New: bulkmarcimport bombs if no match is found
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Bug ID: 13531 Summary: bulkmarcimport bombs if no match is found Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: MARC Bibliographic record staging/import Assignee: gmcharlt@gmail.com Reporter: martin.renvoize@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org When using bulkmarcimport with the below command it would appear that if the match attribute is missing from the import file that the script dies at that point in the import. ../kohaclone/misc/migration_tools/bulkmarcimport.pl -filter="952" -update -match="Control-number,001" -file Results/all_ches_bib.mrc -v -l Logs/overlay.log I feel this should report the error, but more conventiently continue with the rest of the import batch thereafter. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #1 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- The errors presented are: DBD::mysql::st execute failed: Column 'biblio_auth_number' cannot be null at /home/koha/kohaclone/C4/Biblio.pm line 2871. followed by: unable to search the database for duplicates : No query entered at ../kohaclone/misc/migration_tools/bulkmarcimport.pl line 272, <GEN13> line 60379 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |Academy -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Thomas Wright <tomsStudy@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomsStudy@gmail.com Status|NEW |ASSIGNED --- Comment #2 from Thomas Wright <tomsStudy@gmail.com> --- I'll work on this. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #3 from Thomas Wright <tomsStudy@gmail.com> --- Created attachment 35133 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35133&action=edit Bug 13531 - bulkmarcimport bombs if no match is found Changed the die statment to a warn allowing the import to continue. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Thomas Wright <tomsStudy@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35133|0 |1 is obsolete| | --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 35147 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35147&action=edit Bug 13531 - bulkmarcimport bombs if no match is found Changed the die statment to a warn allowing the import to continue. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35147|Bug 13531 - bulkmarcimport |[SIGNED OFF] Bug 13531 - description|bombs if no match is found |bulkmarcimport bombs if no |Changed the die statment to |match is found Changed the |a warn allowing the import |die statment to a warn |to continue. |allowing the import to | |continue. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Thank Thomas, caught the error nicely and good clean handling :) Signing Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Comment on attachment 35147 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35147 [SIGNED OFF] Bug 13531 - bulkmarcimport bombs if no match is found Changed the die statment to a warn allowing the import to continue. Review of attachment 35147: --> (http://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=13531&attachment=35147) ----------------------------------------------------------------- A minor QA comment. ::: misc/migration_tools/bulkmarcimport.pl @@ +271,5 @@
my ( $error, $results, $totalhits ) = C4::Search::SimpleSearch( $query, 0, 3, [$server] ); + # changed to warn so able to continue with one broken record + if ( defined $error ) { + warn "unable to search the database for duplicates : $error"; + next;
For consistency with the rest of the script, should this perhaps be: next RECORD; -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #7 from Thomas Wright <tomsStudy@gmail.com> --- Created attachment 35260 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35260&action=edit Bug 13531: QA follow up A minor QA comment. ::: misc/migration_tools/bulkmarcimport.pl @@ +271,5 @@
my ( $error, $results, $totalhits ) = C4::Search::SimpleSearch( $query, 0, 3, [$server] ); + # changed to warn so able to continue with one broken record + if ( defined $error ) { + warn "unable to search the database for duplicates : $error"; + next;
For consistency with the rest of the script, should this perhaps be: next RECORD; -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Thomas Wright <tomsStudy@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35260|0 |1 is obsolete| | --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 35295 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35295&action=edit Bug 13531: QA follow up A minor QA comment. ::: misc/migration_tools/bulkmarcimport.pl @@ +271,5 @@
my ( $error, $results, $totalhits ) = C4::Search::SimpleSearch( $query, 0, 3, [$server] ); + # changed to warn so able to continue with one broken record + if ( defined $error ) { + warn "unable to search the database for duplicates : $error"; + next;
For consistency with the rest of the script, should this perhaps be: next RECORD; Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Awesome speedy followup there Thomas.. thanks :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35295|Bug 13531: QA follow up |[SIGNED OFF] Bug 13531: QA description| |follow up -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m --- Comment #10 from Jonathan Druart <jonathan.druart@biblibre.com> --- Martin, shouldn't we call the printlog subroutine too? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #11 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- (In reply to Jonathan Druart from comment #10)
Martin, shouldn't we call the printlog subroutine too?
Indeed, I think your right Jonathan.. this is something worth logging to file rather than just dumping to stdout.. it looks like in the above invocations it does both.. which seems a little messy to me.. but I suppose it does make the user aware that they should look at the log file. Cheers for pointing it out.. perhaps another qa followup. I have a feeling Thomas has completed his academy placement now.. I'll leave it for him for a few days just in case he want to complete the work, but go ahead and do it myself if it doesn't get done. Cheers -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #12 from Thomas Wright <tomsStudy@gmail.com> --- (In reply to Martin Renvoize from comment #11)
(In reply to Jonathan Druart from comment #10)
Martin, shouldn't we call the printlog subroutine too?
Indeed, I think your right Jonathan.. this is something worth logging to file rather than just dumping to stdout.. it looks like in the above invocations it does both.. which seems a little messy to me.. but I suppose it does make the user aware that they should look at the log file.
Cheers for pointing it out.. perhaps another qa followup.
I have a feeling Thomas has completed his academy placement now.. I'll leave it for him for a few days just in case he want to complete the work, but go ahead and do it myself if it doesn't get done.
Cheers
Hey Martin, The academy is over, but I am hoping to continue with contribution to Koha. Without Chris to guide me could you please advise me as to where I would find an example of the log syntax. Not sure if I will be able to figure it out, but i'll try! Thanks -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #13 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- (In reply to Thomas Wright from comment #12)
Hey Martin,
The academy is over, but I am hoping to continue with contribution to Koha. Without Chris to guide me could you please advise me as to where I would find an example of the log syntax. Not sure if I will be able to figure it out, but i'll try!
Thanks
That's great news that your tempted to stick around Thomas, we can always do with more keen contributors and you'll find even outside the Catalyst environment you should get allot of support for us.. Have you discovered the IRC channel, it's a great place to seek advice if/whenever your stuck :) As for the log syntax, this script is a bit of a special case for logging (there's a whole debate regarding logging in general for koha going on at the moment). But, having said that, the bulkmarcimport tool already impliments it's own loggin system; An example of this can be seen aroud line: 404 (This should be the best example for your particular required followup ;) ).. to see how the logging itself works (just for interest) you could take a look at the routine itself found on lines 592 -> 595. Cheers Martin -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #14 from Thomas Wright <tomsStudy@gmail.com> --- Created attachment 35486 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35486&action=edit Follow up -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Thomas Wright <tomsStudy@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #15 from Thomas Wright <tomsStudy@gmail.com> --- Not sure if I did the logging correctly, but I had a go! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35486|0 |1 is obsolete| | --- Comment #16 from Magnus Enger <magnus@enger.priv.no> --- Created attachment 36625 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36625&action=edit [SIGNED OFF] Bug 13531 - Follow up Add logging of errors. Signed-off-by: Magnus Enger <magnus@enger.priv.no> More errors are indeed showing up in the log. (I took the liberty of changing the commit message a little bit.) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #17 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I see the warn, but not the message in the log file. Leaving this for Martin to check again. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #18 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Looks good to me Katrin.. those errors will only appear in pretty specific cases.. your probably just not causing them? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #19 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 37044 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37044&action=edit [PASSED QA] Bug 13531 - bulkmarcimport bombs if no match is found Changed the die statment to a warn allowing the import to continue. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #35147|0 |1 is obsolete| | Attachment #35295|0 |1 is obsolete| | Attachment #36625|0 |1 is obsolete| | Attachment #37044|0 |1 is obsolete| | --- Comment #20 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 37045 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37045&action=edit [PASSED QA] Bug 13531 - bulkmarcimport bombs if no match is found Changed the die statment to a warn allowing the import to continue. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #21 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 37046 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37046&action=edit [PASSED QA] Bug 13531: QA follow up A minor QA comment. ::: misc/migration_tools/bulkmarcimport.pl @@ +271,5 @@
my ( $error, $results, $totalhits ) = C4::Search::SimpleSearch( $query, 0, 3, [$server] ); + # changed to warn so able to continue with one broken record + if ( defined $error ) { + warn "unable to search the database for duplicates : $error"; + next;
For consistency with the rest of the script, should this perhaps be: next RECORD; Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 --- Comment #22 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 37047 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37047&action=edit [PASSED QA] Bug 13531 - Follow up Add logging of errors. Signed-off-by: Magnus Enger <magnus@enger.priv.no> More errors are indeed showing up in the log. (I took the liberty of changing the commit message a little bit.) Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Status|Passed QA |Pushed to Master --- Comment #23 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patches pushed to master. Thanks Thomas! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |chris@bigballofwax.co.nz --- Comment #24 from Chris Cormack <chris@bigballofwax.co.nz> --- Pushed to 3.18.x will be in 3.18.6 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13531 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|Academy | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org