https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41927 Bug ID: 41927 Summary: bulkmarcimport stops if encounters bad record Initiative type: --- Sponsorship --- status: Product: Koha Version: 24.11 Hardware: All OS: All Status: NEW Severity: minor Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: bernard.scaife@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz Created attachment 193837 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=193837&action=edit Single badly encoded record example If I have a large file (300k records) to load via bulkmarcimport, it invariably contains a few bad records. The --skip_bad_records does not capture all these and "skip" them. Instead, with some, it reports an error and either stops prematurely or hangs in a loop without being able to progress beyond the offending record until the script is killed. Note that it runs ok in test mode (and identifies the "Bad" record). My only recourse is to extract the offending record(s) from the file beforehand which is very time consuming. What I'd prefer in an ideal world is that these records are sanitized, having captured the error in the log file for later rectification if required. As a bare minimum, I need the script to at least be able to handle this as a "skip" as advertised with --skip_bad_records so that it carries on with the rest of the load. I attach an example of one of the bad records that tripped me up with this: /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -commit 1000 -file /kohadevbox/koha/rogue.mrc -l -v -v -sk -si Characteristic MARC flavour: MARC21 no mapping found for [0x8] at position 11 in t $xtac ar\ $btxt V $2rdac ontent g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308, <GEN3> line 1. :36: parser error : PCDATA invalid Char value 8 <subfield code="a">t $xtac ar\ $btxt V $2rdac ontent</subfield> ^ at /kohadevbox/koha/Koha/Biblio/Metadata.pm line 117, <GEN3> line 1. Invalid data, cannot decode metadata object (biblio_metadata.id=453, biblionumber=509, format=marcxml, schema=MARC21, decoding_error=':36: parser error : PCDATA invalid Char value 8 <subfield code="a">t $xtac ar\ $btxt V $2rdac ontent</subfield> ^')kohadev-koha@kohadevbox:koha(main)$ -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.