[Bug 38933] New: Some oversized records with UTF-8 characters cause import worker to die
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 Bug ID: 38933 Summary: Some oversized records with UTF-8 characters cause import worker to die Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: MARC Bibliographic record staging/import Assignee: koha-bugs@lists.koha-community.org Reporter: januszop@gmail.com QA Contact: testopia@bugs.koha-community.org Some oversized records with UTF-8 characters cause import worker to die. The record would be staged correctly, but then an attempt to import it with C4::ImportBatch::BatchCommitRecords can generate an uncaught die of new_from_usmarc. A solution would be to make use of import_records.marcxml instead of import_records.marc for this step. (Both versions are stored in the import_records table in stage step). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 --- Comment #1 from Janusz Kaczmarek <januszop@gmail.com> --- Created attachment 176845 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=176845&action=edit Test record A test MARCXML record to confirm the issue. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #2 from David Cook <dcook@prosentient.com.au> --- I was actually thinking about this a bit yesterday when fixing bug 38913. There are other places that call "new_from_usmarc". Some of them - like the API - have try/catch around them I think, but I'm sure not all the calls do. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 --- Comment #3 from Janusz Kaczmarek <januszop@gmail.com> --- (In reply to David Cook from comment #2)
I was actually thinking about this a bit yesterday when fixing bug 38913.
There are other places that call "new_from_usmarc". Some of them - like the API - have try/catch around them I think, but I'm sure not all the calls do.
Should be investigated from this angle. But the current can be solved with marcxml that we have anyway. No need to try here around new_from_usmarc if we can effectively import such records with new_from_xml IMO. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=35104 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- Bumped into this one with ./misc/migration_tools/bulkmarcimport.pl In our case, I think it's bad leader data, but $record->as_usmarc is what is killing us. In C4::Biblio::ModBiblioMarc there is a call to $record->as_usmarc to re-calculate the record length, and that was throwing a fatal error, so I've wrapped that one in an eval{}... It looks like bug 38913 will still fall victim to this one too because of $record->as_usmarc used there too. I feel like some people reported issues even after bug 38913 with elastic indexing and maybe that's why... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38933 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=40326 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org