[Bug 25600] New: Koha doesn't check for warnings when parsing (ISO2709) MARC
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 Bug ID: 25600 Summary: Koha doesn't check for warnings when parsing (ISO2709) MARC Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: MARC Bibliographic record staging/import Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au QA Contact: testopia@bugs.koha-community.org I've noticed that Koha doesn't check for warnings when parsing MARC, particularly ISO2709 MARC, which is a more fragile format than MARCXML. We should be using https://metacpan.org/pod/MARC::Record#warnings() to at the very least warn users that there was an issue with their (ISO2709) MARC import. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Recently, I converted some English/Chinese records in ISO2709 MARC from the Chinese GB2312 encoding to UTF-8 and imported them into Koha. At first, it seemed fine. There were no visible warnings or errors. However, when I looked at the records, it was obvious that many of the Chinese strings were truncated. I figured out that the problem was that MARC::File::USMARC reads the ISO2709 format precisely, which was a problem as I was converting from a 2-byte encoding to a 3-byte encoding. The ISO2709 MARC record label and directory were never going to be correct in that scenario. But Koha didn't complain. So I used MARC::File::USMARC on my own and looked at $record->warnings(), and I saw an explosion of problems with the parsing of my ISO2709 MARC records. In the end, I did a workaround by converting from GB2312 to UTF8 using iconv, opened the ISO2709 MARC in MarcEdit which converted it to its own "MarcBreaker" MRK format, and then exported that as MARCXML. Evidently MarcEdit's MarcBreaker is more skeptical of the ISO2709 MARC format and parses the data more carefully. I am tempted to submit a patch to MARC::File::USMARC to have an option for more cautious parsing, but in the meantime I think we should at least raise the parsing warnings in Koha? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Likewise on Bug 17842, I think Katrin was importing a Latin-1 encoded ISO2709 MARC file and Koha didn't complain. It just truncated fields, broke diacritics, etc. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=17842 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- In C4::Charset::MarcToUTF8Record, we look for fatal errors, but not non-fatal warnings, which can often be just as bad if not worse... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25600 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38416 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org