https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43776 Bug ID: 43776 Summary: TransformHtmlToMarc should repair and report invalid characters rather than silently stripping them Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Target Milestone: --- Bug 35104 removed the inline StripNonXmlChars calls from C4::Biblio::TransformHtmlToMarc on the assumption that Koha::Biblio::Metadata->store() now strips centrally. TransformHtmlToMarc never calls store(): it builds a MARC::Record from submitted form values, and callers such as cataloguing/addbiblio.pl (FindDuplicate, duplicate redisplay), cataloguing/merge.pl and svc/cataloguing/automatic_linker.pl use that record without saving it. Control characters pasted into the form therefore reached MARC::Record/XML serialisation again, the same failure bug 34549 fixed. The RM follow-up on bug 35104 restored the two StripNonXmlChars calls as a stop-gap. That only covers non-XML characters in the first subfield of a new field and in control fields; it does not cover the other faults Koha::Biblio::Metadata->repair_marcxml handles (e.g. empty datafields), and it silently discards data without telling the cataloguer. Proposal: - Have TransformHtmlToMarc validate its result with the same logic as repair_marcxml instead of stripping inline. - Recoverable faults are repaired and reported to the caller (so the cataloguer can be shown what was changed). - Unrecoverable input throws Koha::Exceptions::Metadata::Invalid, as store() does. - Callers catch the exception and tell the user: addbiblio.pl and merge.pl show the message in the form/merge page, the automatic_linker svc returns a clear error. authorities/ callers need a decision, since authorities do not go through Koha::Biblio::Metadata. Test plan: 1. Apply the patches on top of bug 35104. 2. In the cataloguing editor, paste a control character (e.g. U+001F) into a subfield of a new record, then use Save/Duplicate. Confirm the record is saved or redisplayed with a warning describing what was stripped, and no 500 error. 3. Submit a form producing an empty datafield. Confirm it is repaired and reported. 4. Run prove t/db_dependent/Biblio/TransformHtmlToMarc.t Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.