[Bug 43026] New: Show metadata repair warnings on the biblio detail and MARC detail pages
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Bug ID: 43026 Summary: Show metadata repair warnings on the biblio detail and MARC detail pages Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Depends on: 35104 Target Milestone: --- Bug 35104 records unresolved metadata repair faults (e.g. stripped non-XML characters) in biblio_metadata_errors. Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl, rather than opening it for editing, currently have no way to know it was automatically repaired on its last save. This patch lists any unresolved biblio_metadata_errors as a warning banner on both pages. Depends on bug 35104. Test plan: 1. Apply this patch on top of bug 35104. 2. Using a biblio with at least one unresolved metadata error (see bug 35104's test plan for how to create one), open catalogue/detail.pl?biblionumber=X. Confirm the warning banner listing the fault appears. 3. Open catalogue/MARCdetail.pl?biblionumber=X for the same biblio. Confirm the same warning appears there too. 4. Open a biblio with no metadata errors on either page. Confirm no warning banner appears. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104 [Bug 35104] We should warn when attempting to save MARC records that contain characters invalid in XML -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201703 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201703&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #201703|0 |1 is obsolete| | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #2 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201729 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201729&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |martin.renvoize@openfifth.c |ity.org |o.uk -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #201729|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 203271 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=203271&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Comma delimited| |OpenFifth list of Sponsors| | Sponsorship status|--- |Sponsored QA Contact|testopia@bugs.koha-communit |nick@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #203271|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 204736 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204736&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 204737 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204737&action=edit Bug 43026: Use repair_marcxml to show structured errors when loading an already-corrupt record The biblio detail page load path (unrelated to bug 35104 itself) fell back to a raw StripNonXmlChars + exception dump when a record was already undecodable before it could be displayed. That gave a much less useful message than the per-fault breakdown already shown elsewhere for faults introduced/fixed during a fresh save. Try the same repair_marcxml() used on save first; only fall back to the old StripNonXmlChars + raw exception display if repair_marcxml cannot identify a known-recoverable fault (i.e. the record is genuinely unrecoverable by this mechanism). Also guards the metadata_warning lookups added in the previous commit (both here and in MARCdetail.pl) against a biblio with no metadata row at all, which would otherwise crash the page outright. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #204736|0 |1 is obsolete| | Attachment #204737|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #6 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207695 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207695&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #7 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207696 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207696&action=edit Bug 43026: Use repair_marcxml to show structured errors when loading an already-corrupt record The biblio detail page load path (unrelated to bug 35104 itself) fell back to a raw StripNonXmlChars + exception dump when a record was already undecodable before it could be displayed. That gave a much less useful message than the per-fault breakdown already shown elsewhere for faults introduced/fixed during a fresh save. Try the same repair_marcxml() used on save first; only fall back to the old StripNonXmlChars + raw exception display if repair_marcxml cannot identify a known-recoverable fault (i.e. the record is genuinely unrecoverable by this mechanism). Also guards the metadata_warning lookups added in the previous commit (both here and in MARCdetail.pl) against a biblio with no metadata row at all, which would otherwise crash the page outright. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #8 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Rebased on current main now that bug 35104 is pushed (clean rebase, no content changes). Notes after reviewing the two RM follow-ups on bug 35104: 1. "Restore StripNonXmlChars in TransformHtmlToMarc" does not touch detail.pl or MARCdetail.pl, so it does not interact with this bug. 2. "Fix regressions.t for Metadata's new auto-repair" writes the corrupt metadata straight to the DBIC row, so detail.pl is still exercised with undecodable data. That subtest asserts on p.biberror ("There is at least one encoding error with this bibliographic record..."). With this patch, a record that repair_marcxml can recover sets decoding_errors rather than decoding_error, but the template keeps the p.biberror paragraph in the decoding_errors branch, so the assertion should still hold. I have not yet run t/db_dependent/selenium/regressions.t against this branch; the fixture there only has control fields, so it may not reach the repair branch at all. 3. The RM left "use repair_marcxml in TransformHtmlToMarc" as a follow-up (it works on full MARCXML and can throw). Bug 43643 (empty fields) looks like the natural home for that. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|35104 |43777 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104 [Bug 35104] We should warn when attempting to save MARC records that contain characters invalid in XML https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43777 [Bug 43777] Koha::Biblio::Metadata->record should repair recoverable MARCXML faults on read -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207695|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #9 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207714 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207714&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #10 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207715 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207715&action=edit Bug 43026: Show the repairs made on read on the biblio detail and MARC detail pages The detail page fell back to a raw exception dump plus a hand-rolled StripNonXmlChars when a record was already undecodable, and the MARC detail page had no handling at all. Koha::Biblio::Metadata->record now repairs recoverable faults itself (bug 43777) and reports them through object_messages, so list those messages on both pages. Only metadata that cannot be repaired at all still shows the raw decoding error and degrades the view. Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207696|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|43777 |43025 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 [Bug 43025] Show metadata repair warnings in the cataloguing edit form https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43777 [Bug 43777] Koha::Biblio::Metadata->record should repair recoverable MARCXML faults on read -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207714|0 |1 is obsolete| | Attachment #207715|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #11 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207728 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207728&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #12 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207729 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207729&action=edit Bug 43026: Show the repairs made on read on the biblio detail and MARC detail pages The detail page fell back to a raw exception dump plus a hand-rolled StripNonXmlChars when a record was already undecodable, and the MARC detail page had no handling at all. Koha::Biblio::Metadata->record now repairs recoverable faults itself (bug 43777) and reports them through object_messages, so list those messages on both pages. Only metadata that cannot be repaired at all still shows the raw decoding error and degrades the view. Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207728|0 |1 is obsolete| | Attachment #207729|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #13 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207777 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207777&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. The banner is the shared metadata_repairs.inc (bug 43025), so it looks the same as the one in the cataloguing editor. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #14 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207778 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207778&action=edit Bug 43026: Show the repairs made on read on the biblio detail and MARC detail pages The detail page fell back to a raw exception dump plus a hand-rolled StripNonXmlChars when a record was already undecodable, and the MARC detail page had no handling at all. Koha::Biblio::Metadata->record now repairs recoverable faults itself and saves the repair along with an error row for each fault (bug 43777), so the "Data quality warnings" banner already lists them on every view, not only on the one that did the repair. Both pages read the record before looking up those errors. Only metadata that cannot be repaired at all still shows the raw decoding error and degrades the view. Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Miranda Nero <mnero@oslri.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mnero@oslri.net -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207777|0 |1 is obsolete| | Attachment #207778|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #15 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 209599 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=209599&action=edit Bug 43026: Show metadata repair warnings on the biblio detail and MARC detail pages Cataloguers reviewing a record via catalogue/detail.pl or catalogue/MARCdetail.pl (rather than the edit form) had no way to know a record had been automatically repaired on last save (see bug 35104). List any unresolved biblio_metadata_errors as a warning banner on both pages. The banner is the shared metadata_repairs.inc (bug 43025), so it looks the same as the one in the cataloguing editor. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 --- Comment #16 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 209600 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=209600&action=edit Bug 43026: Show the repairs made on read on the biblio detail and MARC detail pages The detail page fell back to a raw exception dump plus a hand-rolled StripNonXmlChars when a record was already undecodable, and the MARC detail page had no handling at all. Koha::Biblio::Metadata->record now repairs recoverable faults itself and saves the repair along with an error row for each fault (bug 43777), so the "Data quality warnings" banner already lists them on every view, not only on the one that did the repair. Both pages read the record before looking up those errors. Only metadata that cannot be repaired at all still shows the raw decoding error and degrades the view. Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org