[Bug 43025] New: Show metadata repair warnings in the cataloguing edit form
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Bug ID: 43025 Summary: Show metadata repair warnings in the cataloguing edit form Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Depends on: 35104 Target Milestone: --- Bug 35104 makes Koha::Biblio::Metadata->store() recover non-XML characters and empty datafields automatically (recording what was fixed in biblio_metadata_errors), or throw Koha::Exceptions::Metadata::Invalid when a record's MARCXML cannot be saved at all. This patch surfaces that in the place cataloguers actually hit it: the cataloguing edit form. This patch: - Catches Koha::Exceptions::Metadata::Invalid on save in cataloguing/addbiblio.pl and shows a clear error instead of a raw exception, keeping the form populated so nothing is lost. - Shows a "Record saved with modifications" warning when a save succeeded but required automatic repair. - Lists any pre-existing, unresolved metadata errors when opening a record for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action. - Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to resolve (delete) a metadata error row, used by the "Mark resolved" action. Depends on bug 35104. Test plan: 1. Apply this patch on top of bug 35104. 2. Catalogue a new record (or edit an existing one) and add a control character (e.g. paste text containing a Unit Separator, 0x1F) into a subfield value. Save the record. 3. Confirm the record saves successfully and a "Record saved with modifications" warning banner appears describing what was automatically fixed. 4. Re-open the record for editing. Confirm a "Data quality warnings" banner lists the fault, with a "Go to field" link and a "Mark resolved" button. 5. Click "Go to field" - confirm it switches to the correct tab and scrolls to the affected field. 6. Click "Mark resolved" - confirm the warning disappears immediately and does not reappear after reloading the page. 7. Save a record that cannot be repaired at all (e.g. temporarily force Koha::Biblio::Metadata::store to throw Koha::Exceptions::Metadata::Invalid, or use a MARC::File::XML error you can reproduce) - confirm a clear error message is shown and the form is redisplayed with your data intact, rather than a raw exception or a 500 error. 8. prove t/db_dependent/api/v1/biblios_metadata_errors.t Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104 [Bug 35104] We should warn when attempting to save MARC records that contain characters invalid in XML -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201701 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201701&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #2 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201702 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201702&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #201701|0 |1 is obsolete| | Attachment #201702|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201727 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201727&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201728 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201728&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |martin.renvoize@openfifth.c |ity.org |o.uk -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com Status|Signed Off |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Signed Off Comma delimited| |OpenFifth list of Sponsors| | Sponsorship status|--- |Sponsored -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #201727|0 |1 is obsolete| | Attachment #201728|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 203269 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=203269&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #6 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 203270 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=203270&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #7 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 203656 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=203656&action=edit Bug 43025: Handle encoding errors when viewing and loading records This patch updates the pre-existing handling of errors to use the new recovery routine and display all of the errors correctly formatted Ultimately - fix_marcxml should probably contain the code from store that generates the errors into the DB table, but this a large tree and we should polish once it is in The other patches here are useful, however, I can't actually manage to enter invalid characters in either editor -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #8 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Hi Nick, Thanks for digging into this and leaving a patch — genuinely useful QA, and I agree with the underlying idea (using repair_marcxml() for a proper per-fault breakdown instead of the older record_strip_nonxml + raw exception dump on load). I didn't apply it as-is though, for a few reasons: - There's a typo in the template: nonxml_stripped.char_od should be char_ord — that's the actual key _find_nonxml_chars() returns, so that field would have rendered blank. - repair_marcxml() returns an empty list when it can't identify a known-recoverable fault (i.e. the record is broken in some way other than stray non-XML characters or empty datafields). In that case the patch left $record undef with nothing downstream to catch it. I kept the original record_strip_nonxml fallback for that case, so a genuinely unrecoverable record still degrades gracefully instead of crashing the page. - The patch touched both cataloguing/addbiblio.pl (this bug) and catalogue/detail.pl (bug 43026's territory), so I split it across the two bugs it actually belongs to. I've reworked and applied both halves with those fixes, and credited your patch in the commit messages (Based-on-patch-by:). On your broader point — moving the error-recording itself into a shared routine so a plain load/view also persists to biblio_metadata_errors, not just a save — I agree that's worth doing, but think it deserves its own bug: it means a GET request writing to the DB (locking/concurrency implications on a hot page) and touching Koha::Biblio::Metadata itself rather than any of these follow-up bugs. Happy to file that separately, or if you'd rather pick it up yourself, go for it. Will get updated patches up on this bug and 43026 shortly for another look whenever you have time. Thanks again, -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #203269|0 |1 is obsolete| | Attachment #203270|0 |1 is obsolete| | Attachment #203656|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #9 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 204733 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204733&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #10 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 204734 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204734&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #11 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 204735 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204735&action=edit Bug 43025: Use repair_marcxml to show structured errors when loading an already-corrupt record The cataloguing edit form load path (unrelated to bug 35104 itself) fell back to record_strip_nonxml + a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced/fixed during a fresh save. Try the same repair_marcxml() used on save first; only fall back to the old record_strip_nonxml + raw exception display if repair_marcxml cannot identify a known-recoverable fault (i.e. the record is genuinely unrecoverable by this mechanism). Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |nick@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #204733|0 |1 is obsolete| | Attachment #204734|0 |1 is obsolete| | Attachment #204735|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #12 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 206842 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206842&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #13 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 206843 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206843&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #14 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 206844 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206844&action=edit Bug 43025: Use repair_marcxml to show structured errors when loading an already-corrupt record The cataloguing edit form load path (unrelated to bug 35104 itself) fell back to record_strip_nonxml + a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced/fixed during a fresh save. Try the same repair_marcxml() used on save first; only fall back to the old record_strip_nonxml + raw exception display if repair_marcxml cannot identify a known-recoverable fault (i.e. the record is genuinely unrecoverable by this mechanism). Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #15 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Passing QA here - I couldn't actually add errors via cataloging, however, I forced some into the background: INSERT INTO biblio_metadata_errors (metadata_id,error_type,tag,subfield,message) VALUES (368,'nonxml_chars',520,"a","Ugly chars"),(368,'empty_datafields',515,"b","Empty headed"); I also tested editing record with empty_fields and bad characters and both worked I filed two follow-up bugs which need to be addressed: Display warnings in the advanced editor: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43642 Cannot load records with empty fields: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43643 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43642 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43642 [Bug 43642] Show metadata repair warnings in the advanced cataloguing editor -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #206842|0 |1 is obsolete| | Attachment #206843|0 |1 is obsolete| | Attachment #206844|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #16 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207692 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207692&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #17 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207693 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207693&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #18 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207694 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207694&action=edit Bug 43025: Use repair_marcxml to show structured errors when loading an already-corrupt record The cataloguing edit form load path (unrelated to bug 35104 itself) fell back to record_strip_nonxml + a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced/fixed during a fresh save. Try the same repair_marcxml() used on save first; only fall back to the old record_strip_nonxml + raw exception display if repair_marcxml cannot identify a known-recoverable fault (i.e. the record is genuinely unrecoverable by this mechanism). Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #19 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Just a quick rebase to keep it applying -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|35104 |43777 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104 [Bug 35104] We should warn when attempting to save MARC records that contain characters invalid in XML https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43777 [Bug 43777] Koha::Biblio::Metadata->record should repair recoverable MARCXML faults on read -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207692|0 |1 is obsolete| | Attachment #207693|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #20 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207711 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207711&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #21 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207712 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207712&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #22 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207713 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207713&action=edit Bug 43025: Show the repairs made on read when loading an already-corrupt record The cataloguing edit form load path fell back to record_strip_nonxml and a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced or fixed during a fresh save. Koha::Biblio::Metadata->record now repairs recoverable faults itself (bug 43777) and reports them through object_messages, so list those messages to the cataloguer. Only metadata that cannot be repaired at all still ends up as an error, and the message now says the record cannot be loaded rather than claiming characters were stripped. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207694|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43026 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43026 [Bug 43026] Show metadata repair warnings on the biblio detail and MARC detail pages -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43027 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43027 [Bug 43027] Handle undecodable metadata gracefully when merging bibliographic records -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43028 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43028 [Bug 43028] Isolate one undecodable record from aborting a batch import or revert -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43029 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43029 [Bug 43029] Guard maintenance scripts against biblios with corrupt metadata -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43776 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43776 [Bug 43776] TransformHtmlToMarc should repair and report invalid characters rather than silently stripping them -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207711|0 |1 is obsolete| | Attachment #207712|0 |1 is obsolete| | Attachment #207713|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #23 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207725 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207725&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #24 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207726 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207726&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #25 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207727 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207727&action=edit Bug 43025: Show the repairs made on read when loading an already-corrupt record The cataloguing edit form load path fell back to record_strip_nonxml and a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced or fixed during a fresh save. Koha::Biblio::Metadata->record now repairs recoverable faults itself (bug 43777) and reports them through object_messages, so list those messages to the cataloguer. Only metadata that cannot be repaired at all still ends up as an error, and the message now says the record cannot be loaded rather than claiming characters were stripped. The list is rendered by a new metadata_repairs.inc, so the other pages that need to show the same thing (see the bugs depending on this one) can share it instead of each carrying their own copy. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207725|0 |1 is obsolete| | Attachment #207726|0 |1 is obsolete| | Attachment #207727|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #26 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207774 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207774&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #27 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207775 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207775&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #28 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 207776 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207776&action=edit Bug 43025: Show the repairs made on read when loading an already-corrupt record The cataloguing edit form load path fell back to record_strip_nonxml and a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced or fixed during a fresh save. Koha::Biblio::Metadata->record now repairs recoverable faults itself and saves the repair along with an error row for each fault (bug 43777), so the "Data quality warnings" list, with its links to the affected fields and its "Mark resolved" buttons, already shows them on every load, not only on the one that did the repair. Drop the separate, transient list built from object_messages, which would only repeat them, and say in the notice that the problems were repaired when saved or loaded. The list is now rendered by a new metadata_repairs.inc, taking the error rows directly, so the other pages that need to show the same thing (see the bugs depending on this one) can share it instead of each carrying their own copy. It shows one row per fault: the field, what was wrong, and where, with the offending character and its pointer kept on their own lines so it is still clear where it was. Only metadata that cannot be repaired at all still ends up as an error, and the message says the record cannot be loaded rather than claiming characters were stripped. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43021 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43021 [Bug 43021] A single malformed Z39.50/SRU result can break search and staged record preview -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Miranda Nero <mnero@oslri.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mnero@oslri.net -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207774|0 |1 is obsolete| | Attachment #207775|0 |1 is obsolete| | Attachment #207776|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #29 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 208942 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208942&action=edit Bug 43025: Add REST endpoint to resolve a biblio metadata error Adds DELETE /api/v1/biblios/{biblio_id}/metadata/errors/{error_id} to mark a recorded metadata fault (see bug 35104) as resolved. Deletion is scoped through the biblio's own metadata_errors relation, so an error_id belonging to a different biblio correctly 404s rather than being deletable cross-biblio. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #30 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 208943 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208943&action=edit Bug 43025: Show metadata repair warnings and existing errors in the cataloguing edit form Surfaces bug 35104's exception/recovery behaviour where cataloguers actually encounter it - saving a record: - If the record is unrecoverable, AddBiblio/ModBiblio's Koha::Exceptions::Metadata::Invalid is caught and shown instead of a generic error, with the form re-displayed so nothing is lost. - If the record was recoverable but modified, the save proceeds and a warning banner explains what was automatically fixed (via the new warnings option on AddBiblio/ModBiblio). - Any pre-existing (not yet resolved) errors on the record are listed when opening it for editing, each with a quick-link to the affected field (where the field survived the repair) and a "Mark resolved" action calling the new REST DELETE endpoint. Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com> Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com> Sponsored-by: OpenFifth Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43025 --- Comment #31 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 208944 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208944&action=edit Bug 43025: Show the repairs made on read when loading an already-corrupt record The cataloguing edit form load path fell back to record_strip_nonxml and a raw exception dump when a record was already undecodable before it could be opened for editing. That gave a much less useful message than the per-fault breakdown this bug already shows for faults introduced or fixed during a fresh save. Koha::Biblio::Metadata->record now repairs recoverable faults itself and saves the repair along with an error row for each fault (bug 43777), so the "Data quality warnings" list, with its links to the affected fields and its "Mark resolved" buttons, already shows them on every load, not only on the one that did the repair. Drop the separate, transient list built from object_messages, which would only repeat them, and say in the notice that the problems were repaired when saved or loaded. The list is now rendered by a new metadata_repairs.inc, taking the error rows directly, so the other pages that need to show the same thing (see the bugs depending on this one) can share it instead of each carrying their own copy. It shows one row per fault: the field, what was wrong, and where, with the offending character and its pointer kept on their own lines so it is still clear where it was. Only metadata that cannot be repaired at all still ends up as an error, and the message says the record cannot be loaded rather than claiming characters were stripped. Based-on-patch-by: Nick Clemens <nick@bywatersolutions.com> Sponsored-by: OpenFifth Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org