[Bug 40272] New: Add an alert for incorrect (MARC21) fixed-length control fields
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Bug ID: 40272 Summary: Add an alert for incorrect (MARC21) fixed-length control fields Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: MARC Bibliographic data support Assignee: koha-bugs@lists.koha-community.org Reporter: m.de.rooy@rijksmuseum.nl QA Contact: testopia@bugs.koha-community.org This proposal adds an alert when opening the MARC basic editor while a control field (005-008) has an incorrect length. Note that some historical bugs may have caused 008 fields to be shorter than 40 positions. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 183640 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183640&action=edit Bug 40272: Add an alert for bad fixed-length control fields Development wise it is much simpler to do this check in perl when opening the editor than getting this logic into the template with javascript. Furthermore, older data may be corrupted, but we trust the current plugins to not insert such fields anymore. Test plan: Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Open an existing record in the basic editor. Shorten 008 without plugin. Save and open the record again. Do you get the alert? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183640|0 |1 is obsolete| | --- Comment #2 from Owen Leonard <oleonard@myacpl.org> --- Created attachment 183641 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183641&action=edit Bug 40272: Add an alert for bad fixed-length control fields Development wise it is much simpler to do this check in perl when opening the editor than getting this logic into the template with javascript. Furthermore, older data may be corrupted, but we trust the current plugins to not insert such fields anymore. Test plan: Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Open an existing record in the basic editor. Shorten 008 without plugin. Save and open the record again. Do you get the alert? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Owen Leonard <oleonard@myacpl.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=40284 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183641|0 |1 is obsolete| | --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 186710 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186710&action=edit Bug 40272: Add an alert for bad fixed-length control fields Development wise it is much simpler to do this check in perl when opening the editor than getting this logic into the template with javascript. Furthermore, older data may be corrupted, but we trust the current plugins to not insert such fields anymore. Test plan: Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Open an existing record in the basic editor. Shorten 008 without plugin. Save and open the record again. Do you get the alert? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@openfifth.c |y.org |o.uk Status|Signed Off |Passed QA CC| |martin.renvoize@openfifth.c | |o.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |25.11.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #4 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This proposal adds an alert release notes| |when opening the MARC basic | |editor while a control | |field (005-008) has an | |incorrect length. Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #5 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- This isn't ready for inclusion. With this patchset applied I cannot create a bib with the basic editor: Can't call method "field" on an undefined value at /kohadevbox/koha/Koha/Biblio/Metadata/Extractor/MARC/MARC21.pm line 133 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|25.11.00 | released in| | Status|Pushed to main |Failed QA --- Comment #6 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- I've reverted this one. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Lucas Gass (lukeg) from comment #6)
I've reverted this one.
Taking a look. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186710|0 |1 is obsolete| | --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 187959 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187959&action=edit Bug 40272: Add an alert for bad fixed-length control fields Development wise it is much simpler to do this check in perl when opening the editor than getting this logic into the template with javascript. Furthermore, older data may be corrupted, but we trust the current plugins to not insert such fields anymore. Test plan: Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Open an existing record in the basic editor. Shorten 008 without plugin. Save and open the record again. Do you get the alert? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 187960 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187960&action=edit Bug 40272: (QA follow-up) Fix new biblio/empty metadata case Test plan: Add new biblio record in basic editor. Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Hi Lucas, Good catch. The new biblio case should surely have been included in the original test plan.. I did a bit more than adding the strictly needed "if $biblio"-check in the addbiblio script on the follow-up. I also added a check too in the extractor code on having a MARC::Record object. Actually this should imo be in Extractor::MARC->metadata. But that should enlarge the scope of this report too much for now. Will open a new report for that. And added both cases in unit tests: metadata is just a string, metadata is an empty MARC::Record object. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41031 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41031 [Bug 41031] Extractor::MARC->metadata does not check if metadata is a MARC::Record -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187960|0 |1 is obsolete| | --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 187961 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187961&action=edit Bug 40272: (QA follow-up) Fix new biblio/empty metadata case Test plan: Add new biblio record in basic editor. Run t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Tidied follow-up -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |BLOCKED --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Still looking here -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187961|0 |1 is obsolete| | --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 187978 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187978&action=edit Bug 40272: (QA follow-up) Only add a biblio test And move the comment where it belongs. More changes on 41031. Test plan: Add a new biblio record with basic editor. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #10)
Hi Lucas,
Good catch. The new biblio case should surely have been included in the original test plan..
I did a bit more than adding the strictly needed "if $biblio"-check in the addbiblio script on the follow-up. I also added a check too in the extractor code on having a MARC::Record object. Actually this should imo be in Extractor::MARC->metadata. But that should enlarge the scope of this report too much for now. Will open a new report for that. And added both cases in unit tests: metadata is just a string, metadata is an empty MARC::Record object.
Sorry for the noise here. But I finally decided not to include more changes but do a quick fix here. And add more changes on the new report. Adjusting the check_fixed_length too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #16 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #15)
And add more changes on the new report. Adjusting the check_fixed_length too.
=> Adjusting the check_fixed_length tests too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |25.11.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #17 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Laura Escamilla <Laura.escamilla@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|25.11.00 |25.11.00, 25.05.08 released in| | Status|Pushed to main |Needs documenting CC| |Laura.escamilla@bywatersolu | |tions.com --- Comment #18 from Laura Escamilla <Laura.escamilla@bywatersolutions.com> --- Backported to 25.05 thanks all! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|25.11.00, 25.05.08 |25.11.00,25.05.08 released in| | CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 --- Comment #19 from Fridolin Somers <fridolin.somers@biblibre.com> --- Enhancement not pushed to 24.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |42261 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42261 [Bug 42261] Inaccurate detection of character-length errors in fixed marc fields 006/007 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=42262 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |42512 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42512 [Bug 42512] Marc control field length detection prevents editing of records with invalid MARCXML -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40272 Miranda Nero <mnero@oslri.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mnero@oslri.net -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org