https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43028 Bug ID: 43028 Summary: Isolate one undecodable record from aborting a batch import or revert Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org Depends on: 35104 Target Milestone: --- Bug 35104 makes Koha::Biblio::Metadata->store() throw Koha::Exceptions::Metadata::Invalid for MARCXML it cannot save. Batch import (C4::ImportBatch::BatchCommitRecords/_batchCommitItems), batch revert (BatchRevertRecords), the corresponding background jobs, the CLI tools built on them (misc/commit_file.pl, misc/migration_tools/bulkmarcimport.pl), and Koha::Import::Record::get_marc_record all needed updating so one bad record doesn't take the rest of the batch down with it. This patch: - Catches the exception per-record in BatchCommitRecords (which already commits per-record) and per-item in _batchCommitItems, flagging the record/item with an 'error' status and message and counting it via a new $num_errored return value, instead of letting it propagate. - Does the same for BatchRevertRecords, which runs as a single whole-batch transaction: both the marcxml_old decode step and the ModBiblio/ModAuthority call that follows it are isolated, so one record that can't be restored doesn't roll back every other record already reverted in the same batch. - Threads the new error counts through Koha::BackgroundJob::MARCImportCommitBatch/RevertBatch's job report, and counts errored records towards job progress so a batch containing them doesn't get stuck showing as still running. - Reports the new counts in misc/commit_file.pl's CLI summary and logs a warning line for records automatically repaired during bulkmarcimport.pl. - Surfaces each errored row's reason as a tooltip on its "Error" status badge in the manage-marc-import listing. - Makes Koha::Import::Record::get_marc_record reuse Koha::Biblio::Metadata::repair_marcxml (bug 35104) instead of its own weaker strip-and-retry, so staged import records get the same recovery (including empty-datafield handling) as the interactive cataloguing form. Depends on bug 35104. Test plan: 1. Apply this patch on top of bug 35104. 2. Stage a MARC file for import (Tools > Stage MARC records for import) containing at least one deliberately malformed record (e.g. a datafield with no subfields, or invalid control characters that cannot be repaired) alongside normal records. 3. From Manage staged records, commit the batch. Confirm: the bad record is flagged with an 'error' status and a tooltip explaining why; every other record in the batch imports successfully; the job report/CLI summary shows a "Number of records with errors" count. 4. Revert the same batch. Confirm one record whose marcxml_old cannot be restored does not prevent the rest of the batch from reverting, and the job report reflects the error count. 5. Run misc/migration_tools/bulkmarcimport.pl from the CLI against a file containing a record with a recoverable fault (e.g. a non-XML character). Confirm a WARNING line is logged for that record and the import otherwise completes normally. 6. prove t/db_dependent/ImportBatch.t t/db_dependent/Koha/BackgroundJob/MARCImportCommitBatch.t Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104 [Bug 35104] We should warn when attempting to save MARC records that contain characters invalid in XML -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.