New subject: [Bug 43028] Isolate one undecodable record from aborting a batch import or revert

8 Jul 2026

      https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43028

            Bug ID: 43028
           Summary: Isolate one undecodable record from aborting a batch
                    import or revert
   Initiative type: ---
        Sponsorship ---
            status:
           Product: Koha
           Version: Main
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Tools
          Assignee: koha-bugs@lists.koha-community.org
          Reporter: martin.renvoize@openfifth.co.uk
        QA Contact: testopia@bugs.koha-community.org
        Depends on: 35104
  Target Milestone: ---

Bug 35104 makes Koha::Biblio::Metadata->store() throw
Koha::Exceptions::Metadata::Invalid for MARCXML it cannot save. Batch
import (C4::ImportBatch::BatchCommitRecords/_batchCommitItems), batch
revert (BatchRevertRecords), the corresponding background jobs, the
CLI tools built on them (misc/commit_file.pl,
misc/migration_tools/bulkmarcimport.pl), and
Koha::Import::Record::get_marc_record all needed updating so one bad
record doesn't take the rest of the batch down with it.

This patch:
- Catches the exception per-record in BatchCommitRecords (which
  already commits per-record) and per-item in _batchCommitItems,
  flagging the record/item with an 'error' status and message and
  counting it via a new $num_errored return value, instead of letting
  it propagate.
- Does the same for BatchRevertRecords, which runs as a single
  whole-batch transaction: both the marcxml_old decode step and the
  ModBiblio/ModAuthority call that follows it are isolated, so one
  record that can't be restored doesn't roll back every other record
  already reverted in the same batch.
- Threads the new error counts through
  Koha::BackgroundJob::MARCImportCommitBatch/RevertBatch's job report,
  and counts errored records towards job progress so a batch
  containing them doesn't get stuck showing as still running.
- Reports the new counts in misc/commit_file.pl's CLI summary and logs
  a warning line for records automatically repaired during
  bulkmarcimport.pl.
- Surfaces each errored row's reason as a tooltip on its "Error"
  status badge in the manage-marc-import listing.
- Makes Koha::Import::Record::get_marc_record reuse
  Koha::Biblio::Metadata::repair_marcxml (bug 35104) instead of its
  own weaker strip-and-retry, so staged import records get the same
  recovery (including empty-datafield handling) as the interactive
  cataloguing form.

Depends on bug 35104.

Test plan:
1. Apply this patch on top of bug 35104.
2. Stage a MARC file for import (Tools > Stage MARC records for
   import) containing at least one deliberately malformed record
   (e.g. a datafield with no subfields, or invalid control characters
   that cannot be repaired) alongside normal records.
3. From Manage staged records, commit the batch. Confirm: the bad
   record is flagged with an 'error' status and a tooltip explaining
   why; every other record in the batch imports successfully; the job
   report/CLI summary shows a "Number of records with errors" count.
4. Revert the same batch. Confirm one record whose marcxml_old cannot
   be restored does not prevent the rest of the batch from reverting,
   and the job report reflects the error count.
5. Run misc/migration_tools/bulkmarcimport.pl from the CLI against a
   file containing a record with a recoverable fault (e.g. a
   non-XML character). Confirm a WARNING line is logged for that
   record and the import otherwise completes normally.
6. prove t/db_dependent/ImportBatch.t
   t/db_dependent/Koha/BackgroundJob/MARCImportCommitBatch.t

Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104
[Bug 35104] We should warn when attempting to save MARC records that contain
characters invalid in XML
-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.

[Bug 43028] New: Isolate one undecodable record from aborting a batch import or revert

tags

participants (1)