[Bug 17088] New: Bad MARC XML can halt export_records.pl
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Bug ID: 17088 Summary: Bad MARC XML can halt export_records.pl Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: gmcharlt@gmail.com Reporter: kyle@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz If export_records.pl encounters a record that fails to load correctly as a MARC::Record the export process will fail and only the records encountered up to that point will be exported. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #1 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 54202 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=54202&action=edit Bug 17088 - Bad MARC XML can halt export_records.pl If export_records.pl encounters a record that fails to load correctly as a MARC::Record the export process will fail and only the records encountered up to that point will be exported. Test Plan: 1) Pick a record in your database, and replace the biblioitems.marcxml field with some random characters 2) Export your records using the command: misc/export_records.pl --format xml --record-type bibs --filename /tmp/biblios.xml 3) Note you get an error and the script ends prematurely 4) Apply this patch 5) Repeat step 2 6) Note the full record set is exported ( except for your bad record ) 7) Note an error is still generated for your bad record ( useful for tracking down bad records ). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Testing this now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #54202|0 |1 is obsolete| | --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 55609 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=55609&action=edit Bug 17088 - Bad MARC XML can halt export_records.pl If export_records.pl encounters a record that fails to load correctly as a MARC::Record the export process will fail and only the records encountered up to that point will be exported. Test Plan: 1) Pick a record in your database, and replace the biblioitems.marcxml field with some random characters 2) Export your records using the command: misc/export_records.pl --format xml --record-type bibs --filename /tmp/biblios.xml 3) Note you get an error and the script ends prematurely 4) Apply this patch 5) Repeat step 2 6) Note the full record set is exported ( except for your bad record ) 7) Note an error is still generated for your bad record ( useful for tracking down bad records ). Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 55610 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=55610&action=edit Bug 17088: [Follow-up] Adjust POD in Logger.pm Not important enough to go on its own. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 55611 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=55611&action=edit Bug 17088: [Follow-up] Use Logger for failed exports Fixes a TODO for logging unsupported record_type in _get_record_for_export. Also logs a warning when the record_type parameter is not supplied at all in sub export. Replaces a warn by a log message about an invalid record for format iso2709. Also adds a log message about an invalid record for format xml. Adds a trivial unit test for passing no record_type to export. Test plan: See also first patch. Run t/db_dependent/Exporter/Record.t. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Also tested the log messages for iso2709 and xml by manipulating the record_type parameter with: $params->{record_type}='xx'; before calling _get_record_for_export in Record.pm. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Kyle: Added follow-up for logging. If you prefer to let them go on their own, feel free to move them. They are quite simple however. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|unspecified |master Patch complexity|Trivial patch |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #55609|0 |1 is obsolete| | Attachment #55610|0 |1 is obsolete| | Attachment #55611|0 |1 is obsolete| | --- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 56032 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=56032&action=edit Bug 17088 - Bad MARC XML can halt export_records.pl If export_records.pl encounters a record that fails to load correctly as a MARC::Record the export process will fail and only the records encountered up to that point will be exported. Test Plan: 1) Pick a record in your database, and replace the biblioitems.marcxml field with some random characters 2) Export your records using the command: misc/export_records.pl --format xml --record-type bibs --filename /tmp/biblios.xml 3) Note you get an error and the script ends prematurely 4) Apply this patch 5) Repeat step 2 6) Note the full record set is exported ( except for your bad record ) 7) Note an error is still generated for your bad record ( useful for tracking down bad records ). Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #8 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 56033 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=56033&action=edit Bug 17088: [Follow-up] Adjust POD in Logger.pm Not important enough to go on its own. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 56034 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=56034&action=edit Bug 17088: [Follow-up] Use Logger for failed exports Fixes a TODO for logging unsupported record_type in _get_record_for_export. Also logs a warning when the record_type parameter is not supplied at all in sub export. Replaces a warn by a log message about an invalid record for format iso2709. Also adds a log message about an invalid record for format xml. Adds a trivial unit test for passing no record_type to export. Test plan: See also first patch. Run t/db_dependent/Exporter/Record.t. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Also tested the log messages for iso2709 and xml by manipulating the record_type parameter with: $params->{record_type}='xx'; before calling _get_record_for_export in Record.pm. Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 56035 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=56035&action=edit Bug 17088: Add tests to cover the changes Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Brendan Gallagher <brendan@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |brendan@bywatersolutions.co | |m --- Comment #11 from Brendan Gallagher <brendan@bywatersolutions.com> --- Pushed to Master - Should be in the November Release 16.11 - Thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |frederic@tamil.fr --- Comment #12 from Frédéric Demians <frederic@tamil.fr> --- Pushed in 16.05. Will be in 16.05.05. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com --- Comment #13 from Julian Maurice <julian.maurice@biblibre.com> --- Pushed to 3.22.x, will be in 3.22.12 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17088 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=17196 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org