[Bug 5332] New: Add batch reversion capability to bin/commit_biblios_file.pl
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Summary: Add batch reversion capability to bin/commit_biblios_file.pl Change sponsored?: --- Product: Koha Version: rel_3_2 Platform: All OS/Version: All Status: NEW Severity: enhancement Priority: P5 Component: Tools AssignedTo: gmcharlt@gmail.com ReportedBy: dpk@randomnotes.org QAContact: koha-bugs@lists.koha-community.org Estimated Hours: 0.0 Created attachment 2724 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=2724 version with new functionality commit_biblios_file.pl needs a way to revert a batch in bulk. I have done this code and will attache it to bug for review and potential inclusion. The basis for this enhancement was the a bad batch of MARC records that had some unknown Unicode characters in them which cause the import function from the web interface to fail after importing a portion of the records. At that point you are completely stuck. You can't complete the import due to the errors and there is no way to *revert* either. You are stuck with a half imported batch. My change is to add a --revert flag to commit_biblios_file.pl which provides an interface to the pre-existing BatchRevertBibRecords function. I have also added a --progress-interval flag to allow the default commit interval to be changed to aid in debugging bad import batches. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Doug Kingston <dpk@randomnotes.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 |P3 -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #1 from Doug Kingston <dpk@randomnotes.org> 2010-10-31 12:53:16 UTC --- Created attachment 2725 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=2725 Proposed patch -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chris@bigballofwax.co.nz --- Comment #2 from Chris Cormack <chris@bigballofwax.co.nz> 2010-11-08 02:38:10 UTC --- Pushed to new/enh/bug_5332 along with Galens follow up patches, please test and report so I can merge to master -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #3 from Doug Kingston <dpk@randomnotes.org> 2010-11-10 23:01:10 UTC --- (In reply to comment #2)
Pushed to new/enh/bug_5332 along with Galens follow up patches, please test and report so I can merge to master
I'm new to this process. Do you need me to test this? If so, I'm not sure how to work with the referenc new/enh/bug_5332. Pointers welcome. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #4 from Chris Cormack <chris@bigballofwax.co.nz> 2010-11-12 02:42:44 UTC --- Doug, are you running a development server or a test server anywhere? If so, and if it is running from git, you can do this git checkout -b bug_5332 origin/new/enh/bug_5332 And you will now be running that branch. If you aren't thats fine, just let us know and we will get someone else to test. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #5 from Doug Kingston <dpk@randomnotes.org> 2010-11-12 23:47:50 UTC --- (In reply to comment #4)
Doug, are you running a development server or a test server anywhere?
If so, and if it is running from git, you can do this
git checkout -b bug_5332 origin/new/enh/bug_5332
And you will now be running that branch. If you aren't thats fine, just let us know and we will get someone else to test.
Sorry, at the moment I am running a stock 3.2. We don't have a QA environment yet. I'll try to get more organized and git-aware as I make more changes, but for now I think it will be faster for someone else. Do you need a duff marc import file (with some broken unicode in it) to create a broken import? -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #6 from Chris Cormack <chris@bigballofwax.co.nz> 2010-11-13 20:21:21 UTC --- Yes please a attach a file that breaks it currently, and we can use that to test that it is now fixed. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #7 from Doug Kingston <dpk@randomnotes.org> 2010-11-13 22:55:43 UTC --- Created attachment 2747 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=2747 MARC import file with bad unicode characters - won't fully import Gzip'd MARC21 import file. Use this to trigger a partially imported batch that will not import and cannot be reverted (because its not been fully imported - This is probably a bug in and of itself!). There are 1443 MARC records in this file. I believe 28 are bad. The first bad entry in about record 9. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Doug Kingston <dpk@randomnotes.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #2747|0 |1 is obsolete| | Attachment #2747|MARC import file with bad |Wrong file - please ignore description|unicode characters - won't | |fully import | -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #8 from Doug Kingston <dpk@randomnotes.org> 2010-11-14 10:25:58 UTC --- Created attachment 2748 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=2748 Gzip of MARC import file with bad records This gzip'd MARC21 import file has some bad unicode characters in it that will cause the import to abort part way through. At that point the batch import is stuck. It can't complete due to the errors and cannot be reverted from the web UI since it has not completed. (take8.marc.gz replaces take7.marc.gz which was erroneously uploaded yesterday.) -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #9 from Doug Kingston <dpk@randomnotes.org> 2010-11-16 22:32:15 UTC --- Now I am not sure which one was the problem import file (take7 - now marked obsolete, or take 8). But its one of them. I will try to get a test system up here so I can make a definitive statement. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #10 from Chris Cormack <chris@bigballofwax.co.nz> 2011-01-09 20:19:32 UTC --- Still awaiting testing -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff CC| |jcamins@cpbibliography.com -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #2725|0 |1 is obsolete| | --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 10232 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=10232&action=edit Add batch revert capability (--revert) to commit_biblios_file.pl. Also add --progress-interval to facilitate debugging bad MARC batches. The basis for this enhancement was a bad batch of MARC records that had some unknown Unicode characters in them which caused the batch import function from the web interface to abort after importing only a portion of the records. At this point you are completely stuck using the web user interface. You can't go forward because the only option is to try the import again from the beginning (which fails again). You cannot revert either since there is no way to revert a partially committed batch. This change uses the existing BatchRevertBibRecords function. Bug reference: 5332 Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Rebased to current master 2012-06-11 - please test. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 --- Comment #12 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- This functionality is also added by bug 2060. I overlooked this patch, and wrote more-or-less identical code when I needed the ability to revert from the command line to test authority import. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5332 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |RESOLVED CC| |magnus@enger.priv.no Resolution|--- |DUPLICATE --- Comment #13 from Magnus Enger <magnus@enger.priv.no> --- The main functionality of this patch (the --revert option) is covered by the patch for Bug 2060 (which is already signed off), as pointed out by Jared in Comment 12. I'm marking this bug as a duplicate of that bug. The present patch does introduce another option (--progress-interval) which is not present in the patches for Bug 2060, so that might be implemented on top of 2060 if someone wants to. *** This bug has been marked as a duplicate of bug 2060 *** -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org