[Bug 29387] New: BatchUpdateBiblio does not handle exception correctly
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Bug ID: 29387 Summary: BatchUpdateBiblio does not handle exception correctly Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: ASSIGNED Severity: major Priority: P5 - low Component: Tools Assignee: jonathan.druart+koha@gmail.com Reporter: jonathan.druart+koha@gmail.com QA Contact: testopia@bugs.koha-community.org Depends on: 22417 If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417 [Bug 22417] Add a task queue -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 127205 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127205&action=edit Bug 29387: Stringify exception when logging error during a batch mod biblio If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Test plan: 0. Don't apply the patch In KTD you can run the a batch mod biblio on the 100 first records and the worker will fail with """ countered object 'C4::Biblio::_koha_modify_biblioitem_nonmarc(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'lccn' at row 1 at /kohadevbox/koha/C4/Biblio.pm line 423 ', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at /kohadevbox/koha/Koha/BackgroundJob/BatchUpdateBiblio.pm line 120. """ The UI is not saying anything about this problem. Generate 100 biblionumbers: perl -e 'for (1..100){print "$_\n"}' 1. Apply the patch and confirm than now you see the detail of the batch update and that biblionumber 72 is marked as not been modified with the correct error (raw DBI error) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nugged@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127205|0 |1 is obsolete| | --- Comment #2 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 127207 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127207&action=edit Bug 29387: Stringify exception when logging error during a batch mod biblio If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Test plan: 0. Don't apply the patch In KTD you can run the a batch mod biblio on the 100 first records and the worker will fail with """ countered object 'C4::Biblio::_koha_modify_biblioitem_nonmarc(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'lccn' at row 1 at /kohadevbox/koha/C4/Biblio.pm line 423 ', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at /kohadevbox/koha/Koha/BackgroundJob/BatchUpdateBiblio.pm line 120. """ The UI is not saying anything about this problem. Generate 100 biblionumbers: perl -e 'for (1..100){print "$_\n"}' 1. Apply the patch and confirm than now you see the detail of the batch update and that biblionumber 72 is marked as not been modified with the correct error (raw DBI error) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Joonas Kylmälä <joonas.kylmala@iki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |joonas.kylmala@iki.fi --- Comment #3 from Joonas Kylmälä <joonas.kylmala@iki.fi> --- This only fixes the bug from one place, these need the same fix: ./Koha/BackgroundJob/BatchDeleteAuthority.pm60: error => ($@ ? $@ : 0), ./Koha/BackgroundJob/BatchUpdateAuthority.pm93: error => ($@ ? $@ : 0), ./Koha/BackgroundJob/BatchCancelHold.pm97: ? ( $@ ? $@ : 0 ) ./Koha/BackgroundJob/BatchDeleteBiblio.pm139: error => ($@ ? $@ : $error), -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #4 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 127431 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127431&action=edit Bug 29387: Strinfigy exceptions for other background job modules -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #5 from Fridolin Somers <fridolin.somers@biblibre.com> --- We have two behaviors : error => "$@" error => ($@ ? "$@" : 0) Shouldn't it be only one here ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Joonas Kylmälä <joonas.kylmala@iki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127207|0 |1 is obsolete| | Attachment #127431|0 |1 is obsolete| | --- Comment #6 from Joonas Kylmälä <joonas.kylmala@iki.fi> --- Created attachment 127594 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127594&action=edit Bug 29387: Stringify exception when logging error during a batch mod biblio If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Test plan: 0. Don't apply the patch In KTD you can run the a batch mod biblio on the 100 first records and the worker will fail with """ countered object 'C4::Biblio::_koha_modify_biblioitem_nonmarc(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'lccn' at row 1 at /kohadevbox/koha/C4/Biblio.pm line 423 ', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at /kohadevbox/koha/Koha/BackgroundJob/BatchUpdateBiblio.pm line 120. """ The UI is not saying anything about this problem. Generate 100 biblionumbers: perl -e 'for (1..100){print "$_\n"}' 1. Apply the patch and confirm than now you see the detail of the batch update and that biblionumber 72 is marked as not been modified with the correct error (raw DBI error) JK: add executable bit to BatchUpdateBiblio.t Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #7 from Joonas Kylmälä <joonas.kylmala@iki.fi> --- Created attachment 127595 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127595&action=edit Bug 29387: Strinfigy exceptions for other background job modules Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Joonas Kylmälä <joonas.kylmala@iki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #8 from Joonas Kylmälä <joonas.kylmala@iki.fi> --- (In reply to Fridolin Somers from comment #5)
We have two behaviors : error => "$@" error => ($@ ? "$@" : 0) Shouldn't it be only one here ?
In the error => "$@" cases there is already the if else check in the outer "if else" block so no need for this. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_21_11_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | Status|Signed Off |BLOCKED --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- QAing -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Did we consider the allow_blessed/convert_blessed route in JSON? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|BLOCKED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127594|0 |1 is obsolete| | --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 127825 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127825&action=edit Bug 29387: Stringify exception when logging error during a batch mod biblio If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Test plan: 0. Don't apply the patch In KTD you can run the a batch mod biblio on the 100 first records and the worker will fail with """ countered object 'C4::Biblio::_koha_modify_biblioitem_nonmarc(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'lccn' at row 1 at /kohadevbox/koha/C4/Biblio.pm line 423 ', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at /kohadevbox/koha/Koha/BackgroundJob/BatchUpdateBiblio.pm line 120. """ The UI is not saying anything about this problem. Generate 100 biblionumbers: perl -e 'for (1..100){print "$_\n"}' 1. Apply the patch and confirm than now you see the detail of the batch update and that biblionumber 72 is marked as not been modified with the correct error (raw DBI error) JK: add executable bit to BatchUpdateBiblio.t Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 127826 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127826&action=edit Bug 29387: Stringify exceptions for other background job modules Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 127827 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127827&action=edit Bug 29387: (QA follow-up) Fix modules in test Can't locate object method "new" via package "Koha::BackgroundJob::BatchUpdateBiblio" (perhaps you forgot to load "Koha::BackgroundJob::BatchUpdateBiblio"?) at t/db_dependent/Koha/BackgroundJobs/BatchUpdateBiblio.t line 45. How did it pass at signoff? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127595|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #14 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Marcel de Rooy from comment #13)
Created attachment 127827 [details] [review] Bug 29387: (QA follow-up) Fix modules in test
Can't locate object method "new" via package "Koha::BackgroundJob::BatchUpdateBiblio" (perhaps you forgot to load "Koha::BackgroundJob::BatchUpdateBiblio"?) at t/db_dependent/Koha/BackgroundJobs/BatchUpdateBiblio.t line 45.
How did it pass at signoff?
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Thanks for the follow-up. It got broken by bug 28413 that was pushed in the meanwhile. (In reply to Marcel de Rooy from comment #10)
Did we consider the allow_blessed/convert_blessed route in JSON?
I didn't, we could but I wouldn't introduce such changes for 21.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |21.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #15 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Pushed to master for 21.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED CC| |kyle@bywatersolutions.com Resolution|--- |FIXED --- Comment #16 from Kyle M Hall <kyle@bywatersolutions.com> --- Doesn't apply to 21.05.x. Please rebase and let me know if this is needed in 21.05! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_21_11_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #17 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 127924 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127924&action=edit Bug 29387: [21.05] Stringify exception when logging error during a batch mod biblio If an exception is raised by one the methods/subroutines called when batch biblio update, the job will fail in the middle and some records won't be processed. Test plan: 0. Don't apply the patch In KTD you can run the a batch mod biblio on the 100 first records and the worker will fail with """ countered object 'C4::Biblio::_koha_modify_biblioitem_nonmarc(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'lccn' at row 1 at /kohadevbox/koha/C4/Biblio.pm line 423 ', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at /kohadevbox/koha/Koha/BackgroundJob/BatchUpdateBiblio.pm line 120. """ The UI is not saying anything about this problem. Generate 100 biblionumbers: perl -e 'for (1..100){print "$_\n"}' 1. Apply the patch and confirm than now you see the detail of the batch update and that biblionumber 72 is marked as not been modified with the correct error (raw DBI error) JK: add executable bit to BatchUpdateBiblio.t Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Bug 29387: Stringify exceptions for other background job modules Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Bug 29387: (QA follow-up) Fix modules in test Can't locate object method "new" via package "Koha::BackgroundJob::BatchUpdateBiblio" (perhaps you forgot to load "Koha::BackgroundJob::BatchUpdateBiblio"?) at t/db_dependent/Koha/BackgroundJobs/BatchUpdateBiblio.t line 45. How did it pass at signoff? Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|FIXED |--- Status|RESOLVED |REOPENED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 --- Comment #18 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Kyle M Hall from comment #16)
Doesn't apply to 21.05.x. Please rebase and let me know if this is needed in 21.05!
Yes it's needed, I rebased and squashed the patches. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29387 Bug 29387 depends on bug 22417, which changed state. Bug 22417 Summary: Add a task queue https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org