[Bug 29346] New: Add option for building the holds queue for a single bib in background
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Bug ID: 29346 Summary: Add option for building the holds queue for a single bib in background Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Hold requests Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: gmcharlt@gmail.com Currently the holds queue deletes all currently selected items, gets a list of bibs with holds and loops through the bibs, building the queue for each. If the queue could be run a for a single biblionumber it would allow: - Deleting only holds on that bib: - this would remove the need for bug 29130 - could be run in background when a hold is placed/updated - fulfilled holds are already removed from queue - so we shouldn't need to clear any bibs that aren't updated Ultimately this would remove the need for the build_holds_queue cronjob -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- *** Bug 29130 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Change sponsored?|--- |Sponsored -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131088 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131088&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131089 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131089&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #4 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Submitting early to get feedback from interested community members. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add option for building the |Add option for building the |holds queue for a single |holds queue for a single |bib in background |record in background -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |30181 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30181 [Bug 30181] Koha::BackgroundJob->_derived_class returns an empty object -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #5 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Thanks for the enthusiasm everyone. Moving forward! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add option for building the |Add real-time holds queue |holds queue for a single |update option |record in background | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131088|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131089|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #6 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131811 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131811&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #7 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131812 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131812&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131813 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131813&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com, | |katrin.fischer@bsz-bw.de, | |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 131815 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131815&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |30345 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30345 [Bug 30345] Koha::BackgroundJob->enqueue should set borrowernumber=undef if no userenv -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131811|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131812|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131813|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131815|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132078 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132078&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #11 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132079 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132079&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #12 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132080 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132080&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) The cancel() action is added a *skip_holds_queue* parameter to skip triggering the background job entirely. It targets cases like C4::Biblio::DelBiblio in which all biblio holds are cancelled in a loop. In that case, we just want to cancel them and let a single backgroung job take care of the holds queue, once the biblio is deleted. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries holds -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132081 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132081&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132082 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132082&action=edit Bug 29346: Biblio actions triggers This patch adds the trigger for the holds queue update on teh following methods: - C4::Biblio::DelBiblio - C4::Biblio::ModBiblio The ModBiblio use case could be improved by checking if itemtype is one of the updated attributes... but it felt there was no way to do it without some overhead. So I leave it as-is. It also mocks the ->enqueue method in the tests that call DelBiblio and ModBiblio to avoid breakages [1] Tests are added to check the trigger is called. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t => SUCCESS: Tests pass! It means the trigger is triggered :-D 3. Sign off :-D [1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101 FIXME. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #15 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 132083 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132083&action=edit Bug 29346: Item action trigger This patch makes the following actions trigger a holds queue rebuild for the related biblio: - Adding an item - Updating an item - Deleting an item To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Item.t => SUCCESS: Tests pass! Background job scheduled 3. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Bug 29346 depends on bug 30181, which changed state. Bug 30181 Summary: Koha::BackgroundJob->_derived_class returns an empty object https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30181 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30360 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30360 [Bug 30360] Add helper methods to Koha::BackgroundJobs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- *** Bug 26003 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|30345 |27783 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27783 [Bug 27783] Introduce background job queues https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30345 [Bug 30345] Koha::BackgroundJob->enqueue should set borrowernumber=undef if no userenv -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #17 from Owen Leonard <oleonard@myacpl.org> --- I'm getting an error from the test: t/db_dependent/HoldsQueue.t .. 1/58 No property queue for Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue# Looks like your test exited with 11 just after 2. t/db_dependent/HoldsQueue.t .. Dubious, test returned 11 (wstat 2816, 0xb00) Failed 56/58 subtests Test Summary Report ------------------- t/db_dependent/HoldsQueue.t (Wstat: 2816 Tests: 2 Failed: 0) Non-zero exit status: 11 Parse errors: Bad plan. You planned 58 tests but ran 2. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #18 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Owen Leonard from comment #17)
I'm getting an error from the test:
t/db_dependent/HoldsQueue.t .. 1/58 No property queue for Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue# Looks like your test exited with 11 just after 2. t/db_dependent/HoldsQueue.t .. Dubious, test returned 11 (wstat 2816, 0xb00) Failed 56/58 subtests
Test Summary Report ------------------- t/db_dependent/HoldsQueue.t (Wstat: 2816 Tests: 2 Failed: 0) Non-zero exit status: 11 Parse errors: Bad plan. You planned 58 tests but ran 2.
You need to run dbic on one of the dependencies -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #19 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Tomás Cohen Arazi from comment #18)
(In reply to Owen Leonard from comment #17)
I'm getting an error from the test:
t/db_dependent/HoldsQueue.t .. 1/58 No property queue for Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue# Looks like your test exited with 11 just after 2. t/db_dependent/HoldsQueue.t .. Dubious, test returned 11 (wstat 2816, 0xb00) Failed 56/58 subtests
Test Summary Report ------------------- t/db_dependent/HoldsQueue.t (Wstat: 2816 Tests: 2 Failed: 0) Non-zero exit status: 11 Parse errors: Bad plan. You planned 58 tests but ran 2.
You need to run dbic on one of the dependencies
I've added a [DO NOT PUSH] patch on bug 27783 for that. Please retry :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Sally <sally.healey@cheshiresharedservices.gov.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sally.healey@cheshireshared | |services.gov.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132078|0 |1 is obsolete| | --- Comment #20 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133183 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133183&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132079|0 |1 is obsolete| | --- Comment #21 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133184 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133184&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132080|0 |1 is obsolete| | --- Comment #22 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133185 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133185&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) The cancel() action is added a *skip_holds_queue* parameter to skip triggering the background job entirely. It targets cases like C4::Biblio::DelBiblio in which all biblio holds are cancelled in a loop. In that case, we just want to cancel them and let a single backgroung job take care of the holds queue, once the biblio is deleted. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132081|0 |1 is obsolete| | --- Comment #23 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133186 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133186&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132082|0 |1 is obsolete| | --- Comment #24 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133187 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133187&action=edit Bug 29346: Biblio actions triggers This patch adds the trigger for the holds queue update on teh following methods: - C4::Biblio::DelBiblio - C4::Biblio::ModBiblio The ModBiblio use case could be improved by checking if itemtype is one of the updated attributes... but it felt there was no way to do it without some overhead. So I leave it as-is. It also mocks the ->enqueue method in the tests that call DelBiblio and ModBiblio to avoid breakages [1] Tests are added to check the trigger is called. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t => SUCCESS: Tests pass! It means the trigger is triggered :-D 3. Sign off :-D [1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101 FIXME. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #132083|0 |1 is obsolete| | --- Comment #25 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133188 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133188&action=edit Bug 29346: Item action trigger This patch makes the following actions trigger a holds queue rebuild for the related biblio: - Adding an item - Updating an item - Deleting an item To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Item.t => SUCCESS: Tests pass! Background job scheduled 3. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #26 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133189 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133189&action=edit Bug 29346: (follow-up) Fix typo We set biblio_ids in enqueue then referred to biblio_id in process ;) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #27 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133190 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133190&action=edit Bug 29346: (follow-up) Fix import issue in HoldsQueue For some reason `t/db_dependent/HoldsQueue..t` starts failing for me without this patch.. using the full module path resolves the issue but I'd love to understand why/how this creeps in. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #28 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133191 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133191&action=edit Bug 29346: FIXME: Highlight double enqueue This patch highlights a possible issue with the triggers.. though it may not actually matter in reality. I appear to already see test failures before this patch with a double enqueue of the rebuild for both AddIssue and AddReturn.. I couldn't spot what was causing that but whilst digging I did find another case where it could happen as highlighted here... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #29 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This is cool.. and mostly appears to work in testing. I've added a few tiny follow-ups to fix things I found whilst running the tests.. not idea where the imports get a bit confused and require that second fix though. As for the third patch, it highlights a case I think we may need to deal with.. we're already getting double enqueued on AddIssue and AddReturn (I reckon the test passed before we added the next patches that add triggers to ModBiblio and ModItem).. they now double trigger for these cases.. or even triple trigger in certainly preference setups. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133191|0 |1 is obsolete| | --- Comment #30 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 133205 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133205&action=edit Bug 29346: FIXME: Highlight double enqueue This patch highlights a possible issue with the triggers.. though it may not actually matter in reality. I appear to already see test failures before this patch with a double enqueue of the rebuild for both AddIssue and AddReturn.. I couldn't spot what was causing that but whilst digging I did find another case where it could happen as highlighted here... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #31 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Adding caller in the test here is pretty useful for debugging.. you can now see which module is calling the enqueue each time :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #32 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Adding discussion from Slack.. I don't think we need the trigger in DelBiblio.. but we likely do need to think about preventing Item level triggers for batch operations and then triggering at the end of said batch operations (like BatchDeleteBiblio). I think we can drop the triggers in circ (but keep the tests to ensure the triggers do actually still get triggered once).. circ operations affect items and as such trigger the task there. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Bug 29346 depends on bug 27783, which changed state. Bug 27783 Summary: Introduce background job queues https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27783 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com Status|Needs Signoff |Failed QA --- Comment #33 from David Nind <david@davidnind.com> --- All tests pass except this one: prove t/db_dependent/Circulation_holdsqueue.t t/db_dependent/Circulation_holdsqueue.t .. Use of uninitialized value in addition (+) at /kohadevbox/koha/C4/Biblio.pm line 3107. # Looks like you planned 2 tests but ran 5. t/db_dependent/Circulation_holdsqueue.t .. 1/1 # Failed test 'AddIssue() and AddReturn() real-time holds queue tests' # at t/db_dependent/Circulation_holdsqueue.t line 66. # Looks like you failed 1 test of 1. t/db_dependent/Circulation_holdsqueue.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests Test Summary Report ------------------- t/db_dependent/Circulation_holdsqueue.t (Wstat: 256 Tests: 1 Failed: 1) Failed test: 1 Non-zero exit status: 1 Files=1, Tests=1, 2 wallclock secs ( 0.01 usr 0.01 sys + 1.98 cusr 0.11 csys = 2.11 CPU) Result: FAIL -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #34 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to David Nind from comment #33)
All tests pass except this one:
prove t/db_dependent/Circulation_holdsqueue.t
Hi, David. Thanks for testing! Martin highlighted with this the fact that some calls (circulation) trigger item/biblio updates and thus duplicate actions. I have worked on chasing those cases and will wrap the code and new tests on monday. Have a great weekend! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133184|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133185|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133186|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133187|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133188|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133189|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133190|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133205|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133183|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #35 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133991 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133991&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #36 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133992 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133992&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #37 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133993 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133993&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) The cancel() action is added a *skip_holds_queue* parameter to skip triggering the background job entirely. It targets cases like C4::Biblio::DelBiblio in which all biblio holds are cancelled in a loop. In that case, we just want to cancel them and let a single backgroung job take care of the holds queue, once the biblio is deleted. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #38 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133994 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133994&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #39 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133995 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133995&action=edit Bug 29346: Biblio actions triggers This patch adds the trigger for the holds queue update on teh following methods: - C4::Biblio::DelBiblio - C4::Biblio::ModBiblio The ModBiblio use case could be improved by checking if itemtype is one of the updated attributes... but it felt there was no way to do it without some overhead. So I leave it as-is. It also mocks the ->enqueue method in the tests that call DelBiblio and ModBiblio to avoid breakages [1] Tests are added to check the trigger is called. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t => SUCCESS: Tests pass! It means the trigger is triggered :-D 3. Sign off :-D [1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101 FIXME. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #40 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133996 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133996&action=edit Bug 29346: Item action trigger This patch makes the following actions trigger a holds queue rebuild for the related biblio: - Adding an item - Updating an item - Deleting an item To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Item.t => SUCCESS: Tests pass! Background job scheduled 3. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #41 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133997 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133997&action=edit Bug 29346: (follow-up) Fix typo We set biblio_ids in enqueue then referred to biblio_id in process ;) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #42 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133998 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133998&action=edit Bug 29346: (follow-up) Fix import issue in HoldsQueue For some reason `t/db_dependent/HoldsQueue..t` starts failing for me without this patch.. using the full module path resolves the issue but I'd love to understand why/how this creeps in. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #43 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 133999 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=133999&action=edit Bug 29346: Highlight double enqueue This patch highlights a possible issue with the triggers.. though it may not actually matter in reality. I appear to already see test failures before this patch with a double enqueue of the rebuild for both AddIssue and AddReturn.. I couldn't spot what was causing that but whilst digging I did find another case where it could happen as highlighted here... Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #44 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134000 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134000&action=edit Bug 29346: Avoid duplicate actions tests Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #45 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134001 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134001&action=edit Bug 29346: Add more fine-grained control of holds queue updates This patch deals with the fact that high-level circualtion methods like `AddIssue`, `AddReturn` and `ModDateLastSeen` all eventually call lower-level methods like ModBiblio, Koha::Item->store of UpdateTotalIssues which are expected to trigger holds queue updates (for the object CRUD operations use cases). As the circulation methods need to trigger holds queue update as well, duplicate updates were being requested which is suboptimal, of course. In order to prevent this, and because circulation methdos could trigger holds queue updates several times, actually, I added a new parameter *skip_holds_queue* to the low-level methods, so when they are called from circulation, the trigger is skipped and we have greater control on when and how holds queue updates are scheduled. This patch introduces the `skip_holds_queue` parameter to the following methods: * C4::Biblio::ModBiblio * C4::Biblio::UpdateTotalIssues * Koha::Item->store Calls to those methods from the following methods will include the new parameter, and thus duplicated holds queue updates avoided: * C4::Circulation::AddIssue * C4::Circulation::AddReturn * C4::Items::ModDateLastSeen Tests are added, to verify that the (mocked) BatchUpdateBiblioHoldsQueue task is only scheduled once when they are called. To test: 1. Apply up to the previous patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t \ t/db_dependent/Biblio_holdsqueue.t \ t/db_dependent/Circulation_holdsqueue.t => FAIL: Tests fail! 3. Apply this patch 4. Repeat 2 => SUCCESS: Tests pass! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #46 from David Nind <david@davidnind.com> --- Created attachment 134010 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134010&action=edit Bug 29077: (follow-up) Use another syntax Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134010|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #47 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134225 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134225&action=edit Bug 29346: (QA follow-up) Rebuild queue when adjusting priorities of holds -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #48 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134227 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134227&action=edit Bug 29346: (follow-up) Fix typo 'availabe' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #49 from Nick Clemens <nick@bywatersolutions.com> --- The queue works, test pass, but when I go to the background jobs page: 173 Started 0 / 1 Unknown job type 'update_holds_queue_for_biblios' 04/28/2022 12:08 04/28/2022 12:08 04/28/2022 12:08 And if I click view: Template process failed: file error - background_jobs/update_holds_queue_for_biblios.inc: not found at /kohadevbox/koha/C4/Templates.pm line 127 Also, we should update the queue when we 'Revert waiting status' on a hold -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134225|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133991|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133992|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133993|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133994|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133995|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133996|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133997|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133998|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #133999|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134000|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134001|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134227|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #50 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Sorry, it seems I didn't confirm the upload on my desktop computer. Will do once I get back home. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #51 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134309 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134309&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #52 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134310 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134310&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #53 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134311 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134311&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) The cancel() action is added a *skip_holds_queue* parameter to skip triggering the background job entirely. It targets cases like C4::Biblio::DelBiblio in which all biblio holds are cancelled in a loop. In that case, we just want to cancel them and let a single backgroung job take care of the holds queue, once the biblio is deleted. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #54 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134312 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134312&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #55 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134313 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134313&action=edit Bug 29346: Biblio actions triggers This patch adds the trigger for the holds queue update on teh following methods: - C4::Biblio::DelBiblio - C4::Biblio::ModBiblio The ModBiblio use case could be improved by checking if itemtype is one of the updated attributes... but it felt there was no way to do it without some overhead. So I leave it as-is. It also mocks the ->enqueue method in the tests that call DelBiblio and ModBiblio to avoid breakages [1] Tests are added to check the trigger is called. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t => SUCCESS: Tests pass! It means the trigger is triggered :-D 3. Sign off :-D [1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101 FIXME. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #56 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134314 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134314&action=edit Bug 29346: Item action trigger This patch makes the following actions trigger a holds queue rebuild for the related biblio: - Adding an item - Updating an item - Deleting an item To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Item.t => SUCCESS: Tests pass! Background job scheduled 3. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #57 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134315 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134315&action=edit Bug 29346: (follow-up) Fix typo We set biblio_ids in enqueue then referred to biblio_id in process ;) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #58 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134316 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134316&action=edit Bug 29346: Highlight double enqueue This patch highlights a possible issue with the triggers.. though it may not actually matter in reality. I appear to already see test failures before this patch with a double enqueue of the rebuild for both AddIssue and AddReturn.. I couldn't spot what was causing that but whilst digging I did find another case where it could happen as highlighted here... Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #59 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134317 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134317&action=edit Bug 29346: Avoid duplicate actions tests Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #60 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134318 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134318&action=edit Bug 29346: Add more fine-grained control of holds queue updates This patch deals with the fact that high-level circualtion methods like `AddIssue`, `AddReturn` and `ModDateLastSeen` all eventually call lower-level methods like ModBiblio, Koha::Item->store of UpdateTotalIssues which are expected to trigger holds queue updates (for the object CRUD operations use cases). As the circulation methods need to trigger holds queue update as well, duplicate updates were being requested which is suboptimal, of course. In order to prevent this, and because circulation methdos could trigger holds queue updates several times, actually, I added a new parameter *skip_holds_queue* to the low-level methods, so when they are called from circulation, the trigger is skipped and we have greater control on when and how holds queue updates are scheduled. This patch introduces the `skip_holds_queue` parameter to the following methods: * C4::Biblio::ModBiblio * C4::Biblio::UpdateTotalIssues * Koha::Item->store Calls to those methods from the following methods will include the new parameter, and thus duplicated holds queue updates avoided: * C4::Circulation::AddIssue * C4::Circulation::AddReturn * C4::Items::ModDateLastSeen Tests are added, to verify that the (mocked) BatchUpdateBiblioHoldsQueue task is only scheduled once when they are called. To test: 1. Apply up to the previous patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t \ t/db_dependent/Biblio_holdsqueue.t \ t/db_dependent/Circulation_holdsqueue.t => FAIL: Tests fail! 3. Apply this patch 4. Repeat 2 => SUCCESS: Tests pass! 5. Sign off :-D Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #61 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134319 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134319&action=edit Bug 29346: (QA follow-up) Rebuild queue when adjusting priorities of holds Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #62 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134320&action=edit Bug 29346: (follow-up) Fix typo 'availabe' Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #63 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134321 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134321&action=edit Bug 29346: Add better reporting Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #64 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 134322 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134322&action=edit Bug 29346: Remove unused circular dependency on C4::Search Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #134309|0 |1 is obsolete| | Attachment #134310|0 |1 is obsolete| | Attachment #134311|0 |1 is obsolete| | Attachment #134312|0 |1 is obsolete| | Attachment #134313|0 |1 is obsolete| | Attachment #134314|0 |1 is obsolete| | Attachment #134315|0 |1 is obsolete| | Attachment #134316|0 |1 is obsolete| | Attachment #134317|0 |1 is obsolete| | Attachment #134318|0 |1 is obsolete| | Attachment #134319|0 |1 is obsolete| | Attachment #134320|0 |1 is obsolete| | Attachment #134321|0 |1 is obsolete| | Attachment #134322|0 |1 is obsolete| | --- Comment #65 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134328 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134328&action=edit Bug 29346: Refactor loop code into a subroutine The CreateQueue() method deletes the holds queue data, fetches some configuration (branches to use, transport cost matrix) and then loops through a list of biblionumbers, generating the tmp_holdsqueue and hold_fill_targets rows for the specified biblio. This patch simply moves that last bit that is run inside the biblios loop into a separate sub. The update_queue_for_biblio sub is designed so it does the exact same thing it did inside the loop, but also gets added the capability of querying those parameters if not passed, and it also gets a 'delete' parameter so it deletes the biblio-specific holds queue rows before starting to work. This way, it can be reused to write a background job for real-time holds queue update :-D To test: 1. Run: $ kshell k$ prove t/db_dependent/HoldsQueue.t => SUCCESS: Tests pass! 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! Behavior is kept! 4. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #66 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134329 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134329&action=edit Bug 29346: Add holds queue update background job This patch adds a background job that takes care of updating the holds queue rows for a given list of biblios. It is designed to be used for real-time batch updating the holds queue when required. Tasks would be added by the relevant methods under the right use cases. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #67 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134330 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134330&action=edit Bug 29346: Hold actions triggers This patch makes several holds related actions schedule the background job for real-time update of the holds queue. This actions are: - place (C4::Reserves::AddReserve) - fill (Koha::Hold->fill) - cancel (Koha::Hold->cancel) - suspend (Koha::Hold->suspend) - resume (Koha::Hold->resume) The cancel() action is added a *skip_holds_queue* parameter to skip triggering the background job entirely. It targets cases like C4::Biblio::DelBiblio in which all biblio holds are cancelled in a loop. In that case, we just want to cancel them and let a single backgroung job take care of the holds queue, once the biblio is deleted. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Hold.t \ t/db_dependent/Reserves.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #68 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134331 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134331&action=edit Bug 29346: Circulation actions triggers This patch introduces triggers for real-time updating the holds queue at check out and check in. The following high-level methods are involved: - C4::Circulation::AddIssue - C4::Circulation::AddReturn To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Circulation_holdsqueue.t => SUCCESS: Tests pass! Triggers are triggered 3. Sign off :-D Note: I put the tests on a separate file because the other one was too big already. Sponsored-by: Montgomery County Public Libraries Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #69 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134332 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134332&action=edit Bug 29346: Biblio actions triggers This patch adds the trigger for the holds queue update on teh following methods: - C4::Biblio::DelBiblio - C4::Biblio::ModBiblio The ModBiblio use case could be improved by checking if itemtype is one of the updated attributes... but it felt there was no way to do it without some overhead. So I leave it as-is. It also mocks the ->enqueue method in the tests that call DelBiblio and ModBiblio to avoid breakages [1] Tests are added to check the trigger is called. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t => SUCCESS: Tests pass! It means the trigger is triggered :-D 3. Sign off :-D [1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101 FIXME. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #70 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134333 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134333&action=edit Bug 29346: Item action trigger This patch makes the following actions trigger a holds queue rebuild for the related biblio: - Adding an item - Updating an item - Deleting an item To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Koha/Item.t => SUCCESS: Tests pass! Background job scheduled 3. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #71 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134334 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134334&action=edit Bug 29346: (follow-up) Fix typo We set biblio_ids in enqueue then referred to biblio_id in process ;) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #72 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134335 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134335&action=edit Bug 29346: Highlight double enqueue This patch highlights a possible issue with the triggers.. though it may not actually matter in reality. I appear to already see test failures before this patch with a double enqueue of the rebuild for both AddIssue and AddReturn.. I couldn't spot what was causing that but whilst digging I did find another case where it could happen as highlighted here... Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #73 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134336 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134336&action=edit Bug 29346: Avoid duplicate actions tests Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #74 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134337 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134337&action=edit Bug 29346: Add more fine-grained control of holds queue updates This patch deals with the fact that high-level circualtion methods like `AddIssue`, `AddReturn` and `ModDateLastSeen` all eventually call lower-level methods like ModBiblio, Koha::Item->store of UpdateTotalIssues which are expected to trigger holds queue updates (for the object CRUD operations use cases). As the circulation methods need to trigger holds queue update as well, duplicate updates were being requested which is suboptimal, of course. In order to prevent this, and because circulation methdos could trigger holds queue updates several times, actually, I added a new parameter *skip_holds_queue* to the low-level methods, so when they are called from circulation, the trigger is skipped and we have greater control on when and how holds queue updates are scheduled. This patch introduces the `skip_holds_queue` parameter to the following methods: * C4::Biblio::ModBiblio * C4::Biblio::UpdateTotalIssues * Koha::Item->store Calls to those methods from the following methods will include the new parameter, and thus duplicated holds queue updates avoided: * C4::Circulation::AddIssue * C4::Circulation::AddReturn * C4::Items::ModDateLastSeen Tests are added, to verify that the (mocked) BatchUpdateBiblioHoldsQueue task is only scheduled once when they are called. To test: 1. Apply up to the previous patch 2. Run: $ kshell k$ prove t/db_dependent/Biblio.t \ t/db_dependent/Biblio_holdsqueue.t \ t/db_dependent/Circulation_holdsqueue.t => FAIL: Tests fail! 3. Apply this patch 4. Repeat 2 => SUCCESS: Tests pass! 5. Sign off :-D Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #75 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134338 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134338&action=edit Bug 29346: (QA follow-up) Rebuild queue when adjusting priorities of holds Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #76 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134339 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134339&action=edit Bug 29346: (follow-up) Fix typo 'availabe' Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #77 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134340 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134340&action=edit Bug 29346: Add better reporting Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #78 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134341 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134341&action=edit Bug 29346: Remove unused circular dependency on C4::Search Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #79 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134342 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134342&action=edit Bug 29346: Use fully qualified names for C4:Circulation routines in C4::HoldsQueue I suppose this is similar to circular dependency on other patch HoldsQueue uses Circulation uses BatchUpdateBiblioHoldsQueueuse HoldsQueue Without this the background job builds the queue, but reports failure: Holds queue for biblio The Jacobite clans of the Great Glen, 1650-1784 /. An error occurred (Undefined subroutine &C4::HoldsQueue::GetTransfers called at /kohadevbox/koha/C4/HoldsQueue.pm line 351. ) Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #80 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 134343 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=134343&action=edit Bug 29346: (follow-up) Build holds queue when AlterPriority or RevertWaitingStatus is called To test: 1 - Place 3 holds on a bib with a single item 2 - Confirm bib shows in holds queue 3 - Check in item and cnofirm hold 4 - Bib is no longer in queue 5 - Revert the waiting status 6 - The hold is in the queue again 7 - Move top hold to bottom 8 - Confirm queue selects hold for new top priorty patron Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|30360 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30360 [Bug 30360] Add helper methods to Koha::BackgroundJobs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add real-time holds queue |Real-time holds queue |update option |update QA Contact|testopia@bugs.koha-communit |nick@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_22_05_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Medium patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #81 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to master for 22.05, thanks to everybody involved 🦄 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #82 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- There is something wrong with the update_holds_queue_for_biblios task. Try this, in fresh ktd delete itemnumbers 958,960 in a batch and check "delete the biblio if no item remain". You will see 3 update_holds_queue_for_biblios new tasks. I think we need to revisit this enhancement for the batch tools. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #83 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #82)
There is something wrong with the update_holds_queue_for_biblios task. Try this, in fresh ktd delete itemnumbers 958,960 in a batch and check "delete the biblio if no item remain". You will see 3 update_holds_queue_for_biblios new tasks.
I think we need to revisit this enhancement for the batch tools.
This makes sense, one update each when we remove the items, then a third update when we delete one of the biblios Ideally we would only update once, but the code handles item deletion, then biblio deletion, so would need a refactoring to support this. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30710 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30710 [Bug 30710] Optimize holds queue on batch deletion -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 --- Comment #84 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Nick Clemens from comment #83)
(In reply to Jonathan Druart from comment #82)
There is something wrong with the update_holds_queue_for_biblios task. Try this, in fresh ktd delete itemnumbers 958,960 in a batch and check "delete the biblio if no item remain". You will see 3 update_holds_queue_for_biblios new tasks.
I think we need to revisit this enhancement for the batch tools.
This makes sense, one update each when we remove the items, then a third update when we delete one of the biblios
data: {"biblio_ids":[429],"messages":[{"type":"success","biblio_id":429,"code":"holds_queue_updated"}],"report":{"total_biblios":1,"total_success":1}} data: {"biblio_ids":[430],"messages":[{"code":"holds_queue_updated","biblio_id":430,"type":"success"}],"report":{"total_success":1,"total_biblios":1}} data: {"messages":[{"biblio_id":430,"type":"success","code":"holds_queue_updated"}],"biblio_ids":[430],"report":{"total_biblios":1,"total_success":1}} Yes, it's what it does. But it does not make sense, it's a bug. With 1 batchmod of 2 items, we end up with: 1 task for the main one 3 holds_queue_updated 1 indexing -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30727 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30727 [Bug 30727] Holds queue updates can be called multiple times on batch record deletion -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30728 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30728 [Bug 30728] Allow real-time holds queue opt-out -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |Enabled by default, this release notes| |feature can be opted out | |using the | |*RealTimeHoldsQueue* system | |preference. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|30728 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30728 [Bug 30728] Allow real-time holds queue opt-out -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|30727 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30727 [Bug 30727] Holds queue updates can be called multiple times on batch record deletion -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|Pushed to master |RESOLVED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_22_05_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25579 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |31948 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31948 [Bug 31948] Add timestamp to tmp_holdsqueue -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25891 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |34822 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34822 [Bug 34822] BatchUpdateBiblioHoldsQueue should be called once per import batch when using RealTimeHoldsQueue -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28205 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35819 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35819 [Bug 35819] "No job found" error for BatchUpdateBiblioHoldsQueue (race condition) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29346 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org