[Bug 32305] New: Backround worker doesn't check jobs receive from rabbitmq are in 'new' state.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Bug ID: 32305 Summary: Backround worker doesn't check jobs receive from rabbitmq are in 'new' state. Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: didier.gautheron@biblibre.com QA Contact: testopia@bugs.koha-community.org How to reproduce : 1) stop rabbitmq and koha-worker services. 2) start koha-worker , without rabbitmq it's polling the database. 3) start rabbitmq, koha-worker is not connecting to it. 4) submit a job. A new row is add to background_jobs table and the job is enqueued in rabbitmq. 5) the job is processed by koha-worker from the database. 6) restart koha-worker, this time connection to rabbitmq succeeded 7) koha-worker receive job message from rabbitmq and rerun it. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #1 from Didier Gautheron <didier.gautheron@biblibre.com> --- Created attachment 144156 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144156&action=edit Bug 32305: Only process jobs in 'new' state -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |major -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |didier.gautheron@biblibre.c |ity.org |om Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32330 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #2 from Fridolin Somers <fridolin.somers@biblibre.com> --- I've created Bug 32330 to optimize query of background_jobs_worker.pl when pooling jobs directly from database -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Amit Gupta <amitddng135@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |amit.gupta@informaticsgloba | |l.com, | |amitddng135@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Amit Gupta <amitddng135@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Backround worker doesn't |Background worker doesn't |check jobs receive from |check jobs receive from |rabbitmq are in 'new' |rabbitmq are in 'new' |state. |state. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Background worker doesn't |Background worker doesn't |check jobs receive from |check jobs receive from |rabbitmq are in 'new' |rabbitmq are in 'new' state |state. | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au, | |jonathan.druart+koha@gmail. | |com, | |martin.renvoize@ptfs-europe | |.com, tomascohen@gmail.com --- Comment #3 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I am not sure this is the correct place, shouldn't it be in Koha::BackgroundJob->process? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #4 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 144278 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144278&action=edit [ALTERNATIVE-PATCH] Bug 32305: Don't process a job if its status is not 'new' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Patch for discussion, will need tests if we go with it. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I think Jonathans approach is the right way to go.. Though I must admit, I found it confusing when looking at the 'start' helper method too.. that already checks status and throws an exception if the status isn't new. Having both 'process' and 'start' methods is a tad confusing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #7 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 144389 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144389&action=edit Bug 32305: Don't process a job if its status is not 'new' Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Should we force a refetch from the DB just prior to checking this status.. just in case we have a stale object? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #9 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to Martin Renvoize from comment #8)
Should we force a refetch from the DB just prior to checking this status.. just in case we have a stale object?
Sounds like a good idea to me -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #8)
Should we force a refetch from the DB just prior to checking this status.. just in case we have a stale object?
I suppose this goes back to what Jonathan was saying at bug 32395... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Please tell me what we are still discussing here? I came from 30943 which is mnore or less linked to this dev. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |32395 --- Comment #12 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Let's move forward with Jonathan's approach, and please add the refetch. We then make bug 32395 depend on this one, for further cleanup. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 [Bug 32395] Why do we send jobs params to the broker -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #144156|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #144278|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Signed Off Assignee|didier.gautheron@biblibre.c |jonathan.druart+koha@gmail. |om |com Keywords| |additional_work_needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #13 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I don't understand why we would need a fetch here, and I actually think we should not get from db. The ->process call is from the worker, which should happen right after ->find. If the object does not have a status==new then we should explode as the developer certainly messed up with something. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|additional_work_needed | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |BLOCKED QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Looking here -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #5)
Patch for discussion, will need tests if we go with it.
This speaks for itself :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #16 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- How much sense does it make to check the status in the base object if most background jobs override it with their own? Hmm, looks like we always start in the base object and go to derived class.. And note that derived often begins with a ->start that contains an exception already for a status <> new. Should we add the start in the general process instead ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #17 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #13)
I don't understand why we would need a fetch here, and I actually think we should not get from db.
The ->process call is from the worker, which should happen right after ->find. If the object does not have a status==new then we should explode as the developer certainly messed up with something.
Sounds good to me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |Failed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|jonathan.druart+koha@gmail. |koha-bugs@lists.koha-commun |com |ity.org -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #18 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I won't have more time to dedicate to this, feel free to take over. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=31245 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | Status|Failed QA |ASSIGNED --- Comment #19 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #18)
I won't have more time to dedicate to this, feel free to take over.
I will now in light of the pending bug 34990. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|m.de.rooy@rijksmuseum.nl |testopia@bugs.koha-communit | |y.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #20 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- My solution is resolve the inconsistency between MQ and DB "mode": MQ my $job = Koha::BackgroundJobs->find($args->{job_id}); DB my $jobs = Koha::BackgroundJobs->search({ status => 'new', queue => \@queues }); We should do the same checks for MQ mode, so: my $job = Koha::BackgroundJobs->search( { id => $args->{job_id}, status => 'new, queue => \@queues } ); So check status new AND check queue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=34990 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #144389|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #21 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157256 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157256&action=edit Bug 32305: Worker should ignore wrong status or queue in MQ mode MQ mode: The worker has seen that MQ is running and consumes messages instead of polling db only. Test plan: Stop worker for long tasks. Make sure that Rabbit MQ runs. Stage a file. (This adds a long task.) Goto staff view of jobs and cancel this job. Check if job is still in MQ with rabbitmqctl list_queues. Now start worker for long tasks. Check if job is gone in MQ with rabbitmqctl list_queues. And check logfile for the adjusted warning like: [WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Background worker doesn't |Background worker doesn't |check jobs receive from |check job status when |rabbitmq are in 'new' state |received from rabbitmq -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #22 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- We should do it too for es_indexer_daemon.pl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #23 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157320&action=edit Bug 32305: Counterpart for es_indexer_daemon.pl This is a kind of copy-and-paste variant of background worker. We probably could do better than having two scripts here ;) Test plan: See former test plan. Apply it to ES indexing. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157256|0 |1 is obsolete| | Attachment #157320|0 |1 is obsolete| | --- Comment #24 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 157321 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157321&action=edit Bug 32305: Worker should ignore wrong status or queue in MQ mode MQ mode: The worker has seen that MQ is running and consumes messages instead of polling db only. Test plan: Stop worker for long tasks. Make sure that Rabbit MQ runs. Stage a file. (This adds a long task.) Goto staff view of jobs and cancel this job. Check if job is still in MQ with rabbitmqctl list_queues. Now start worker for long tasks. Check if job is gone in MQ with rabbitmqctl list_queues. And check logfile for the adjusted warning like: [WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #25 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 157322 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157322&action=edit Bug 32305: Counterpart for es_indexer_daemon.pl This is a kind of copy-and-paste variant of background worker. We probably could do better than having two scripts here ;) Test plan: See former test plan. Apply it to ES indexing. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35092 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |34990 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34990 [Bug 34990] Backgroundjob->enqueue does not send persistent header -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #26 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Please QA. We need this one to go with 34990. See also omnibus. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #27 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Note for QA: Jonathan suggested to move the selection to a filter_by method in BackgroundJobs. I agree. We should do that not on the compact bugfix here imo, but make that part of some refactoring as described further on the omnibus. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com --- Comment #28 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- QAing now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157321|0 |1 is obsolete| | --- Comment #29 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 157496 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157496&action=edit Bug 32305: Worker should ignore wrong status or queue in MQ mode MQ mode: The worker has seen that MQ is running and consumes messages instead of polling db only. Test plan: Stop worker for long tasks. Make sure that Rabbit MQ runs. Stage a file. (This adds a long task.) Goto staff view of jobs and cancel this job. Check if job is still in MQ with rabbitmqctl list_queues. Now start worker for long tasks. Check if job is gone in MQ with rabbitmqctl list_queues. And check logfile for the adjusted warning like: [WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157322|0 |1 is obsolete| | --- Comment #30 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 157497 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157497&action=edit Bug 32305: Counterpart for es_indexer_daemon.pl This is a kind of copy-and-paste variant of background worker. We probably could do better than having two scripts here ;) Test plan: See former test plan. Apply it to ES indexing. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #31 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- All looking good here.. yes, it would be great to move to a filter_by.. but I totally agree it's OK to do that as part of the wider refactoring work going on. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157496|0 |1 is obsolete| | --- Comment #32 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157500 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157500&action=edit Bug 32305: Worker should ignore wrong status or queue in MQ mode MQ mode: The worker has seen that MQ is running and consumes messages instead of polling db only. Test plan: Stop worker for long tasks. Make sure that Rabbit MQ runs. Stage a file. (This adds a long task.) Goto staff view of jobs and cancel this job. Check if job is still in MQ with rabbitmqctl list_queues. Now start worker for long tasks. Check if job is gone in MQ with rabbitmqctl list_queues. And check logfile for the adjusted warning like: [WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> [EDIT] Removed queue from query at MQ side. Discussed on IRC. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157497|0 |1 is obsolete| | --- Comment #33 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157501 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157501&action=edit Bug 32305: Counterpart for es_indexer_daemon.pl This is a kind of copy-and-paste variant of background worker. We probably could do better than having two scripts here ;) Test plan: See former test plan. Apply it to ES indexing. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> [EDIT] Removed queue from query at MQ side. Discussed on IRC. -- You are receiving this mail because: You are watching all bug changes.
From IRC: [11:00] <Joubu> if the worker is getting an ID from the MQ it's not possible
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #34 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- that the ID is from another queue. That should be true indeed. Being cautious.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #35 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I think we can remove the queue check.. it's superflous and could lead to confusion later once we've all forgotten this discussion. Happy for that to be a quick follow-up please :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #36 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Martin Renvoize from comment #35)
I think we can remove the queue check.. it's superflous and could lead to confusion later once we've all forgotten this discussion.
Happy for that to be a quick follow-up please :)
See above. Done -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #37 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.11.00 |23.11.00,23.05.05 released in| | Status|Pushed to master |Pushed to stable --- Comment #38 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 23.05.x for 23.05.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.11.00,23.05.05 |23.11.00,23.05.05,22.11.12 released in| | Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 --- Comment #39 from Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> --- Nice work everyone! Pushed to oldstable for 22.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |michaela.sieber@kit.edu, | |raphael.straub@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC|martin.renvoize@ptfs-europe | |.com | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz Resolution|--- |FIXED Status|Pushed to oldstable |RESOLVED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org