[Koha-bugs] [Bug 35092] [OMNIBUS] Remaining background job/worker issues

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092

--- Comment #1 from Marcel de Rooy <m.de.rooy at rijksmuseum.nl> ---
Some thoughts on the sync issues between MQ and DB (bugs 32305, 34990 and
34997):

What are the different scenarios where the (Rabbit) message queue (MQ) and the
background_jobs table (DB) are out of sync ? FIrst note that we can assume that
enqueued jobs are always saved in the DB; a message without a job in the DB
will be discarded when reading it from MQ.

[1] New jobs in DB are not found in MQ.
CAUSE: If MQ is not running, Koha only writes to the DB. If MQ is restarted,
all jobs on MQ are gone too (solved by 34990).
If you start the worker after MQ in this context, the worker only processes MQ
jobs. So you will have "orphaned jobs" in the DB. But be aware that they will
pop up again as soon as you start the worker before MQ !
As long as you restart the worker before or without MQ, things will be "fine".
Actually we are just ignoring MQ.
We need to decide how to change the algorithm in the worker to resolve this.
See further below.

[2] New jobs in MQ have another status in DB
CAUSE: The worker has been started before MQ. It only processes DB jobs. So a
large number of jobs is accumulating on MQ.
Without bug 34990, we can restart MQ to clear its queue and restart the worker
after it processed all DB jobs to be in sync again. If we do not start MQ, we
will just keep processing DB entries.
With bug 34990 and 32305, new jobs on MQ that have another status in DB will
just be discarded. Restarting the worker after MQ will restore a synced
situation. If we restart worker before or without MQ, the situation does not
actually change.
To improve, we should not send messages to MQ if we are not processing them in
the worker at this moment. See further below.

What benefit have bugs 32305 + 34990 ?
Bug 34990 reduces the number of orphaned (lost) DB jobs after a MQ restart when
actually using MQ.
Bug 32305 prevents running a job that has been processed already. If the worker
was in DB mode but MQ is active, a worker restart would switch to MQ mode and
reprocess MQ jobs. The number could even increase when we would only have 34990
since it persists MQ jobs. The combination of 34990 and 32305 resolves that.
If you do not use MQ at all, the bugs do not change anything.

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.