[Bug 32395] New: Why do we send job's params to the broker
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Bug ID: 32395 Summary: Why do we send job's params to the broker Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: jonathan.druart+koha@gmail.com QA Contact: testopia@bugs.koha-community.org CC: dcook@prosentient.com.au, m.de.rooy@rijksmuseum.nl, martin.renvoize@ptfs-europe.com, tomascohen@gmail.com We are: - Building the parameters - Insert a job and its parameters in DB - Send a message to the broken with the parameters - Retrieve the message from the worker - Process the job using the parameter from the broker Why don't we simply pass the job_id to the broker, then retrieve the parameters from the DB in the worker? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |22417 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417 [Bug 22417] Add a task queue -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32393 --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- We would have avoided all the JSON encoding/decoding stuffs we had recently, and bug 32393. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #0)
Why don't we simply pass the job_id to the broker, then retrieve the parameters from the DB in the worker?
In theory, a loosely coupled message consumer might not have database access to the app, so you'd want to encapsulate everything it needs in the message. However, the system designed for Koha is tightly coupled, so yeah... we could just pass the job_id in the message and have the worker fetch the details. It adds more database I/O, but it probably wouldn't be significant. In some cases, it could actually be more efficient. I don't know what the best practice is, and I imagine we should just do whatever makes sense for us. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- In the context of bug 32393 I see your point even more clearly. Since we're tracking jobs in the database and not doing a fire and forget pattern, it probably does make sense to just pass the job_id, because the message really is just "oh hey, I've got a job. Here's it's ID. Grab it and do stuff with it." -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Why do we send job's params |Why do we send jobs params |to the broker |to the broker -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also|https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32393 | Depends on| |32393 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32393 [Bug 32393] background job worker explodes if JSON is incorrect -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 --- Comment #4 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 144434 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144434&action=edit Bug 32395: Do not send job's parameters to the broker -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- The logic is here, but I don't think this is enough. It must be more robust and clear. Do we allow other parameters than job_id? Should we simply pass the id as a string an completely remove JSON? I don't think so, we should keep flexibility and keep in mind we could have jobs that don't go through the DB. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 --- Comment #6 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Jonathan Druart from comment #5)
The logic is here, but I don't think this is enough. It must be more robust and clear. Do we allow other parameters than job_id? Should we simply pass the id as a string an completely remove JSON?
I don't think so, we should keep flexibility and keep in mind we could have jobs that don't go through the DB.
Let's pass a JSON with the job_id. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |32305 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 [Bug 32305] Background worker doesn't check jobs receive from rabbitmq are in 'new' state -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Bug 32395 depends on bug 32393, which changed state. Bug 32393 Summary: background job worker explodes if JSON is incorrect https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32393 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35092 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|35092 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35092 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 Bug 32395 depends on bug 32305, which changed state. Bug 32305 Summary: Background worker doesn't check job status when received from rabbitmq https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32305 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org