[Bug 35655] New: Make it possible to switch off RabbitMQ without any warns in logs/about page
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Bug ID: 35655 Summary: Make it possible to switch off RabbitMQ without any warns in logs/about page Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: katrin.fischer@bsz-bw.de QA Contact: testopia@bugs.koha-community.org A recent survey on IRC revealed that a lot of people are still running Koha without RabbitMQ. It's not a small dependency and for smaller libraries it might not be needed ever. Also we still see issues with jobs not being processed correctly that has people turning it off. We should not create any warns or logs, but have a proper switch to turn RabbitMQ off without Koha complaining until we can fully rely on RabbitMQ at least. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com, | |michaela.sieber@kit.edu, | |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no --- Comment #1 from Magnus Enger <magnus@libriotech.no> --- +1 We have turned RabbitMQ off on a lot of sites, because things work better without it. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35092 CC| |m.de.rooy@rijksmuseum.nl --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- See the discussion on 35092. And the several options suggested. Your feedback is appreciated. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Marcel de Rooy from comment #2)
See the discussion on 35092. And the several options suggested. Your feedback is appreciated.
Thanks Marcel, for pointing this one out. I am all for fixing the remaining issues, but I feel we need a solution for stable installations until and I also see a use case for very small installations as RabbitMQ is taking additional resources. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 davewood@gmx.at changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |davewood@gmx.at --- Comment #4 from davewood@gmx.at --- I just ran into problems with background jobs again (koha-common/stable,now 23.11.04-4 all [installed]) There has been a discussion regarding RabbitMQ at the workshop in Marseille. I still think we would be better off without rabbitMQ. + no parallel solution (DBpoll+rabbitmq) + no extra service that has to be maintained (sysop) and i dont see the benefits rabbitmq brings. if i knew for sure the DB poll mechanism is here to stay i would love to have a config param to disable rabbitmq. and not have debian pull in the rabbitmq dependency. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to davewood from comment #4) Thx for your feedback. See also bug 35092.
I just ran into problems with background jobs again (koha-common/stable,now 23.11.04-4 all [installed])
Could you provide some more details here? What are those problems? Are they related to insufficient resources? Or ... ? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@trust-box.at -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | CC| |tomascohen@gmail.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com, | |liz@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #6 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 171412 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171412&action=edit Bug 35655: Add 'JobsNotificationMethod' system preference -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #7 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 171413 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171413&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) enqueue step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8. Repeat 2-4 => FAIL: You get the same errors, because it defaults to STOMP and it is not available 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all 10. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12. Repeat 2-4 => SUCCESS: No warnings about STOMP 13. Sign off :-D Dedicated to Liz -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Ok, I'm putting this patches here to start the conversation. This shouldn't affect an implementation of the 'Centralize code from workers' bug, but aligns with it. Let's get the ball rolling on this one, as it is hurting us right now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Note: I haven't touched the workers code, so they would still show a warning on startup. If there's a minimum consensus on going this route, I will provide a patch for that (super trivial actually) or happily sign off on other's follow-ups. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #10 from Mark Hofstetter <koha@trust-box.at> --- first of all thx for getting this started - and it seems to work as intended (ie no stomp messages) but minor notices/questions 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all + checking "about" page (cgi-bin/koha/about.pl) BEFORE stopping rabbitmq still says "Using RabbitMQ" which is unexpected from my POV, because if I switch it of I want to stop using it regardless whether it's running or not + if you do a stop rabbitmq => "Using SQL polling" + a change of a "system preference" which requires a restart? + related: is there an easy way to find out how a background job was started? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171413|0 |1 is obsolete| | --- Comment #11 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171503 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171503&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8. Repeat 2-4 => FAIL: You get the same errors, because it defaults to STOMP and it is not available 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all 10. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 11a. check about page (<hostname>/cgi-bin/koha/about.pl) says "Message broker: Using SQL polling" 12. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #12 from Mark Hofstetter <koha@trust-box.at> --- I did a reflow of the code so that the "JobsNotificationMethod" is checked only once at connect time, so that a warning is only emitted when you actually WANT to connect to rabbitmq but it is not working. I also fixed the about page which now has three states: + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171503|0 |1 is obsolete| | --- Comment #13 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171504 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171504&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171504|0 |1 is obsolete| | --- Comment #14 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171505 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171505&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171505|0 |1 is obsolete| | --- Comment #15 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171506 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171506&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #16 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171507 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171507&action=edit full patch I hate bugzilla patches guts git diff main > patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #17 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi Mark, the last patch won't apply with our tools (git bz). If you are using koha-testing-docker, you want to do: 1) sign off on the patches: git so 2 2) git bz attach -e HEAD~2.. If you are not using ktd or git bz is not set up correctly, something like git format-patch will do the trick to create patch files that have the right headers the diff file is missing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #18 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- ... was missing the bug number: git bz attach -e 35655 HEAD~2.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #19 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Any changes to the original patch set should be done in a follow-up commit/separate patch file. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171506|0 |1 is obsolete| | --- Comment #20 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171509 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171509&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #21 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171510 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171510&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8. Repeat 2-4 => FAIL: You get the same errors, because it defaults to STOMP and it is not available 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all 10. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 11a. check about page (<hostname>/cgi-bin/koha/about.pl) says "Message broker: Using SQL polling" 12. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D Signed-off-by: Mark Hofstetter <mark@hofstetter.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #22 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171511 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171511&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D Signed-off-by: Mark Hofstetter <mark@hofstetter.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171509|0 |1 is obsolete| | Attachment #171510|0 |1 is obsolete| | Attachment #171511|0 |1 is obsolete| | --- Comment #23 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 171512 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171512&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171507|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171412|0 |1 is obsolete| | Attachment #171512|0 |1 is obsolete| | --- Comment #24 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172027 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172027&action=edit Bug 35655: Add 'JobsNotificationMethod' system preference Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #25 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172028 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172028&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #26 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172029 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172029&action=edit Bug 35655: Add unit tests Add the ability to specify STOMP connection info via environment variables and utilize those in unit tests. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172027|0 |1 is obsolete| | --- Comment #27 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 172157 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172157&action=edit Bug 35655: Add 'JobsNotificationMethod' system preference Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172028|0 |1 is obsolete| | --- Comment #28 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 172158 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172158&action=edit Bug 35655: Only use STOMP if JobsNotificationMethod set to do so This patch makes the background jobs (jobs from now on) connect step not try to connect to the STOMP broker, unless explicitly told to do so. This is achieved by using a newly introduced system preference `JobsNotificationMethod` which falls back to STOMP if undefined. The "about" page now has 3 states for + Message broker: Using SQL polling + Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) + Message broker: Using RabbitMQ The goal for this patch is to stop flooding the logs on very busy sites when Rabbit is not being used. This shouldn't predate any efforts on making the STOMP based jobs brokering mechanism solid and usable. To test: 1. On clean `main`, run: $ ktd --shell 2. Open the logs on a separate terminal: $ ktd --shell k$ tail -f /var/log/koha/kohadev/*.log 3. Stage some MARC for import => SUCCESS: No warnings about STOMP 4. Import the staged records => SUCCESS: No warnings about STOMP 5. Stop rabbitmq: k$ sudo service rabbitmq-server stop 6. Repeat 2-4 => FAIL: The logs say it couldn't connect to STOMP 7. Apply this patches and run: k$ restart_all 8a. See the "about" page, should read: Message broker: Using SQL polling (Fallback, Error connecting to RabbitMQ) 8b. Repeat 2-4 should work with one warning in the 9. Start rabbit: k$ sudo service rabbitmq-server start k$ restart_all (not really necessary) 10a. See the "about" page, should read: Message broker: Using RabbitMQ 10b. Repeat 2-4 => SUCCESS: No warnings about STOMP 11. Change the `JobsNotificationMethod` system preference to *polling* and stop rabbit: k$ sudo service rabbitmq-server stop k$ restart_all 12a. See the "about" page, should read: Message broker: Using SQL polling 12b. Repeat 2-4 => SUCCESS: No warnings about STOMP, not even on startup 13. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172029|0 |1 is obsolete| | --- Comment #29 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 172159 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172159&action=edit Bug 35655: Add unit tests Add the ability to specify STOMP connection info via environment variables and utilize those in unit tests. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #30 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- (In reply to Mark Hofstetter from comment #12)
I did a reflow of the code so that the "JobsNotificationMethod" is checked only once at connect time, so that a warning is only emitted when you actually WANT to connect to rabbitmq but it is not working.
Excelsior! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |24.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #31 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Thanks for all the hard work! Pushed to main for the next 24.11.00 release as RM Assistant -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #32 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 172232 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172232&action=edit Bug 35655: (follow-up) Fix perlcritic error Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cornelius.amzar@bsz-bw.de, | |markus.john@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Status|Pushed to main |Needs documenting --- Comment #33 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Enhancement won't be backported to 24.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38988 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 --- Comment #34 from Mark Hofstetter <koha@trust-box.at> --- maybe a good place to put this information too - remove rabbitmq from koha-common debian package apt download koha-common dpkg-deb -x koha-common_24.11.01-2_all.deb koha-common dpkg-deb --control koha-common_24.11.01-2_all.deb koha-common/DEBIAN # remove rabbitmq package vim koha-common/DEBIAN/control # remove rabbitmq line eg restart vim ../koha-common/DEBIAN/postinst # build new package dpkg -b koha-common koha-common_24.11.01-2_all_wo_rabbitmq.deb apt-get install ./koha-common_24.11.01-2_all_wo_rabbitmq.deb you may still run into some other trouble, but at least not with rabbitmq not restarting -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |39286 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39286 [Bug 39286] BackgroundJob.t should mock_config -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Aude Charillon <aude.charillon@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aude.charillon@openfifth.co | |.uk Documentation| |https://gitlab.com/koha-com submission| |munity/koha-manual/-/merge_ | |requests/1091 Documentation| |Aude Charillon contact| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Aude Charillon <aude.charillon@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED --- Comment #35 from Aude Charillon <aude.charillon@openfifth.co.uk> --- New system preference added to the Koha Manual. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |40820 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40820 [Bug 40820] STOMP errors even when JobsNotificationMethod='polling' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655 Oliver Behnke <oliver.behnke@aei.mpg.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41506 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org