[Bug 43729] New: Remove STOMP/RabbitMQ background jobs support
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Bug ID: 43729 Summary: Remove STOMP/RabbitMQ background jobs support Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org Depends on: 42077 Blocks: 41506 Target Milestone: --- Follow-up to bug 42077 (RabbitMQ made optional, polling the default) and part of the bug 41506 effort to remove RabbitMQ. Now that SQL polling is the default and RabbitMQ is no longer a hard dependency, this bug removes the STOMP/RabbitMQ background-jobs code path entirely and makes polling the only mechanism. Scope: * Koha/BackgroundJob.pm: drop Net::Stomp, the connect() method and the STOMP enqueue path (send_with_receipt). Keep the enqueue() contract (store status='new') and process($args). * Centralize the generic worker loop into a new Koha::Worker class; misc/workers/background_jobs_worker.pl becomes a thin wrapper (Koha::Worker->new->run). This supersedes the 'centralize worker code' work on bug 35920, whose patches will be re-attached here. * misc/workers/es_indexer_daemon.pl: strip the STOMP branch, keep the DB-polling batched indexer loop. * Remove the JobsNotificationMethod system preference: delete from sysprefs.sql, add an atomicupdate to DELETE it, and remove the entry from admin.pref. * about.pl / about.tt: remove the job_notification_method_check() broker-connection check, the warnConnectBroker warning and the 'Message broker' row. * Tests: drop STOMP-specific coverage in t/db_dependent/Koha/BackgroundJob.t (connect subtest, send_with_receipt mock) and t/db_dependent/Koha/BackgroundJobs.t. Out of scope (future): unifying zebraqueue + elastic_index into a shared indexing_queue and a batched worker abstraction; folding rebuild_zebra.pl -d into it. Depends on: bug 42077. See also: bug 35655 (introduced the pref), bug 35920 (worker centralization). Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41506 [Bug 41506] Should we remove RabbitMQ? https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42077 [Bug 42077] Move RabbitMQ from mandatory to optional and make SQL polling the default -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED CC| |tomascohen@gmail.com Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #1 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207314 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207314&action=edit Bug 43729: Remove STOMP support from the background jobs worker RabbitMQ/STOMP is being retired in favour of SQL polling (see bug 42077, which already made polling the default and RabbitMQ optional). This patch removes the STOMP code path from the generic background jobs worker. Koha::BackgroundJob no longer connects to a message broker: the connect() method and the Net::Stomp enqueue path (send_with_receipt) are dropped. enqueue() now simply stores the job with status 'new' for a worker to pick up by polling. The process($args) contract is unchanged, so job classes keep receiving their decoded arguments. The worker loop is centralised into a new Koha::Worker class, so misc/workers/background_jobs_worker.pl becomes a thin wrapper that builds a Koha::Worker and calls run(). Koha::Worker polls for 'new' jobs in the requested queues, forks via Parallel::ForkManager, decodes each job's data and passes it to the job's process method. Test plan: 1. Set up a Koha instance without RabbitMQ. 2. Run misc/workers/background_jobs_worker.pl --queue default --queue long_tasks 3. Trigger a background job (e.g. batch item modification). 4. Confirm the job is processed and reaches 'finished', with no broker connection warnings in the logs. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #2 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207315 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207315&action=edit Bug 43729: Remove STOMP support from the Elasticsearch indexer daemon With STOMP being retired, the Elasticsearch indexer daemon no longer needs to connect to a message broker. This patch strips the STOMP branch (connect/subscribe/receive_frame and the frame ack/nack handling) and keeps only the DB-polling loop, which already existed as the broker-less fallback. The batched behaviour is preserved: the daemon still fetches up to --batch_size 'new' jobs from the elastic_index queue, aggregates their record ids and commits them to Elasticsearch in chunks, resetting the batch to 'new' on a NoNodes error so it is retried after reconnection. Now-unused imports (JSON::decode_json, Time::HiRes) and the frame not-found retry bookkeeping are removed, and the POD is updated to drop the Stomp/RabbitMQ wording. Test plan: 1. Use Elasticsearch as the search engine, without RabbitMQ. 2. Run misc/workers/es_indexer_daemon.pl 3. Modify a record so an indexing job is enqueued. 4. Confirm the record is (re)indexed and the job reaches 'finished', with no broker connection warnings in the logs. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #3 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207316 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207316&action=edit Bug 43729: Remove the JobsNotificationMethod system preference The JobsNotificationMethod preference let administrators choose between STOMP and polling for background job notification. With STOMP support removed, polling is the only mechanism, so the preference no longer has any effect and is removed. Changes: - installer/data/mysql/mandatory/sysprefs.sql: drop the preference. - installer/data/mysql/atomicupdate/bug_43729.pl: DELETE it on upgrade. - admin.pref: remove the preference (and its now-empty 'Jobs' section). Test plan: 1. Apply the atomic update (updatedatabase.pl). 2. Confirm 'Removed system preference JobsNotificationMethod' is reported. 3. Confirm the preference is gone from Administration > System preferences. 4. prove t/db_dependent/check_sysprefs.t -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #4 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207317 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207317&action=edit Bug 43729: Remove the message broker check from the About page The About page tested the STOMP broker connection and warned when it could not be reached. With STOMP removed there is no broker to check, so this patch drops that logic. Changes: - about.pl: remove job_notification_method_check() and its call sites; the warnConnectBroker template param is no longer set. - about.tt: remove the 'Message broker' row from the system information table and the warnConnectBroker warning (and its entry in the warnings summary condition). Test plan: 1. Visit About > System information. 2. Confirm there is no 'Message broker' row and no broker-related warning. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #5 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207318 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207318&action=edit Bug 43729: Remove STOMP coverage from background jobs tests With the STOMP code path gone, the tests that exercised it no longer apply. Changes: - t/db_dependent/Koha/BackgroundJob.t: drop the 'connect' subtest and the Net::Stomp send_with_receipt mock used to force an enqueue exception; adjust the plan counts and remove the now-unused Test::Warn. - t/db_dependent/Koha/BackgroundJobs.t: drop the Net::Stomp mock. Test plan: 1. prove t/db_dependent/Koha/BackgroundJob.t t/db_dependent/Koha/BackgroundJobs.t 2. Confirm both pass. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207314|0 |1 is obsolete| | Attachment #207315|0 |1 is obsolete| | Attachment #207316|0 |1 is obsolete| | Attachment #207317|0 |1 is obsolete| | Attachment #207318|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #6 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207319 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207319&action=edit Bug 43729: Remove STOMP support from the background jobs worker RabbitMQ/STOMP is being retired in favour of SQL polling (see bug 42077, which already made polling the default and RabbitMQ optional). This patch removes the STOMP code path from the generic background jobs worker. Koha::BackgroundJob no longer connects to a message broker: the connect() method and the Net::Stomp enqueue path (send_with_receipt) are dropped. enqueue() now simply stores the job with status 'new' for a worker to pick up by polling. The process($args) contract is unchanged, so job classes keep receiving their decoded arguments. The worker loop is centralised into a new Koha::Worker class, so misc/workers/background_jobs_worker.pl becomes a thin wrapper that builds a Koha::Worker and calls run(). Koha::Worker polls for 'new' jobs in the requested queues, forks via Parallel::ForkManager, decodes each job's data and passes it to the job's process method. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/BackgroundJob.t \ t/db_dependent/Koha/BackgroundJobs.t \ t/db_dependent/Koha/BackgroundJob/ => SUCCESS: Tests pass! 3. Set up an instance without RabbitMQ, run the worker and trigger a background job (e.g. batch item modification); confirm it reaches 'finished' with no broker connection warnings in the logs. => SUCCESS: Job processed, no warnings! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #7 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207320&action=edit Bug 43729: Remove STOMP support from the Elasticsearch indexer daemon With STOMP being retired, the Elasticsearch indexer daemon no longer needs to connect to a message broker. This patch strips the STOMP branch (connect/subscribe/receive_frame and the frame ack/nack handling) and keeps only the DB-polling loop, which already existed as the broker-less fallback. The batched behaviour is preserved: the daemon still fetches up to --batch_size 'new' jobs from the elastic_index queue, aggregates their record ids and commits them to Elasticsearch in chunks, resetting the batch to 'new' on a NoNodes error so it is retried after reconnection. Now-unused imports (JSON::decode_json, Time::HiRes) and the frame not-found retry bookkeeping are removed, and the POD is updated to drop the Stomp/RabbitMQ wording. Test plan: 1. Apply patch 2. Use Elasticsearch as the search engine, without RabbitMQ 3. Run: $ ktd --shell k$ perl -c misc/workers/es_indexer_daemon.pl => SUCCESS: syntax OK! 4. Run the daemon, modify a record so an indexing job is enqueued, and confirm the record is (re)indexed and the job reaches 'finished', with no broker connection warnings in the logs. => SUCCESS: Record reindexed, no warnings! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #8 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207321 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207321&action=edit Bug 43729: Remove the JobsNotificationMethod system preference The JobsNotificationMethod preference let administrators choose between STOMP and polling for background job notification. With STOMP support removed, polling is the only mechanism, so the preference no longer has any effect and is removed. Changes: - installer/data/mysql/mandatory/sysprefs.sql: drop the preference. - installer/data/mysql/atomicupdate/bug_43729.pl: DELETE it on upgrade. - admin.pref: remove the preference (and its now-empty 'Jobs' section). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl installer/data/mysql/updatedatabase.pl => SUCCESS: 'Removed system preference JobsNotificationMethod' is reported! 3. Go to Administration > System preferences and search for 'JobsNotificationMethod' => SUCCESS: The preference is gone! 4. Run: k$ prove t/db_dependent/check_sysprefs.t => SUCCESS: Tests pass! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #9 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207322 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207322&action=edit Bug 43729: Remove the message broker check from the About page The About page tested the STOMP broker connection and warned when it could not be reached. With STOMP removed there is no broker to check, so this patch drops that logic. Changes: - about.pl: remove job_notification_method_check() and its call sites; the warnConnectBroker template param is no longer set. - about.tt: remove the 'Message broker' row from the system information table and the warnConnectBroker warning (and its entry in the warnings summary condition). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl -c about.pl => SUCCESS: syntax OK! 3. Visit About > System information => SUCCESS: No 'Message broker' row, no broker-related warning! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #10 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207323 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207323&action=edit Bug 43729: Remove STOMP coverage from background jobs tests With the STOMP code path gone, the tests that exercised it no longer apply. Changes: - t/db_dependent/Koha/BackgroundJob.t: drop the 'connect' subtest and the Net::Stomp send_with_receipt mock used to force an enqueue exception; adjust the plan counts and remove the now-unused Test::Warn. - t/db_dependent/Koha/BackgroundJobs.t: drop the Net::Stomp mock. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/BackgroundJob.t \ t/db_dependent/Koha/BackgroundJobs.t => SUCCESS: Tests pass! 3. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #11 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207324 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207324&action=edit Bug 43729: Drop rabbitmq-server from the Debian packaging Bug 42077 moved rabbitmq-server from a hard dependency to Suggests. Now that STOMP support is removed entirely, RabbitMQ is no longer used by Koha at all, so it is dropped from the packaging. Changes: - debian/control.in: remove rabbitmq-server from the Suggests of koha-common and koha-deps. - debian/control: regenerated to match. Test plan: 1. Apply patch 2. Run: $ grep -c rabbitmq debian/control debian/control.in => SUCCESS: No matches! 3. Confirm debian/control matches debian/control.in (debian/update-control produces no change beyond this). 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207319|0 |1 is obsolete| | Attachment #207320|0 |1 is obsolete| | Attachment #207321|0 |1 is obsolete| | Attachment #207322|0 |1 is obsolete| | Attachment #207323|0 |1 is obsolete| | Attachment #207324|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #12 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207325 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207325&action=edit Bug 43729: Remove STOMP support from the background jobs worker RabbitMQ/STOMP is being retired in favour of SQL polling (see bug 42077, which already made polling the default and RabbitMQ optional). This patch removes the STOMP code path from the generic background jobs worker. Koha::BackgroundJob no longer connects to a message broker: the connect() method and the Net::Stomp enqueue path (send_with_receipt) are dropped. enqueue() now simply stores the job with status 'new' for a worker to pick up by polling. The process($args) contract is unchanged, so job classes keep receiving their decoded arguments. The worker loop is centralised into a new Koha::Worker class, so misc/workers/background_jobs_worker.pl becomes a thin wrapper that builds a Koha::Worker and calls run(). Koha::Worker polls for 'new' jobs in the requested queues, forks via Parallel::ForkManager, decodes each job's data and passes it to the job's process method. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/BackgroundJob.t \ t/db_dependent/Koha/BackgroundJobs.t \ t/db_dependent/Koha/BackgroundJob/ => SUCCESS: Tests pass! 3. Set up an instance without RabbitMQ, run the worker and trigger a background job (e.g. batch item modification); confirm it reaches 'finished' with no broker connection warnings in the logs. => SUCCESS: Job processed, no warnings! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #13 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207326 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207326&action=edit Bug 43729: Remove STOMP support from the Elasticsearch indexer daemon With STOMP being retired, the Elasticsearch indexer daemon no longer needs to connect to a message broker. This patch strips the STOMP branch (connect/subscribe/receive_frame and the frame ack/nack handling) and keeps only the DB-polling loop, which already existed as the broker-less fallback. The batched behaviour is preserved: the daemon still fetches up to --batch_size 'new' jobs from the elastic_index queue, aggregates their record ids and commits them to Elasticsearch in chunks, resetting the batch to 'new' on a NoNodes error so it is retried after reconnection. Now-unused imports (JSON::decode_json, Time::HiRes) and the frame not-found retry bookkeeping are removed, and the POD is updated to drop the Stomp/RabbitMQ wording. Test plan: 1. Apply patch 2. Use Elasticsearch as the search engine, without RabbitMQ 3. Run: $ ktd --shell k$ perl -c misc/workers/es_indexer_daemon.pl => SUCCESS: syntax OK! 4. Run the daemon, modify a record so an indexing job is enqueued, and confirm the record is (re)indexed and the job reaches 'finished', with no broker connection warnings in the logs. => SUCCESS: Record reindexed, no warnings! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #14 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207327 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207327&action=edit Bug 43729: Remove the JobsNotificationMethod system preference The JobsNotificationMethod preference let administrators choose between STOMP and polling for background job notification. With STOMP support removed, polling is the only mechanism, so the preference no longer has any effect and is removed. Changes: - installer/data/mysql/mandatory/sysprefs.sql: drop the preference. - installer/data/mysql/atomicupdate/bug_43729.pl: DELETE it on upgrade. - admin.pref: remove the preference (and its now-empty 'Jobs' section). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl installer/data/mysql/updatedatabase.pl => SUCCESS: 'Removed system preference JobsNotificationMethod' is reported! 3. Go to Administration > System preferences and search for 'JobsNotificationMethod' => SUCCESS: The preference is gone! 4. Run: k$ prove t/db_dependent/check_sysprefs.t => SUCCESS: Tests pass! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #15 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207328 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207328&action=edit Bug 43729: Remove the message broker check from the About page The About page tested the STOMP broker connection and warned when it could not be reached. With STOMP removed there is no broker to check, so this patch drops that logic. Changes: - about.pl: remove job_notification_method_check() and its call sites; the warnConnectBroker template param is no longer set. - about.tt: remove the 'Message broker' row from the system information table and the warnConnectBroker warning (and its entry in the warnings summary condition). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl -c about.pl => SUCCESS: syntax OK! 3. Visit About > System information => SUCCESS: No 'Message broker' row, no broker-related warning! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #17 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207330 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207330&action=edit Bug 43729: Drop rabbitmq-server from the Debian packaging Bug 42077 moved rabbitmq-server from a hard dependency to Suggests. Now that STOMP support is removed entirely, RabbitMQ is no longer used by Koha at all, so it is dropped from the packaging. Changes: - debian/control.in: remove rabbitmq-server from the Suggests of koha-common and koha-deps. - debian/control: regenerated to match. Test plan: 1. Apply patch 2. Run: $ grep -c rabbitmq debian/control debian/control.in => SUCCESS: No matches! 3. Confirm debian/control matches debian/control.in (debian/update-control produces no change beyond this). 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com --- Comment #18 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- *** Bug 35920 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207325|0 |1 is obsolete| | Attachment #207326|0 |1 is obsolete| | Attachment #207327|0 |1 is obsolete| | Attachment #207328|0 |1 is obsolete| | Attachment #207329|0 |1 is obsolete| | Attachment #207330|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #19 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207331 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207331&action=edit Bug 43729: Remove STOMP support from the background jobs worker RabbitMQ/STOMP is being retired in favour of SQL polling (see bug 42077, which already made polling the default and RabbitMQ optional). This patch removes the STOMP code path from the generic background jobs worker. Koha::BackgroundJob no longer connects to a message broker: the connect() method and the Net::Stomp enqueue path (send_with_receipt) are dropped. enqueue() now simply stores the job with status 'new' for a worker to pick up by polling. The process($args) contract is unchanged, so job classes keep receiving their decoded arguments. The worker loop is centralised into a new Koha::Worker class, so misc/workers/background_jobs_worker.pl becomes a thin wrapper that builds a Koha::Worker and calls run(). Koha::Worker polls for 'new' jobs in the requested queues, forks via Parallel::ForkManager, decodes each job's data and passes it to the job's process method. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/BackgroundJob.t \ t/db_dependent/Koha/BackgroundJobs.t \ t/db_dependent/Koha/BackgroundJob/ => SUCCESS: Tests pass! 3. Set up an instance without RabbitMQ, run the worker and trigger a background job (e.g. batch item modification); confirm it reaches 'finished' with no broker connection warnings in the logs. => SUCCESS: Job processed, no warnings! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #20 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207332 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207332&action=edit Bug 43729: Remove STOMP support from the Elasticsearch indexer daemon With STOMP being retired, the Elasticsearch indexer daemon no longer needs to connect to a message broker. This patch strips the STOMP branch (connect/subscribe/receive_frame and the frame ack/nack handling) and keeps only the DB-polling loop, which already existed as the broker-less fallback. The batched behaviour is preserved: the daemon still fetches up to --batch_size 'new' jobs from the elastic_index queue, aggregates their record ids and commits them to Elasticsearch in chunks, resetting the batch to 'new' on a NoNodes error so it is retried after reconnection. Now-unused imports (JSON::decode_json, Time::HiRes) and the frame not-found retry bookkeeping are removed, and the POD is updated to drop the Stomp/RabbitMQ wording. Test plan: 1. Apply patch 2. Use Elasticsearch as the search engine, without RabbitMQ 3. Run: $ ktd --shell k$ perl -c misc/workers/es_indexer_daemon.pl => SUCCESS: syntax OK! 4. Run the daemon, modify a record so an indexing job is enqueued, and confirm the record is (re)indexed and the job reaches 'finished', with no broker connection warnings in the logs. => SUCCESS: Record reindexed, no warnings! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #21 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207333 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207333&action=edit Bug 43729: Remove the JobsNotificationMethod system preference The JobsNotificationMethod preference let administrators choose between STOMP and polling for background job notification. With STOMP support removed, polling is the only mechanism, so the preference no longer has any effect and is removed. Changes: - installer/data/mysql/mandatory/sysprefs.sql: drop the preference. - installer/data/mysql/atomicupdate/bug_43729.pl: DELETE it on upgrade. - admin.pref: remove the preference (and its now-empty 'Jobs' section). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl installer/data/mysql/updatedatabase.pl => SUCCESS: 'Removed system preference JobsNotificationMethod' is reported! 3. Go to Administration > System preferences and search for 'JobsNotificationMethod' => SUCCESS: The preference is gone! 4. Run: k$ prove t/db_dependent/check_sysprefs.t => SUCCESS: Tests pass! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #22 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207334 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207334&action=edit Bug 43729: Remove the message broker check from the About page The About page tested the STOMP broker connection and warned when it could not be reached. With STOMP removed there is no broker to check, so this patch drops that logic. Changes: - about.pl: remove job_notification_method_check() and its call sites; the warnConnectBroker template param is no longer set. - about.tt: remove the 'Message broker' row from the system information table and the warnConnectBroker warning (and its entry in the warnings summary condition). Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ perl -c about.pl => SUCCESS: syntax OK! 3. Visit About > System information => SUCCESS: No 'Message broker' row, no broker-related warning! 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #23 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207335 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207335&action=edit Bug 43729: Remove STOMP coverage from background jobs tests With the STOMP code path gone, the tests that exercised it no longer apply. Changes: - t/db_dependent/Koha/BackgroundJob.t: drop the 'connect' subtest and the Net::Stomp send_with_receipt mock used to force an enqueue exception; adjust the plan counts and remove the now-unused Test::Warn. - t/db_dependent/Koha/BackgroundJobs.t: drop the Net::Stomp mock. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/BackgroundJob.t \ t/db_dependent/Koha/BackgroundJobs.t => SUCCESS: Tests pass! 3. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #24 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207336 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207336&action=edit Bug 43729: Drop rabbitmq-server from the Debian packaging Bug 42077 moved rabbitmq-server from a hard dependency to Suggests. Now that STOMP support is removed entirely, RabbitMQ is no longer used by Koha at all, so it is dropped from the packaging. Changes: - debian/control.in: remove rabbitmq-server from the Suggests of koha-common and koha-deps. - debian/control: regenerated to match. Test plan: 1. Apply patch 2. Run: $ grep -c rabbitmq debian/control debian/control.in => SUCCESS: No matches! 3. Confirm debian/control matches debian/control.in (debian/update-control produces no change beyond this). 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #25 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207337 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207337&action=edit Bug 43729: Stop enabling the RabbitMQ STOMP plugin on install koha-common.postinst enabled the rabbitmq_stomp plugin and restarted rabbitmq-server on every install/upgrade so background jobs could reach the broker over STOMP. With STOMP support removed this is dead, so the RabbitMQ block is dropped. The memcached restart is kept. Test plan: 1. Apply patch 2. Read debian/koha-common.postinst => SUCCESS: No rabbitmq-plugins / rabbitmq-server handling remains, the memcached restart is still there! 3. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43729 --- Comment #26 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 207338 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207338&action=edit Bug 43729: Stop writing the message broker config in koha-create koha-create wrote a <message_broker> block (host/port/user/pass/vhost) into koha-conf.xml and exposed --mb-host/--mb-port/--mb-user/--mb-pass/ --mb-vhost options to configure it. This was only read by the STOMP code path, which has been removed, so it is dropped. Changes: - debian/scripts/koha-create: remove the --mb-* options, their defaults, the set_message_broker() helper, the call, and the __MESSAGE_BROKER_*__ substitutions. - debian/templates/koha-conf-site.xml.in: drop the <message_broker> block. - etc/koha-conf.xml and t/data/koha-conf.xml: drop the <message_broker> block. - t/Koha/Config.t: drop message_broker from the expected parsed config. - debian/docs/koha-create.xml: remove the --mb-* documentation. Existing instances can keep their <message_broker> block; it is simply ignored. The accompanying atomic update points this out. Test plan: 1. Apply patch 2. Run: $ bash -n debian/scripts/koha-create => SUCCESS: syntax OK! 3. Run: $ ktd --shell k$ prove t/Koha/Config.t => SUCCESS: Tests pass! 4. Create a new instance with koha-create and confirm koha-conf.xml has no <message_broker> block and the instance works. => SUCCESS: Instance created, no message broker config! 5. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org