https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43633 Bug ID: 43633 Summary: RFC: task scheduling strategy - cron, background jobs, and plugin hooks Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org Target Milestone: --- This is a discussion RFC, not a patch. No code changes are attached. Motivation Koha's approach to periodic/background work has grown organically across three different mechanisms, and it is not obvious which one new work should use: 1. misc/cronjobs/ - 56 standalone scripts wired into system cron (in packaged installs, via koha-common's shared /etc/cron.d, cron.daily, cron.hourly and cron.monthly files, which loop over every enabled instance with koha-foreach). 2. Koha::BackgroundJob - a message-queue-backed (STOMP/RabbitMQ) async task system, enqueued explicitly at user/API-action time and run by a persistent worker (misc/background_jobs_worker.pl). 3. The cronjob_nightly() plugin hook, invoked once per install by misc/cronjobs/plugins_nightly.pl, which fans out to every installed plugin's own nightly method (filterable by -m name=). There is also a community plugin, koha-plugin-crontab, which lets staff manage an instance's own crontab from the UI instead of editing it over SSH. That raised the underlying question this RFC wants to settle: should core lean into cron more (e.g. by encouraging/adopting per-instance user crontabs the way that plugin expects), or should more scheduling move into the background jobs framework, so that fewer new features need their own misc/cronjobs/*.pl script at all? Current state (verified against main) - koha-common's packaged cron model is multi-instance and root-owned: a handful of shared cron.d/daily/hourly/monthly files call koha-foreach across every enabled instance. It is not "one crontab per instance" today. - koha-plugin-crontab operates on a single instance's own user crontab (or a configured cron file) - a different ownership model from the above. Making core lean on that model would mean changing koha-common packaging itself (dropping the shared cron files, generating per-instance crontabs at koha-create time), and would not help non-packaged/dev installs at all. The plugin also has no plugin-to-plugin registration API; it is a UI over the existing crontab, not an execution backend other code can depend on. - Koha::BackgroundJob has no concept of recurring/time-based execution at all (no run_at, no cadence/cron expression). It is strictly an on-demand queue for work triggered by a user or API call. Adding scheduling to it would mean building a new subsystem (a cadence table plus a periodic dispatcher), and that dispatcher would still need something to trigger it - i.e. cron or a systemd timer underneath - while adding a hard dependency on the message broker and worker daemon staying up, which today's plain cron scripts do not need. - The cronjob_nightly() plugin hook already solves "how does a plugin get periodic execution" without adding its own crontab line or misc/cronjobs/ script: one core cron entry fans out to every installed plugin. Proposal for discussion - Do not migrate core's default packaged cron model to per-instance user crontabs. The koha-foreach/shared-cron-file model fits the common multi-instance production deployment; koha-plugin-crontab can remain an optional convenience layer for installs that want self-service scheduling, without core depending on it. - Do not move general-purpose recurring scheduling into Koha::BackgroundJob. It solves async execution of user-triggered work, not scheduling, and a scheduler bolted onto it would duplicate cron while adding a broker/worker dependency. - Do grow the cronjob_nightly()-style hook (for example additional cronjob_hourly()/cronjob_frequent() variants) as the sanctioned way for plugins to get periodic execution, instead of new misc/cronjobs/*.pl scripts or bespoke crontab entries per plugin. - Separately, for the subset of existing cron scripts that are already heavy/long-running (merge_authorities.pl, automatic_renewals.pl, and similar batch jobs), there may be value in having cron enqueue a Koha::BackgroundJob instead of running the logic inline, to get retry/status/permission-scoped visibility in the staff UI. That is an execution-model improvement for specific jobs, not a scheduling-model change, and it still needs cron (or a systemd timer) as the trigger. Open questions for the list 1. Is there appetite for formalising additional cronjob_<cadence>() plugin hooks (hourly, frequent/every-N-minutes) alongside the existing cronjob_nightly()? 2. Are there core cron scripts that the community would like to see converted to enqueue a Koha::BackgroundJob, to gain the existing background jobs UI/retry/permissions model? 3. Is per-instance user-crontab management (as koha-plugin-crontab provides) something core packaging should ever adopt, or should it stay a third-party/optional layer? Filed to start discussion on the koha-devel list / Bugzilla; no patch attached. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.