[Bug 43937] New: Koha::Plugins::Loader does not cache the empty list of enabled plugins
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43937 Bug ID: 43937 Summary: Koha::Plugins::Loader does not cache the empty list of enabled plugins Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Plugin architecture Assignee: koha-bugs@lists.koha-community.org Reporter: adolfo.rodriguez@xercode.es QA Contact: testopia@bugs.koha-community.org CC: kyle@bywatersolutions.com, martin.renvoize@openfifth.co.uk, tomascohen@gmail.com Target Milestone: --- Koha::Plugins::Loader->get_enabled_plugins caches the list of enabled plugins in Koha::Cache::Memory::Lite, but only when it is not empty: Koha::Cache::Memory::Lite->set_in_cache( $cache_key, \@plugins ) if @plugins; So when plugins are enabled in koha-conf.xml (enable_plugins) but no plugin is enabled, every call to Koha::Plugins->call, Koha::Plugins->get_enabled_plugins or Koha::Plugins->feature_enabled checks that the plugin_data table exists and queries it again. Some plugin hooks are called for each item: for instance, Koha::Item->store calls two of them. The code before bug 38384 cached the empty list too. In KTD (no plugin enabled), storing the 3000 items of a record in a loop runs 6000 queries on plugin_data and 6000 table_exists checks, and takes 77 s. With the empty list cached: 1 query, 1 check, 47 s. This was discussed in bug 38384 comment 55, and the test was adjusted to the behaviour. The reason the empty list was not cached (comment 36) was that the early loading could not find the plugins yet; that loading has since moved to the BEGIN block of Koha::Plugins, after pluginsdir is added to @INC. However, caching the empty list has a side effect on the workers. Koha::Cache::Memory::Lite is flushed for each request under Plack, but background_jobs_worker.pl and es_indexer_daemon.pl never flush it, and they load Koha::Plugins (and so the enabled plugins) when they start: each job runs in a child process that inherits the cached list. This already means that, when at least one plugin is enabled, the workers do not see the plugins enabled or disabled afterwards until they are restarted. With no plugin enabled they currently see the first enabled plugin, only because the empty list is not cached. Proposed fix: - cache the empty list in Koha::Plugins::Loader - flush Koha::Cache::Memory::Lite in the workers before each job, as Plack does for each request, so that plugin changes are picked up without restarting them (es_indexer_daemon.pl already flushes the L1 cache before each job, for the system preferences) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43937 Adolfo Rodríguez Taboada <adolfo.rodriguez@xercode.es> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38384, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=34121 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43937 Adolfo Rodríguez Taboada <adolfo.rodriguez@xercode.es> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |adolfo.rodriguez@xercode.es |ity.org | Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43937 --- Comment #1 from Adolfo Rodríguez Taboada <adolfo.rodriguez@xercode.es> --- Created attachment 209652 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=209652&action=edit Bug 43937: Cache the empty list of enabled plugins Koha::Plugins::Loader->get_enabled_plugins only cached the list of enabled plugins when it was not empty. With no plugin enabled, every plugin hook call checked that the plugin_data table exists and queried it again. Some hooks are called for each item (Koha::Item->store calls two of them), so this added four queries per item. This patch caches the empty list too, as the code before bug 38384 did. The workers load the enabled plugins when they start, and never flushed Koha::Cache::Memory::Lite, so each job inherited the cached list: when at least one plugin was enabled, they did not see the plugins enabled or disabled afterwards until they were restarted. With no plugin enabled they did, only because the empty list was not cached. This patch makes background_jobs_worker.pl and es_indexer_daemon.pl flush the memory cache before each job, as Plack does for each request, so that plugin changes are picked up without restarting them. Steps to follow: 1. Make sure that enable_plugins is 1 in koha-conf.xml and that no plugin is enabled (Administration > Manage plugins). 2. Run this script, which stores all the items of a record and counts the queries on plugin_data (use a record with many items, e.g. add 200 items to a record with "Add multiple copies"): perl -e ' use Modern::Perl; use Time::HiRes qw(time); use Koha::Biblios; use Koha::Plugins::Datas; my $n = 0; my $orig = Koha::Plugins::Datas->can("search"); { no warnings "redefine"; *Koha::Plugins::Datas::search = sub { $n++; goto &$orig } } my $t = time; $_->store( { skip_record_index => 1 } ) for Koha::Biblios->find(shift)->items->as_list; printf "%.2f s, %d queries on plugin_data\n", time - $t, $n; ' <biblionumber> => 2 queries per item. 3. Apply the patch and repeat step 2: no queries (the empty list was already cached when Koha::Plugins was loaded), and it is faster. 4. The workers: this script does what they do. It loads the enabled plugins at start, then enables or disables a plugin from another process, flushes the memory cache if asked (as the workers now do before each job), and forks a child (a job) that counts the enabled plugins. Install a plugin (e.g. the kitchen sink plugin), disable it, and run it with its class name: cat > /tmp/workers.pl <<'END' use Modern::Perl; use Koha::Plugins; use Koha::Cache::Memory::Lite; my ( $class, $action, $flush ) = @ARGV; system( $^X, '-e', "use Koha::Plugins; use $class; $class->new->$action" ); Koha::Cache::Memory::Lite->flush if $flush; unless (fork) { say scalar grep { ref $_ eq $class } Koha::Plugins->get_enabled_plugins; exit } wait; END perl /tmp/workers.pl Koha::Plugin::Com::ByWaterSolutions::KitchenSink enable 0 perl /tmp/workers.pl Koha::Plugin::Com::ByWaterSolutions::KitchenSink disable 0 Without the patch: 1 and 1 (the disabled plugin is still used). With the patch and without the flush: 0 and 1 (nothing is seen). With the patch and the flush (third argument 1, as in the workers): 1 and 0. 5. prove t/db_dependent/Koha/Plugins/Loader.t Signed-off-by: Adolfo Rodríguez <adolfo.rodriguez@xercode.es> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org