[Bug 15341] New: Performance - Retrieve all sysprefs at once
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Bug ID: 15341 Summary: Performance - Retrieve all sysprefs at once Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: jonathan.druart@bugs.koha-community.org Reporter: jonathan.druart@bugs.koha-community.org QA Contact: testopia@bugs.koha-community.org C4::Context->preference generates a call to Koha::Config::SysPrefs->find for all different sysprefs we are searching for. It seems to be a performance killer and be better to retrieve them all together. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 45535 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=45535&action=edit Bug 15341: Retrieve all sysprefs at once C4::Context->preference generates a call to Koha::Config::SysPrefs->find for all different sysprefs we are searching for. It seems to be a performance killer and be better to retrieve them all together. I have enabled general_log in the MariaDB config file, the GET circ/circulation.pl?borrowernumber=42 There are 293 queries of the following form: SELECT me.variable, me.value, me.options, me.explanation, me.type FROM systempreferences me WHERE ( me.variable = 'MY_SYSPREF' ) Then benchmarking before and after this patch (using plackup, and the misc/plack/koha.psgi file with Profiler::NYTProf enabled): Before: Profile of /home/koha/perl5/bin/plackup for -2.07s (of 616ms), executing 158857 statements and 588479 subroutine calls in 427 source files and 73 string evals. 885 174ms 4.03s C4::Context::preference After: Profile of /home/koha/perl5/bin/plackup for -1.70s (of 349ms), executing 94273 statements and 397525 subroutine calls in 437 source files and 79 string evals. 1138 194ms 1.99s C4::Context::preference (calls) (exclusive time) (inclusive time) Could someone confirm that? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |13967 CC| |kyle@bywatersolutions.com, | |martin.renvoize@ptfs-europe | |.com, | |mirko@abunchofthings.net, | |tomascohen@gmail.com Referenced Bugs: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13967 [Bug 13967] Add package for System preferences -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |15342 Referenced Bugs: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15342 [Bug 15342] Performance 3.22 - Omnibus -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |frederic@tamil.fr --- Comment #2 from Frédéric Demians <frederic@tamil.fr> --- On my box, without plack, execution time: - Without your patch: 1.70s - With your patch: 1.54s I can see in NYTProf stats that 1/3 of execution time is spent in C4::Context::preference. Isn't plack supposed to bring some persistency? And so couldn't we have preferences available somewhere from one page to the other? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #3 from Frédéric Demians <frederic@tamil.fr> ---
I can see in NYTProf stats that 1/3 of execution time is spent in C4::Context::preference.
But, this time is in fact spent in Koha::Database::schema, than DBIx::Class::Schema::load_namespaces. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Frédéric Demians from comment #2)
Isn't plack supposed to bring some persistency? And so couldn't we have preferences available somewhere from one page to the other?
Both psgi files (debian/templates/plack.psgi and misc/plack/koha.psgi) disable the syspref cache (C4::Context->disable_syspref_cache();), see bug 13805. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |abl@biblos.pk.edu.pl -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #5 from Frédéric Demians <frederic@tamil.fr> --- See comment 3. The main issue isn't in preferences by themselves, but in DBIx::Class startup. Let see that: https://metacpan.org/pod/distribution/DBIx-Class/lib/DBIx/Class/Manual/Cookb... Loading all Koha schema classes take more than 0.5 seconds on a recent server. With plack, schema shouldn't be reloaded for each page. That's where plack persistency will save time. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=15350 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #6 from Jacek Ablewicz <abl@biblos.pk.edu.pl> --- (In reply to Jonathan Druart from comment #4)
(In reply to Frédéric Demians from comment #2)
Isn't plack supposed to bring some persistency? And so couldn't we have preferences available somewhere from one page to the other?
Both psgi files (debian/templates/plack.psgi and misc/plack/koha.psgi) disable the syspref cache (C4::Context->disable_syspref_cache();), see bug 13805.
Not good ;). But in my tests, with this patch applied, it is even worse - when syspref cache is disabled: #!/usr/bin/perl use Modern::Perl; use C4::Context; C4::Context->disable_syspref_cache(); my @sysprefs = qw/ UseTransportCostMatrix StaticHoldsQueueWeight RandomizeHoldsQueueWeight/; for (my $i = 0; $i < 1000; $i++) { for my $syspref_name (@sysprefs) { my $syspref_val = C4::Context->preference($syspref_name); } } Runnig time - without patch: 4.4sec, with patch: 47.2 sec.. With syspref cache enabled, the difference is negglible (0.36 patched, 0.37 sec unpatched), but the patched version still seems to be a bit slower (but not much, <2 usec per call - hard to measure, in real life scenarios I guess it will also depend on the mysql query_cache_limit/size/type settings etc.). -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Jacek Ablewicz from comment #6)
(In reply to Jonathan Druart from comment #4)
(In reply to Frédéric Demians from comment #2)
Isn't plack supposed to bring some persistency? And so couldn't we have preferences available somewhere from one page to the other?
Both psgi files (debian/templates/plack.psgi and misc/plack/koha.psgi) disable the syspref cache (C4::Context->disable_syspref_cache();), see bug 13805.
Not good ;). But in my tests, with this patch applied, it is even worse - when syspref cache is disabled:
#!/usr/bin/perl use Modern::Perl; use C4::Context;
C4::Context->disable_syspref_cache();
my @sysprefs = qw/ UseTransportCostMatrix StaticHoldsQueueWeight RandomizeHoldsQueueWeight/;
for (my $i = 0; $i < 1000; $i++) { for my $syspref_name (@sysprefs) { my $syspref_val = C4::Context->preference($syspref_name); } }
Runnig time - without patch: 4.4sec, with patch: 47.2 sec..
With syspref cache enabled, the difference is negglible (0.36 patched, 0.37 sec unpatched), but the patched version still seems to be a bit slower (but not much, <2 usec per call - hard to measure, in real life scenarios I guess it will also depend on the mysql query_cache_limit/size/type settings etc.).
Yes, I have tested *with* the syspref cache (I should have mentioned that in the commit msg!) What you described is expected, you are not running under Plack and for each call all pref are retrieved from the DB :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #8 from Frédéric Demians <frederic@tamil.fr> --- Is there a way to have Plack running without packages? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- /etc/apache2/sites-available/koha.conf <Proxy> Order deny,allow Allow from 127.0.0.1,localhost </Proxy> <location /cgi-bin/koha/> ProxyPass http://localhost:5001/cgi-bin/koha/ </location> Then I use plackup with misc/plack/koha.psgi -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Can't we use debian/templates/plack.psgi instead? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- You can use the one you want. I have just shared the method I use for years now. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15341 Heather Braum <hbraum@nekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hbraum@nekls.org -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org