[Bug 16156] New: Do not ensure the dbh is connected all the time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Bug ID: 16156 Summary: Do not ensure the dbh is connected all the time Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: normal Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: jonathan.druart@bugs.koha-community.org Reporter: jonathan.druart@bugs.koha-community.org QA Contact: testopia@bugs.koha-community.org Blocks: 15342 I have profiled `/catalogue/search.pl?q=d` (20 results, xslt) and found than at least 600ms are spent in DBIx::Class::Storage::DBI::ensure_connected, which is called 5749 times (almost 1 per query). Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15342 [Bug 15342] Performance 3.22 - Omnibus -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 49632 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=49632&action=edit Bug 16156: Do not ensure the dbh is connected all the time I have profiled `/catalogue/search.pl?q=d` (20 results, xslt) and found that at least 600ms are spent in DBIx::Class::Storage::DBI::ensure_connected, which is called 5749 times (almost 1 per query). Ideally we could work on the basis that the connection is only checked when a thread is started. This is what does this patch. Unfortunately this patch will revive timeout issues with long run process (SIP server for instance). Before this patch: 901398 statements and 346866 subroutine calls in 406 source files and 55 string evals spent 637ms (33.0+604) within DBIx::Class::Storage::DBI::ensure_connected which was called 5749 times, avg 111µs/call After this patch: 817161 statements and 265940 subroutine calls in 406 source files and 55 string evals spent 49.8ms (6.58+43.2) within DBIx::Class::Storage::DBI::ensure_connected which was called 343 times, avg 145µs/call -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jweaver@bywatersolutions.co | |m, | |martin.renvoize@ptfs-europe | |.com, tomascohen@gmail.com Status|ASSIGNED |In Discussion --- Comment #2 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- We could also imagine a 'ensure_connected' flag, not to use the $context->{dbh} even if exists, for long run processes. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |abl@biblos.pk.edu.pl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=16168 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #3 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- And? Is this stupid? Any thoughts? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |glasklas@gmail.com --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- *** Bug 19819 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #5 from David Cook <dcook@prosentient.com.au> --- I don't have time to think about this right now, but I'd do things with persistent/long running processes as the target? I figure this is one of those things that is best handled at the start of a request but I don't know. I ran into this problem with Wildfly/JBoss and this is how I dealt with it: https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_appli... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to David Cook from comment #5)
I don't have time to think about this right now, but I'd do things with persistent/long running processes as the target?
To answer I am going to quote content from previous comments: (In reply to Jonathan Druart from comment #1)
Created attachment 49632 [details] [review] Unfortunately this patch will revive timeout issues with long run process (SIP server for instance).
(In reply to Jonathan Druart from comment #2)
We could also imagine a 'ensure_connected' flag, not to use the $context->{dbh} even if exists, for long run processes.
It is in discussion. So first "Do you think it makes sense?" then either we abandon or we go further and I continue to implement the idea. As you can see I have been waiting for "discussion" for more than 3 years now ;) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- As we want to minimize ensure_connected, we probably should not add a connected test to Koha::Database like bug 20089#c2 suggested? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #7)
As we want to minimize ensure_connected, we probably should not add a connected test to Koha::Database like bug 20089#c2 suggested?
c2 => c1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #9 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I don't think this is still valid. I am not seeing ensure_connect calls anymore (or at least way less), but I had a bit of trouble to make Nytprof working with the last ktd (?) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16156 --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Jonathan Druart from comment #9)
I don't think this is still valid. I am not seeing ensure_connect calls anymore (or at least way less), but I had a bit of trouble to make Nytprof working with the last ktd (?)
So... can we close? -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org