[Bug 13226] New: Significant slowdown of checkins with fix for notices ignoring the dateformat preference
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Bug ID: 13226 Summary: Significant slowdown of checkins with fix for notices ignoring the dateformat preference Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Circulation Assignee: koha-bugs@lists.koha-community.org Reporter: oleonard@myacpl.org QA Contact: testopia@bugs.koha-community.org CC: gmcharlt@gmail.com, kyle.m.hall@gmail.com I noticed that checkins were being processed very slowly and ran some tests. I found that before the patch for Bug 11244 checking in an item which is checked out to a restricted patron took around 2.5 seconds (as measured by Firebug's Net panel). After Bug 11244 the same type of checkin took over 50 seconds. Checking in an item which is checked out to a patron in good standing doesn't seem to be affected. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=11244 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Very interesting! I assume that your patrons get "check in" emails? We don't do those kinds of emails so I never even thought about that happening... Amazing that it would go from 2.5 seconds to 50 seconds though :S. That's much much higher than the worst time we recorded for generating overdues. Olli-Antti Kivilahti processed 12 messages in 31 seconds. I processed 35 messages in 31 seconds. Astounding that it would be 1 message in 50 seconds. I suppose that regex would be running over every single replaceable field. Does your check in message have a lot of fields to replace with database data? I'm not sure how to fix this one. The regex idea seemed like the best way of dealing with the dateformat issue in notices :S. I thought about using DBIC to check what type of field we were using, but you can't do that because many table names don't translate to DBIC resultset names. DBIC is pretty slow too though, so I don't know. I'm fresh out of ideas on this one :( -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m --- Comment #2 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to David Cook from comment #1)
I suppose that regex would be running over every single replaceable field. Does your check in message have a lot of fields to replace with database data?
Yes, could you give a copy of the template letter used? I have the default one (installer/data/mysql/en/mandatory/sample_notices.sql), I manually added a restriction to a patron, checkin, and it took the same time as before (I didn't measured precisely but not 50sec). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #3 from Owen Leonard <oleonard@myacpl.org> --- (In reply to Jonathan Druart from comment #2)
Yes, could you give a copy of the template letter used?
I have the default one. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #4 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 33426 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33426&action=edit time checkin Owen, could you execute this script like: time perl time_checkin.pl and give me the result please? Before, fill the $barcode_to_checkin and $branchcode variable with correct values. I would like to be sure the 50sec are spent in this subroutine. Here it spent 1sec. I check an item out to a patron, enable the CHECKIN notice for this patron and add a restriction on it. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #5 from Owen Leonard <oleonard@myacpl.org> ---
time perl time_checkin.pl and give me the result please?
real 0m51.725s user 0m50.347s sys 0m0.468s -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #6 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 33485 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33485&action=edit DO NOT PUSH: profiling for _parseletter -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #7 from Jonathan Druart <jonathan.druart@biblibre.com> --- You can also try to put some warns (see this patch). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33485|0 |1 is obsolete| | --- Comment #8 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 33494 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33494&action=edit Bug 13226: 9999-12-31 should not considered as a valid date DateTime::Format::DateParse (called in Koha::DateUtils::dt_from_string) does not manage to parse 9999-12-31 if a time zone is given. my $date = DateTime->new(year => 9999, month => 12, day => 31); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31' ); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31', 'America/Los_Angeles' ); => KO (~20sec on my laptop) It should not be considered as a valid date when the letter is parsed. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |jonathan.druart@biblibre.co |ity.org |m -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33494|0 |1 is obsolete| | --- Comment #9 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 33495 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33495&action=edit Bug 13226: 9999-12-31 should not considered as a valid date DateTime::Format::DateParse (called in Koha::DateUtils::dt_from_string) does not manage to parse 9999-12-31 if a time zone is given. my $date = DateTime->new(year => 9999, month => 12, day => 31); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31' ); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31', 'America/Los_Angeles' ); => KO (~20sec on my laptop) It should not be considered as a valid date when the letter is parsed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #10 from Jonathan Druart <jonathan.druart@biblibre.com> --- It seems that this patch does not completely fix the problem. I still have a 9999-12-31 somewhere else. But not introduced by bug 11244. Owen could you test it please? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #11 from Owen Leonard <oleonard@myacpl.org> --- (In reply to Jonathan Druart from comment #10)
It seems that this patch does not completely fix the problem.
With this patch I see a significant improvement: Check-ins from the account of a patron who is restricted indefinitely have gone from taking around 50 seconds to between 7 and 10 seconds. It's still slower than the 2-3 seconds for other checkins, but much better. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33495|0 |1 is obsolete| | --- Comment #12 from Owen Leonard <oleonard@myacpl.org> --- Created attachment 33496 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33496&action=edit [SIGNED-OFF] Bug 13226: 9999-12-31 should not considered as a valid date DateTime::Format::DateParse (called in Koha::DateUtils::dt_from_string) does not manage to parse 9999-12-31 if a time zone is given. my $date = DateTime->new(year => 9999, month => 12, day => 31); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31' ); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31', 'America/Los_Angeles' ); => KO (~20sec on my laptop) It should not be considered as a valid date when the letter is parsed. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Note that to reproduce the problem you much be checking in items from an account which has been restricted indefinitely (either manually or by the overdues process). With this patch such checkins go from taking around 50 seconds (in my test system) to around 7 to 10 seconds. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #13 from David Cook <dcook@prosentient.com.au> --- (In reply to Owen Leonard from comment #12)
Note that to reproduce the problem you much be checking in items from an account which has been restricted indefinitely (either manually or by the overdues process). With this patch such checkins go from taking around 50 seconds (in my test system) to around 7 to 10 seconds.
Does that mean that check-ins for non-restricted patrons are processed quickly (e.g. 2-3 seconds)? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #14 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to David Cook from comment #13)
(In reply to Owen Leonard from comment #12)
Note that to reproduce the problem you much be checking in items from an account which has been restricted indefinitely (either manually or by the overdues process). With this patch such checkins go from taking around 50 seconds (in my test system) to around 7 to 10 seconds.
Does that mean that check-ins for non-restricted patrons are processed quickly (e.g. 2-3 seconds)?
Yes. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 --- Comment #15 from Jonathan Druart <jonathan.druart@biblibre.com> --- Owen, please have a look at bug 13242 to get rid of some other seconds. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=13250 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33496|0 |1 is obsolete| | --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 33539 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33539&action=edit [PASSED QA] Bug 13226: 9999-12-31 should not considered as a valid date DateTime::Format::DateParse (called in Koha::DateUtils::dt_from_string) does not manage to parse 9999-12-31 if a time zone is given. my $date = DateTime->new(year => 9999, month => 12, day => 31); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31' ); => OK DateTime::Format::DateParse->parse_datetime( '9999-12-31', 'America/Los_Angeles' ); => KO (~20sec on my laptop) It should not be considered as a valid date when the letter is parsed. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Note that to reproduce the problem you much be checking in items from an account which has been restricted indefinitely (either manually or by the overdues process). With this patch such checkins go from taking around 50 seconds (in my test system) to around 7 to 10 seconds. Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Good catch! Works as described, no problems found. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13226 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |tomascohen@gmail.com --- Comment #17 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Fix pushed to master. Thanks Jonathan! -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org