[Bug 41343] New: Overdue report is too intensive on systems with many overdues
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Bug ID: 41343 Summary: Overdue report is too intensive on systems with many overdues Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Circulation Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: gmcharlt@gmail.com, kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #1 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 190062 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=190062&action=edit Bug 41343: Don't fetch patron objects per overdue We use SQL here to get all the needed info - the query says it is slow, but isn't so bad. Worse is fetching the patron for each borrower with overdues (and more than once if they have several) It appears we started grabbing the patron so we could use patron title, but we can just pass the info already fetched. I additionally fetch the patron category types and update patron-title to allow for hide_patron_infos_if_needed to work without a patron object - we can just use the branchcode as ultimately that's all we need To test: 1 - Add many items to your catalog (can use my handy repo randitems.pl) https://github.com/kidclamp/handy-koha-script 2 - Add many issues (can use randissues.pl from above) 3 - I tested with about 1000 4 - Browse to Circulation->Overdues 5 - Note load time 6 - Apply patch 7 - Restart all 8 - Note load time is faster -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #2 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 190063 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=190063&action=edit Bug 41343: Further improvements Remove unused patron attribute values sent to the template Send preferred name Use SQL to do date comparison to see if an item is overdue -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #3 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- For me this went from ~9 seconds to ~6 seconds with 1000 items Testing the SQL on a system with 100k checkouts, it takes less then 3 seconds, so could do better, but isn't too bad. We should maybe consider not loading all overdues by default, maybe a syspref to limit, or just default to 1000 items and then allow for loading more? See bug 29098 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #190062|0 |1 is obsolete| | --- Comment #4 from David Nind <david@davidnind.com> --- Created attachment 194199 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=194199&action=edit Bug 41343: Don't fetch patron objects per overdue We use SQL here to get all the needed info - the query says it is slow, but isn't so bad. Worse is fetching the patron for each borrower with overdues (and more than once if they have several) It appears we started grabbing the patron so we could use patron title, but we can just pass the info already fetched. I additionally fetch the patron category types and update patron-title to allow for hide_patron_infos_if_needed to work without a patron object - we can just use the branchcode as ultimately that's all we need To test: 1 - Add many items to your catalog (can use my handy repo randitems.pl) https://github.com/kidclamp/handy-koha-script 2 - Add many issues (can use randissues.pl from above) 3 - I tested with about 1000 4 - Browse to Circulation->Overdues 5 - Note load time 6 - Apply patch 7 - Restart all 8 - Note load time is faster Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #190063|0 |1 is obsolete| | --- Comment #5 from David Nind <david@davidnind.com> --- Created attachment 194200 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=194200&action=edit Bug 41343: Further improvements Remove unused patron attribute values sent to the template Send preferred name Use SQL to do date comparison to see if an item is overdue Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #6 from David Nind <david@davidnind.com> --- Testing notes (using KTD): 1. Performance improvements with 1609 overdue items: - Before: 11.86s - After: 7.06s 2. Used the hand scripts to create items and issues (had to Ctrl-C as they were taking a while): - perl ./randitems.pl -n 1000 - perl ./randissues.pl -p 42 3. In Chromium, used the developer tools to get the times: More tools > Developer tools > Performance tab -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #194199|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #194200|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 198126 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198126&action=edit Bug 41343: Don't fetch patron objects per overdue We use SQL here to get all the needed info - the query says it is slow, but isn't so bad. Worse is fetching the patron for each borrower with overdues (and more than once if they have several) It appears we started grabbing the patron so we could use patron title, but we can just pass the info already fetched. I additionally fetch the patron category types and update patron-title to allow for hide_patron_infos_if_needed to work without a patron object - we can just use the branchcode as ultimately that's all we need To test: 1 - Add many items to your catalog (can use my handy repo randitems.pl) https://github.com/kidclamp/handy-koha-script 2 - Add many issues (can use randissues.pl from above) 3 - I tested with about 1000 4 - Browse to Circulation->Overdues 5 - Note load time 6 - Apply patch 7 - Restart all 8 - Note load time is faster Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 198127 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198127&action=edit Bug 41343: Further improvements Remove unused patron attribute values sent to the template Send preferred name Use SQL to do date comparison to see if an item is overdue Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |26.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #9 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 26.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Jacob O'Mara <jacob.omara@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|26.05.00 |26.05.00,25.11.05 released in| | Status|Pushed to main |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 --- Comment #10 from Jacob O'Mara <jacob.omara@openfifth.co.uk> --- Thanks all, pushed to 25.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41343 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |42930 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42930 [Bug 42930] circ/overdue.pl incorrectly adds title into overdueloop instead of borrowertitle -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org