[Bug 29454] New: Stash branches and itemtypes in plugin objects to reduce DB calls
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Bug ID: 29454 Summary: Stash branches and itemtypes in plugin objects to reduce DB calls Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Moved from bug 28436 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 127515 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127515&action=edit Bug 29454: Add local cache to Template Plugins for branches and itemtypes This patch simply stores the branches an itemtypes in a hash for lookup. This improves efficiency on pages where we are loading many items To test: 1 - Add ~1000 items to a record 2 - Note load time 3 - Apply patch 4 - Reload and note faster load 5 - Edit a branchname/itemtype description 6 - Reload page 7 - Confirm the updated name is reflected in display -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28436 Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127515|0 |1 is obsolete| | --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 127520 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127520&action=edit Bug 29454: Add local cache to Template Plugins for branches and itemtypes This patch simply stores the branches an itemtypes in a hash for lookup. This improves efficiency on pages where we are loading many items To test: 1 - Add ~1000 items to a record 2 - Note load time 3 - Apply patch 4 - Reload and note faster load 5 - Edit a branchname/itemtype description 6 - Reload page 7 - Confirm the updated name is reflected in display Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #3 from David Nind <david@davidnind.com> --- Testing notes using koha-testing-docker. I found that for the detail page for the record in the staff interface that the load time with 1,001 items was significantly improved. On the edit items page I didn't notice much, if any, improvement. After the patch was applied I did a flush_memcached and restart_all, but not fater changing the item type and library names. Curious as to why we don't use pagination when the there are lots of items, on the dipslay item page in the staff interface and OPAC. Some numbers (a bit variable, but generally around the same) - YMMV: Before patch applied ~~~~~~~~~~~~~~~~~~~~ Item page in the staff interface: 1027 requests 6.74 MB / 49.24 KB transferred Finish: 20.45 s DOMContentLoaded: 18.52 s load: 20.01 s Edit items: 25 requests 5.50 MB / 82.78 KB transferred Finish: 1.10 min DOMContentLoaded: 1.10 min load: 1.12 min After the patch applied ~~~~~~~~~~~~~~~~~~~~~~~ Item page: 1030 requests 7.19 MB / 970.08 KB transferred Finish: 17.07 s DOMContentLoaded: 14.66 s load: 16.11 s Edit items page: 27 requests 5.63 MB / 107.81 KB transferred Finish: 56.97 s DOMContentLoaded: 56.77 s load: 58.19 s After the patch applied and change to item type and library names ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Used the refresh button. Item page: 1027 requests 7.21 MB / 48.92 KB transferred Finish: 17.27 s DOMContentLoaded: 15.27 s load: 16.87 s Edit items: 36 requests 6.40 MB / 1.24 MB transferred Finish: 1.10 min DOMContentLoaded: 1.09 min load: 1.12 min -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com Status|Signed Off |In Discussion See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=26587 --- Comment #4 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- This is a duplicate of bug 26587. However I think the approach there is better (use of Koha::Cache::Memory::Lite), but it only deals with 'Branches'. If you agree with using the L1 cache, could you add your QA stamp there and provide a patch here for itemtypes? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #127520|0 |1 is obsolete| | --- Comment #5 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 128472 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=128472&action=edit Bug 29454: Use Koha Cache Memory Lite to stash itemtype descriptions for template plugin To test: 1 - Add 1000 items to a record, of varying item types 2 - Bring up the details page 3 - Note time to load 4 - Apply patch 5 - Reload page and compare to previous 6 - Confirm information is correct 7 - Confirm some performance benefit -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |jonathan.druart+koha@gmail. |y.org |com --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Thanks, Nick! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Stash branches and |Stash itemtypes in plugin |itemtypes in plugin objects |objects to reduce DB calls |to reduce DB calls | CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #7 from Fridolin Somers <fridolin.somers@biblibre.com> --- Maybe add an early return q{} if itemtypecode is undefined, to avoid warning in concatenation : my $cache_key = $want_parent ? "Itemtype_parent_description:".$itemtypecode : "Itemtype_description:" . $itemtypecode; -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=30920 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #128472|0 |1 is obsolete| | --- Comment #8 from Fridolin Somers <fridolin.somers@biblibre.com> --- Created attachment 136077 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136077&action=edit Bug 29454: Use Koha Cache Memory Lite to stash itemtype descriptions for template plugin Returns empty string if given item type is undefined or unknown To test: 1 - Add 1000 items to a record, of varying item types 2 - Bring up the details page 3 - Note time to load 4 - Apply patch 5 - Reload page and compare to previous 6 - Confirm information is correct 7 - Confirm some performance benefit Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #9 from Fridolin Somers <fridolin.somers@biblibre.com> --- Created attachment 136078 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136078&action=edit Bug 29454: Add unit test for Koha::Template::Plugin::ItemTypes -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #10 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to Fridolin Somers from comment #7)
Maybe add an early return q{} if itemtypecode is undefined, to avoid warning in concatenation : my $cache_key = $want_parent ? "Itemtype_parent_description:".$itemtypecode : "Itemtype_description:" . $itemtypecode;
I've added this. And also a write of empty string in cache when item type is not found. I signed first patch. Added a unit test for Koha::Template::Plugin::ItemTypes. This needs SO. Best regards, Run Koha, run ... ^^ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #136078|0 |1 is obsolete| | --- Comment #11 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 136090 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136090&action=edit Bug 29454: Add unit test for Koha::Template::Plugin::ItemTypes Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #12 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 136091 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136091&action=edit Bug 29454: (follow-up) Cover more test cases -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #13 from Fridolin Somers <fridolin.somers@biblibre.com> --- Great thanks. BTW shouldn't there be some space around "->" when there is a parent ? "Translated itemtype A->Translated itemtype B" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #136077|0 |1 is obsolete| | --- Comment #14 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 136149 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136149&action=edit Bug 29454: Use Koha Cache Memory Lite to stash itemtype descriptions for template plugin Returns empty string if given item type is undefined or unknown To test: 1 - Add 1000 items to a record, of varying item types 2 - Bring up the details page 3 - Note time to load 4 - Apply patch 5 - Reload page and compare to previous 6 - Confirm information is correct 7 - Confirm some performance benefit Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #136090|0 |1 is obsolete| | --- Comment #15 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 136150 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136150&action=edit Bug 29454: Add unit test for Koha::Template::Plugin::ItemTypes Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #136091|0 |1 is obsolete| | --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 136151 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=136151&action=edit Bug 29454: (follow-up) Cover more test cases Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA CC| |martin.renvoize@ptfs-europe | |.com --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work.. works well, tests passing and coverage is good.. PQA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|jonathan.druart+koha@gmail. |martin.renvoize@ptfs-europe |com |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |22.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 --- Comment #18 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.11.00 |22.11.00, 22.05.04 released in| | CC| |lucas@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #19 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for 22.05.04 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable Version(s)|22.11.00, 22.05.04 |22.11.00, 22.05.04, released in| |21.11.11 CC| |arthur.suzuki@biblibre.com --- Comment #20 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- backported to 21.11.x for 21.11.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29454 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|Pushed to oldstable |RESOLVED CC| |victor@tuxayo.net --- Comment #21 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Not backported to oldoldstable (21.05.x). Feel free to ask if it's needed. Nothing to document, marking resolved. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org