[Bug 43714] New: stockrotation.pl runs out for memory when rotas have too many ites
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Bug ID: 43714 Summary: stockrotation.pl runs out for memory when rotas have too many ites Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Staff interface Assignee: koha-bugs@lists.koha-community.org Reporter: lucas@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: gmcharlt@gmail.com Target Milestone: --- To recreate: 0. Enable StockRotation 1. Add a rota 2. Give the rota at least 2 stages 3. The rota needs lots of items, 1500+ ( i will add a small script to help build this ) 4. Got Cataloging -> Manage -> Items for that rota 5. The page will not load, or it will take a very long time to do so. ( in KTD with my hardware, my laptop crashed at 2x at this step ) 6. Eventually in the plack logs I see: Child process 3370045 exited with status 1 Out of memory! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #1 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Out of memory! cannot opendir for /tmp/cgi_compile_3378415: Cannot allocate memory at /usr/share/perl5/CGI/Compile.pm line 213. Child process 3378415 exited with status 1 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #2 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- I profiled this with NYProf. Shout out to kidclamp who is presenting about NyProf at KohaCon next week, check it out if you can! A stock rotation with 2 stages and 300 items in KTD: Profile of tools/stockrotation.pl for 68.4s (of 117s), executing 91865169 statements and 22445977 subroutine calls in 1607 source files and 495 string evals. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|stockrotation.pl runs out |stockrotation.pl runs out |for memory when rotas have |for memory when rotas have |too many ites |too many items and more | |than 1 stage -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|stockrotation.pl runs out |stockrotation.pl runs out |for memory when rotas have |for memory when rotas have |too many items and more |too many items in a stage |than 1 stage | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #3 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 207256 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207256&action=edit script to populate many items in a stockrota -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207256|0 |1 is obsolete| | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #4 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 207257 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207257&action=edit script to populate many items in a stockrota -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #5 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- In stockrotations.pl: my $sritems = Koha::StockRotationItems->search( { 'stage.rota_id' => $params{rota_id} }, { prefetch => { stage => { 'stockrotationitems' => { 'itemnumber' => 'biblionumber' } } } } ); The prefetch is getting: the stages, then the stockrotationitems, then ALL the items in the stockrotationitems, then all those item's biblios. Does this once for EVERY item in the rota. There should be a less round about way to arrive here: { prefetch => [ 'stage', { itemnumber => 'biblionumber' } ] } -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #6 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 207258 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207258&action=edit Bug 43714: Fix prefetch in stockrotation.pl To test: 0. Enable StockRotation 1. Add a rota 2. Give the rota at least 2 stages 3. The rota needs lots of items, 1500+ ( i will add a small script to help build this ) 4. Got Cataloging -> Manage -> Items for that rota 5. The page will not load, or it will take a very long time to do so. ( in KTD with my hardware, my laptop crashed at 2x at this step ) 6. For me, it never loads. Eventually it times out 7. APPLY PATCH and restart all. 8. Repeat sept 4. 9. This time the page should load much faster. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |lucas@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207258|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 --- Comment #7 from David Nind <david@davidnind.com> --- Created attachment 207264 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207264&action=edit Bug 43714: Fix prefetch in stockrotation.pl To test: 0. Enable StockRotation 1. Add a rota 2. Give the rota at least 2 stages 3. The rota needs lots of items, 1500+ ( i will add a small script to help build this ) 4. Got Cataloging -> Manage -> Items for that rota 5. The page will not load, or it will take a very long time to do so. ( in KTD with my hardware, my laptop crashed at 2x at this step ) 6. For me, it never loads. Eventually it times out 7. APPLY PATCH and restart all. 8. Repeat sept 4. 9. This time the page should load much faster. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #8 from David Nind <david@davidnind.com> --- Thanks for providing the script to add a lot of items to a stock rotation! Testing notes (using KTD): 1. To use the script: - download into root directory for Koha git clone - run using: perl script_to_populate_stockrota.pl - note: takes a while to complete 2. Before patch, page times out/fails after about 42 seconds with: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request Reason: Error reading from remote server Apache/2.4.68 (Debian) Server at 127.0.0.1 Port 8081 3. After the patch, the page loads in about 8 seconds. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43714 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@openfifth.c |y.org |o.uk -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org