[Bug 43523] New: cleanborrowers.pl is slow
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Bug ID: 43523 Summary: cleanborrowers.pl is slow Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: lucas@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Target Milestone: --- I have been looking at cleanborrowers.pl and find that it can be slow when dealing with many borrowers. I have identified some areas where I think improvements can be easily made. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=42509 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 --- Comment #1 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 205598 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205598&action=edit Bug 43523: Replace _skip_borrowers_with_nonzero_balance with SQL Patch from commit 8138883 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|cleanborrowers.pl is slow |Initial patron lookup in | |cleanborrowers.pl is slow -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 --- Comment #2 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- This improves the speed of the initial patron lookup but doesn't help with the actual deletion. That should be moved to a background job, in my opinion. I filed Bug 43525. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 --- Comment #3 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- I loaded 10,000 patrons into KTD. And did a query that would return all of the, I chose "patrons who have not borrowed since: 1900/01/01 WITHOUT THE PATCH: the initial lookup prior to deleting takes 43.14 seconds WITH PATCH: the initial lookup prior to deleting takes less than 1 second -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |lucas@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #205598|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 --- Comment #4 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 205631 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205631&action=edit Bug 43523: Replace _skip_borrowers_with_nonzero_balance with SQL To test: 0. Add many borrowers to your system, I was testing with 10,000 each time 1. First without the patch, go to Tools -> Batch patron deletion and anonymization 2. Check 'Verify you want to delete patrons' and choose paramaters that will return most, or all, of the 10k borrowers. I used "who have not borrowed since: 1900/01/01" 3. Click Next >> and confirm. 4. The script should now take significant time to load. For me, with 10k borrowers, it was 3minutes 45 seconds 5. APPLY PATCH, restart_all 6. Load 10,000 new borrowers since the old ones are now deleted. 7. Try the steps again. 8. Load times should be better. For me, it now loads in under 1 second. 9. prove -v t/db_dependent/Members.t NOTE: This doesn't effect the actual deletion which still takes quite a long time. I have filed Bug 43525. as the deletion should be moved to a background job. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 --- Comment #5 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Created attachment 205632 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205632&action=edit Bug 43523: Add a new test Patch from commit c500a00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43523 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org