New subject: [Bug 31735] Circulation optimizations

11 Oct 2022

      https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31735

            Bug ID: 31735
           Summary: Circulation optimizations
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Architecture, internals, and plumbing
          Assignee: koha-bugs@lists.koha-community.org
          Reporter: glasklas@gmail.com
        QA Contact: testopia@bugs.koha-community.org

We recently had time-out issues and high server loads for patrons with high
checkout counts (some patrons have several hundreds, up to about 400). One of
the major bottlenecks turned out to be C4::Circulation::CanBookBeRenewed. The
primary reason it performed so poorly was constant re-fetching of different
database entities which could just as well be passed down directly instead of
passing ids which requires re-fetching further down the call stack.

After refactoring the most performance critical subroutines to take objects
instead of object ids the calls to expensive DBIx-methods has been greatly
reduced. In addition some other optimizations and bug fixes related to non
working caching resulted in further speedups. The performance gain for a patron
with around 400 checkouts is currently about 500%.

I made an effort to fix all tests affected by the changes, but I can not
guarantee there are absolutely no regressions left in the test-suite. Since
many tests seem to fail due to reasons not related to this patch, mainly due to
the state of my local instance triggering tests-bugs (for example caused by
preferences not being mocked correctly, or relying on database test-entries
that are not guaranteed to exist), it was impractical and very time consuming
to make sure all tests still run successfully. So subset of tests where the
changed subroutines are called directly was selected and these should now pass.

The following tests should be effected by the changes and should be run as a
part of the test-plan:

t/db_dependent/api/v1/biblios.t
t/db_dependent/api/v1/checkouts.t
t/db_dependent/api/v1/return_claims.t
t/db_dependent/Circulation/CalcDateDue.t
t/db_dependent/Circulation/CheckIfIssuedToPatron.t
t/db_dependent/Circulation/dateexpiry.t
t/db_dependent/Circulation/GetPendingOnSiteCheckouts.t
t/db_dependent/Circulation/GetTopIssues.t
t/db_dependent/Circulation_holdsqueue.t
t/db_dependent/Circulation/IsItemIssued.t
t/db_dependent/Circulation/issue.t
t/db_dependent/Circulation/MarkIssueReturned.t
t/db_dependent/Circulation/maxsuspensiondays.t
t/db_dependent/Circulation/ReturnClaims.t
t/db_dependent/Circulation/Returns.t
t/db_dependent/Circulation/SwitchOnSiteCheckouts.t
t/db_dependent/Circulation.t
t/db_dependent/Circulation/TooMany.t
t/db_dependent/Circulation/transferbook.t
t/db_dependent/DecreaseLoanHighHolds.t
t/db_dependent/Holds/DisallowHoldIfItemsAvailable.t
t/db_dependent/HoldsQueue.t
t/db_dependent/Holds/RevertWaitingStatus.t
t/db_dependent/Illrequests.t
t/db_dependent/ILSDI_Services.t
t/db_dependent/Items.t
t/db_dependent/Koha/Account/Line.t
t/db_dependent/Koha/Acquisition/Order.t
t/db_dependent/Koha/Biblio.t
t/db_dependent/Koha/Holds.t
t/db_dependent/Koha/Items.t
t/db_dependent/Koha/Item.t
t/db_dependent/Koha/Object.t
t/db_dependent/Koha/Patrons.t
t/db_dependent/Koha/Plugins/Circulation_hooks.t
t/db_dependent/Koha/Pseudonymization.t
t/db_dependent/Koha/Recalls.t
t/db_dependent/Koha/Recall.t
t/db_dependent/Koha/Template/Plugin/CirculationRules.t
t/db_dependent/Letters/TemplateToolkit.t
t/db_dependent/Members/GetAllIssues.t
t/db_dependent/Members/IssueSlip.t
t/db_dependent/Patron/Borrower_Discharge.t
t/db_dependent/Patron/Borrower_PrevCheckout.t
t/db_dependent/Reserves/GetReserveFee.t
t/db_dependent/Reserves.t
t/db_dependent/rollingloans.t
t/db_dependent/selenium/regressions.t
t/db_dependent/SIP/ILS.t

The following still fails for me but should either work on a fresh Koha
installation, or fail for unrelated reasons (that is they also fail in Koha
master):

t/db_dependent/Koha/Plugins/Circulation_hooks.t
t/db_dependent/Koha/Items.t
t/db_dependent/SIP/ILS.t

To be able to run t/db_dependent/Circulation.t in my Koha instance there where
some existing bugs that needed fixing, so this patch have a dependency on Bug
31676.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.

[Bug 31735] New: Circulation optimizations

tags

participants (1)