[Bug 40856] New: Improve Standard backend metadata retrieval
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Bug ID: 40856 Summary: Improve Standard backend metadata retrieval Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: ILL Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org CC: lisette@bywatersolutions.com, pedro.amorim@openfifth.co.uk, tomascohen@gmail.com The metadata() method in the Standard ILL backend uses an inefficient pattern for filtering attributes, performing a linear search through an ignore list for every attribute using grep inside a while loop. This approach becomes increasingly slow as the number of attributes grows, and manually iterates through all attributes instead of leveraging database-level filtering. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |40855 Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40855 [Bug 40855] Standard backend uses plain SQL -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |38341 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38341 [Bug 38341] Standard backend is missing unit tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 --- Comment #1 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 186742 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186742&action=edit Bug 40856: Add unit tests for Standard backend metadata method This patch adds comprehensive unit tests for the metadata() method in the Standard ILL backend to establish baseline behavior before optimization. The tests cover: - Metadata structure (returns hashref) - Included attributes (title, author, isbn, year, custom fields) - Excluded attributes (ignored fields like requested_partners, type, etc.) - Display name transformation (attribute types become display names) Current behavior tested: - metadata() filters out predefined ignore list attributes - Uses linear search with grep for each attribute - Transforms attribute types to display names (title -> Title) - Custom fields get ucfirst transformation (custom_field -> Custom_field) - Returns hashref with display names as keys These tests document the current inefficient implementation and will ensure the upcoming optimization maintains compatibility. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t => SUCCESS: Tests pass! Current metadata behavior documented 3. Tests establish baseline for upcoming optimization 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 --- Comment #2 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 186743 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186743&action=edit Bug 40856: Optimize Standard backend metadata method with database-level filtering This patch optimizes the metadata() method in the Standard ILL backend to use database-level filtering instead of inefficient linear searches. Performance improvement: - Before: Load all attributes, then filter each with linear search - After: Load only relevant attributes using database index Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t => SUCCESS: Tests pass! Optimization maintains compatibility 3. Test ILL request metadata display: - View ILL request details - Verify metadata shows correctly - Check ignored fields are excluded => SUCCESS: Same functionality, better performance 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186742|0 |1 is obsolete| | --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 186760 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186760&action=edit Bug 40856: Add unit tests for Standard backend metadata method This patch adds comprehensive unit tests for the metadata() method in the Standard ILL backend to establish baseline behavior before optimization. The tests cover: - Metadata structure (returns hashref) - Included attributes (title, author, isbn, year, custom fields) - Excluded attributes (ignored fields like requested_partners, type, etc.) - Display name transformation (attribute types become display names) Current behavior tested: - metadata() filters out predefined ignore list attributes - Uses linear search with grep for each attribute - Transforms attribute types to display names (title -> Title) - Custom fields get ucfirst transformation (custom_field -> Custom_field) - Returns hashref with display names as keys These tests document the current inefficient implementation and will ensure the upcoming optimization maintains compatibility. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t => SUCCESS: Tests pass! Current metadata behavior documented 3. Tests establish baseline for upcoming optimization 4. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186743|0 |1 is obsolete| | --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 186761 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186761&action=edit Bug 40856: Optimize Standard backend metadata method with database-level filtering This patch optimizes the metadata() method in the Standard ILL backend to use database-level filtering instead of inefficient linear searches. Performance improvement: - Before: Load all attributes, then filter each with linear search - After: Load only relevant attributes using database index Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t => SUCCESS: Tests pass! Optimization maintains compatibility 3. Test ILL request metadata display: - View ILL request details - Verify metadata shows correctly - Check ignored fields are excluded => SUCCESS: Same functionality, better performance 4. Sign off :-D Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |martin.renvoize@openfifth.c | |o.uk --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Simple internal API improvement, well covered by tests. Straight for QA, Passed. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@openfifth.c |y.org |o.uk Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |25.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 --- Comment #6 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Laura Escamilla <Laura.escamilla@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Needs documenting CC| |Laura.escamilla@bywatersolu | |tions.com --- Comment #7 from Laura Escamilla <Laura.escamilla@bywatersolutions.com> --- This is an enhancement and will not be backported to the 25.05.x branch. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Aude Charillon <aude.charillon@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED CC| |aude.charillon@openfifth.co | |.uk Resolution|--- |FIXED --- Comment #8 from Aude Charillon <aude.charillon@openfifth.co.uk> --- No need for any updates to the Manual. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40856 Bug 40856 depends on bug 40855, which changed state. Bug 40855 Summary: Standard backend uses plain SQL https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40855 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org