[Bug 39070] New: Elasticsearch Facets are not used/needed when finding record matches
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Bug ID: 39070 Summary: Elasticsearch Facets are not used/needed when finding record matches Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org We have found that in production, when loading large record sets, we can sometimes overload the ES server and cause timeouts. One possible cause here is the cost of generating facets (aggregations) when looking for matching records. We don't use the values here, so shouldn't request them -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 --- Comment #1 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 177655 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177655&action=edit Bug 39070: Don't request facets when searching for recrod matches This patch adds a new option to Elasticsearch to skip facets in build_query and build_query_compat. There should be no behavior change, however, queries to ES should be simpler To test: 1 - Export some record from Koha 2 - Stage the records for import, matching on 999c 3 - Confirm expected matches 4 - Switch batch to use ISBN for matching 5 - Confirm expected matches 6 - Apply patches, restart all 7 - Switch batch to use 999c 8 - Confirm matches 9 - Switch batch to use ISBN 10 - Confirm matches -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | CC| |tomascohen@gmail.com --- Comment #2 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Makes a whole lot of sense! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177655|0 |1 is obsolete| | --- Comment #3 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 177656 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177656&action=edit Bug 39070: Don't request facets when searching for recrod matches This patch adds a new option to Elasticsearch to skip facets in build_query and build_query_compat. There should be no behavior change, however, queries to ES should be simpler To test: 1 - Export some record from Koha 2 - Stage the records for import, matching on 999c 3 - Confirm expected matches 4 - Switch batch to use ISBN for matching 5 - Confirm expected matches 6 - Apply patches, restart all 7 - Switch batch to use 999c 8 - Confirm matches 9 - Switch batch to use ISBN 10 - Confirm matches Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 --- Comment #4 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- I agree with the approach. There's a lot of improvements to make in the area to avoid hitting ES too much. I followed the test plan ('Elasticsearch' needs to be selected). QA tools happy. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |dcook@prosentient.com.au |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177656|0 |1 is obsolete| | --- Comment #5 from David Cook <dcook@prosentient.com.au> --- Created attachment 177715 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177715&action=edit Bug 39070: Don't request facets when searching for recrod matches This patch adds a new option to Elasticsearch to skip facets in build_query and build_query_compat. There should be no behavior change, however, queries to ES should be simpler To test: 1 - Export some record from Koha 2 - Stage the records for import, matching on 999c 3 - Confirm expected matches 4 - Switch batch to use ISBN for matching 5 - Confirm expected matches 6 - Apply patches, restart all 7 - Switch batch to use 999c 8 - Confirm matches 9 - Switch batch to use ISBN 10 - Confirm matches Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: David Cook <dcook@prosentient.com.au> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elasticsearch Facets are |Elasticsearch facets are |not used/needed when |not used/needed when |finding record matches |finding record matches -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |25.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 25.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Miranda Nero <mnero@oslri.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mnero@oslri.net -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|25.05.00 |25.05.00,24.11.10 released in| | Status|Pushed to main |Pushed to stable CC| |fridolin.somers@biblibre.co | |m --- Comment #7 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 24.11.x for 24.11.10 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39070 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Needs documenting --- Comment #8 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Not backporting to 22.11.x as it wasn't backported to 24.05.x -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org