[Bug 30882] New: Add max_result_window to index config
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Bug ID: 30882 Summary: Add max_result_window to index config Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: fridolin.somers@biblibre.com Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-module... We use this setting : https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c... I propose we add this setting in index config. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=27043 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |fridolin.somers@biblibre.co |ity.org |m -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #1 from Fridolin Somers <fridolin.somers@biblibre.com> --- This can be usefull for really big catalogs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #2 from Fridolin Somers <fridolin.somers@biblibre.com> --- Created attachment 135572 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=135572&action=edit Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-module... We use this setting : https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c... I propose we add this setting in index config. Test plan : 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes : misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index : curl '<cluster>:9200/<myindex>_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see : "max_result_window" : "1000000" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=30883 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #3 from David Nind <david@davidnind.com> --- Hi Fridolin. Using koha-testing-docker, how do I get step 4 to work? The curl command provided does not work for me. Thanks! David -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #4 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to David Nind from comment #3)
Hi Fridolin.
Using koha-testing-docker, how do I get step 4 to work?
The curl command provided does not work for me.
Thanks!
David
In KTD, use : curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #5 from David Nind <david@davidnind.com> --- (In reply to Fridolin Somers from comment #4)
In KTD, use : curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**. max_result_window'
Thanks Fridolin! That worked after adding this to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" However, the result I got was: "max_result_window" : "10000" I think that is because this is the value in the patch (index.max_result_window: 10000)! Should that be index.max_result_window: 1000000? 8-) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #6 from Fridolin Somers <fridolin.somers@biblibre.com> --- Ahhh good catch. Could you fix the test plan when you push your patch ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #7 from David Nind <david@davidnind.com> --- (In reply to Fridolin Somers from comment #6)
Ahhh good catch. Could you fix the test plan when you push your patch ?
No problem! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #135572|0 |1 is obsolete| | --- Comment #8 from David Nind <david@davidnind.com> --- Created attachment 135751 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=135751&action=edit Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-module... We use this setting: https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c... I propose we add this setting in index config. Test plan: 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes: misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index (when using koha-testing-docker*): curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see: "max_result_window" : "1000000" * You also need to add this setting to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #9 from David Nind <david@davidnind.com> --- I amended the patch to change 10000 to 1000000. I also amended the commit message to update the test plan - updated the curl statement and added koha-testing-docker information. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Nind from comment #9)
I amended the patch to change 10000 to 1000000.
Thats a courageous change! I agree that 10K is too low. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #11 from David Nind <david@davidnind.com> --- (In reply to Marcel de Rooy from comment #10)
Thats a courageous change!
Fridolin is the courageous one! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #135751|0 |1 is obsolete| | --- Comment #12 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 137445 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137445&action=edit Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-module... We use this setting: https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c... I propose we add this setting in index config. Test plan: 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes: misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index (when using koha-testing-docker*): curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see: "max_result_window" : "1000000" * You also need to add this setting to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |victor@tuxayo.net CC| |victor@tuxayo.net Status|Signed Off |Passed QA --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- I don't know what value should we have so I'm following Fridolin and Marcel ^^ Works, qa script happy, code looks good, passing QA :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 --- Comment #15 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 137834 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137834&action=edit Bug 30882: (QA follow-up) Fix tests Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This updates the number of release notes| |results set by default in | |Elasticsearch for the | |setting | |"index.max-result-window" | |from 10,000 to 1,000,000. | |This can be useful for | |really large catalogs. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.11.00 |22.11.00, 22.05.05 released in| | CC| |lucas@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #16 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for 22.05.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable Version(s)|22.11.00, 22.05.05 |22.11.00, 22.05.05, released in| |21.11.12 CC| |arthur.suzuki@biblibre.com --- Comment #17 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- thx pushed to 21.11.x for 21.11.12 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30882 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED --- Comment #18 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Not backported to oldoldstable (21.05.x). Feel free to ask if it's needed. Nothing to document, marking resolved. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org