[Bug 23649] New: Elasticsearch offer an option to weight search results by type
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Bug ID: 23649 Summary: Elasticsearch offer an option to weight search results by type Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com Some libraries would like to be able to prioritize "books" over "large print" , audios, etc in the search results -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Barbara Johnson <barbara.johnson@bedfordtx.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |barbara.johnson@bedfordtx.g | |ov --- Comment #1 from Barbara Johnson <barbara.johnson@bedfordtx.gov> --- Providing a way to organize the different formats for a title would provide an expected and reliable results display. It would improve searching for both patrons and staff. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Mwoo <mary.woodward@bedfordtx.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mary.woodward@bedfordtx.gov --- Comment #2 from Mwoo <mary.woodward@bedfordtx.gov> --- Assigning "weight" values to items would be a huge timesaver and alleviate a lot of patron frustrations. Our patrons will often put a hold on a large print book simply because it shows up first in their search. Invariably they didn't want the large print and are mad and disappointed. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #3 from Mwoo <mary.woodward@bedfordtx.gov> --- Assigning "weight" values to items would be a huge timesaver and alleviate a lot of patron frustrations. Our patrons will often put a hold on a large print book simply because it shows up first in their search. Invariably they didn't want the large print and are mad and disappointed. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com Status|NEW |In Discussion -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Christopher Davis <cgdavis@uintah.utah.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cgdavis@uintah.utah.gov -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Björn Nylén <bjorn.nylen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bjorn.nylen@ub.lu.se --- Comment #4 from Björn Nylén <bjorn.nylen@ub.lu.se> --- Created attachment 146585 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=146585&action=edit [Do not push] - Experimental support for function_score We've been experimenting a bit with function_score query in our ES-searches as we're having the same problems with not very relevant records showing high in the results. The patch allows us to assign different function_score-functions to promote/demote records in the resultlist. It's a bit of work in progress but if someone wants to try it out: Apply patch Create a Local use syspref, ElasticsearchFunctionScore , textarea Fill in syspref with yaml representing av array of function scores as understood by ES. Eg(modify itype to local condidtions): - filter: match: itype: "3" weight: 1.5 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #5 from Björn Nylén <bjorn.nylen@ub.lu.se> --- More interesting things can be done eg. maths: In this case we want to promote newer books over older. So define a new numeric field 'pubyear-numeric' in the ES mappings and map it to 008/07-10. After some existance- and sanity-checking we weight records (in this case) newer than 1980 higher. - script_score: script: source: "long y = doc['pubyear-numeric'].size()==0 ? params.start : doc['pubyear-numeric'].value; y = (y<params.start || y>2100) ? params.start : y; return (y-params.start)/params.divider+1;" params: start: 1980 divider: 20 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Michael Adamyk <madamyk@ckls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |madamyk@ckls.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #6 from Michael Adamyk <madamyk@ckls.org> --- This would be great! We often get random items like cake pans, kits, etc. near the top of our search results, which can be frustrating. More customization for search weights would be fantastic. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Steve, OSLRI, USA <sspohn@oslri.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bywater@oslri.net, | |sspohn@oslri.net -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 carthur@slolibrary.org <carthur@slolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |carthur@slolibrary.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #7 from carthur@slolibrary.org <carthur@slolibrary.org> --- This would be useful for SLO Libraries as well. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Chip Halvorsen <Chip.Halvorsen@WestlakeLibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |Chip.Halvorsen@WestlakeLibr | |ary.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Jason Robb <jrobb@sekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jrobb@sekls.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |domm@plix.at --- Comment #8 from Thomas Klausner <domm@plix.at> --- We would also be interested in this feature. I will aks the client if they want to fund it. If not, I will definitely QA it :-) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Mathieu Saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mathsabypro@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Lauren Denny <lauren_denny@sil.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lauren_denny@sil.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Sally <sally.healey@cheshiresharedservices.gov.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sally.healey@cheshireshared | |services.gov.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 George Williams (NEKLS) <george@nekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |george@nekls.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Michelle Spinney <mspinney@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mspinney@clamsnet.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |pedro.amorim@openfifth.co.u |ity.org |k CC| |pedro.amorim@openfifth.co.u | |k Status|In Discussion |ASSIGNED --- Comment #9 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Hi all, I'll be working on this. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #10 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200650 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200650&action=edit Bug 23649: Add search_field_value_boost DB table Adds a new table search_field_value_boost to store per-value relevance multipliers for facetable search fields (e.g. boost itype 'BK' by 2.0). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #11 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200651 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200651&action=edit Bug 23649: [DO NOT PUSH] DBIC schema for search_field_value_boost Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #12 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200652 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200652&action=edit Bug 23649: Add atomicupdate for search_field_value_boost table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #13 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200653 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200653&action=edit Bug 23649: Add Koha::SearchFieldValueBoost ORM classes Adds Koha::SearchFieldValueBoost (singular) and Koha::SearchFieldValueBoosts (set) as thin Koha::Object wrappers around the search_field_value_boost table. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #14 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200654 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200654&action=edit Bug 23649: Apply function_score value boosts in ES QueryBuilder Wraps the biblios Elasticsearch query in a function_score query when any search_field_value_boost rows are configured. Each row becomes a filter+weight function: documents matching a boosted value have their relevance score multiplied by the configured weight; documents matching no filter are unaffected (neutral 1.0 multiplier). score_mode 'max' takes the highest matching weight per document; boost_mode 'multiply' applies it against the original query score. Boost functions are cached for 5 minutes under the elasticsearch_value_boost_functions key, cleared alongside the existing search fields cache when mappings are saved. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #15 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200655 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200655&action=edit Bug 23649: Add value boost configuration to ES mappings admin Adds a 'Relevance boosts' section to the biblios tab of the Elasticsearch mappings admin page. Librarians can add per-value weight multipliers for any facetable field (itype, location, ccode, etc.) and delete existing ones. Each entry is saved to search_field_value_boost and the boost cache is invalidated on save. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #16 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Created attachment 200656 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200656&action=edit Bug 23649: Add tests for function_score value boosts in QueryBuilder Tests _get_value_boost_functions and the function_score wrapping in build_query. Covers: empty arrayref when no boosts configured, correct filter/weight structure per boost, score_mode/boost_mode values, the original bool query preserved inside function_score, and authorities index not wrapped. Mocks _get_value_boost_functions at module level so existing build_query tests are unaffected by any value boosts present in the test DB or cache. 0) Launch ktd with elastic search: $ ktd --search-engine es8 up 1) Apply patches, updatedatabase and restart plack. 2) Do a search for 'music' and leave this tab available (so we can come back to it and compare relevance sorting): <staff_url>/cgi-bin/koha/catalogue/search.pl?q=music 3) Go to the Elasticsearch mappings admin page: <staff_url>/cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl 4) Click the 'bibliographic records' tab. Confirm a 'Relevance boosts' section appears at the bottom with an empty table and an add form. 5) In the add form, select a facetable field (e.g. 'Item type'), enter value 'MU' and weight '2.00'. Click 'Add'. 6) Confirm the boost appears in the table with the correct field, value and weight. 7) Add a second boost on the same field (Item type) with value 'BK' and weight '0.5'. Click 'Add'. 8) Confirm both boosts are listed. 9) Perform the same 'music' search on a new tab: <staff_url>/cgi-bin/koha/catalogue/search.pl?q=music 10) Confirm that 'MU' (music) biblios now appear first. And BK (Books) only show more prominently towards the end of the results. 11) Click 'Delete' on the MU boost. Confirm it disappears from the table. 12) Perform the search again. Notice many 'MU' music biblios still show first, because BK (Books) are still being penalized (0.50 weight). But e.g. biblio 437 'Clothes, clothes, clothes' now appears in position #15 instead of position #20. 12) Go to the 'authorities' tab. Confirm no 'Relevance boosts' section appears there. 13) Try adding a new search field value boost with the '+Add' button with an empty field value. Confirm an error message appears and nothing is saved. 14) Try submitting with weight '0' or a negative number. Confirm a "must be a positive decimal" error appears. Note: boosts are applied at search time, not index time — changing or deleting a boost takes effect immediately with no reindexing required. prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t prove t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Reset.t prove t/db_dependent/Koha/SearchField.t prove t/00-load.t Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elasticsearch offer an |Elasticsearch offer an |option to weight search |option to weight search |results by type |results by value of facet | |field -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 --- Comment #17 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Hi all, I've widened the scope here a bit to allow configuring weight based on value of a facet field including, but not limited to, item type. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Jason Robb <jrobb@sekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #18 from Jason Robb <jrobb@sekls.org> --- Thanks for expanding this beyond item type! Overall the boosting seems to be working well. I had to skip over the first attachment (146585) to get things to work. The main issue I ran into is that the big yellow Save button below the table clears any boosts that you set. The Save button itself is misleadingly close to the boost settings so it is pretty easy to click it thinking it will save your boosts. UX-wise, adding weights is a little frustrating since it is at the bottom of a very long page and you can only do one at a time. Every time you add or delete a weight, you have to click the "Bibliographic record" tab again and scroll all the way down past the 370-row table. It would be great if the reload sent you back directly to the same area of the page. The placeholder text for the Value inputs "e.g. BK" is referencing an AV that may not exist in production systems since item type codes can vary. It would be handy if the Value column could use AV dropdowns for relevant Search fields to avoid input errors. I hit a few quirks while searching as well. Using a tilde operator with a test search of bird~ gave me 7 results. For both homebranch and holdingbranch I tried weighting MPL to 2 which put the Midway items at 1, 2 and 4 with a lone Fairview at 3. Weighting CPL puts Centerville items at 1, 2, 3 and 5 with the Fairview item at 4. The weighting does seem to be accurate in regular (no tilde) searches. When testing author weights, weighting a single author only worked if the punctuation was included (i.e. weighting "Aristophanes" doesn't work, but "Aristophanes." with the period does.) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23649 Fiona Borthwick <fiona.borthwick@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fiona.borthwick@openfifth.c | |o.uk -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org