https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74245|0 |1 is obsolete| | --- Comment #2 from David Gustafsson <glasklas@gmail.com> --- Created attachment 74249 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74249&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) No reindex is really required, but still, reindex authorities and biblios to verify that this still works. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74249|0 |1 is obsolete| | --- Comment #3 from David Gustafsson <glasklas@gmail.com> --- Created attachment 74301 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74301&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) No reindex is really required, but still, reindex authorities and biblios to verify that this still works. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74301|0 |1 is obsolete| | --- Comment #5 from David Gustafsson <glasklas@gmail.com> --- Created attachment 74373 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74373&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) No reindex is really required, but still, reindex authorities and biblios to verify that this still works. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74373|0 |1 is obsolete| | --- Comment #7 from David Gustafsson <glasklas@gmail.com> --- Created attachment 74418 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74418&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) No reindex is really required, but still, reindex authorities and biblios to verify that this still works. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74418|0 |1 is obsolete| | --- Comment #9 from David Gustafsson <glasklas@gmail.com> --- Created attachment 74419 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74419&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) No reindex is really required, but still, reindex authorities and biblios to verify that this still works. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18316 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |glasklas@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alex.arnaud@biblibre.com --- Comment #10 from Alex Arnaud <alex.arnaud@biblibre.com> --- Searchable parameter on search fields (staff/opac) is nice. But, is searchable parameter on mapping useful? We can just remove the mapping, right? This patch could be rebased on bug 18316, that brings boost for simple search. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #12 from David Gustafsson <glasklas@gmail.com> --- And about the searchable parameter, I think we are going to need this in the future so included it as did not see a major downside of doing so. The problem this might solve is that we would want to have exclusive "sort"-field mappings, not included in the main field. The idea is to allow mappings to be written on the form "035a|035b|035c", and this would mean that the first field that has a value gets mapped, the rest skipped. We have disabled sorting since it's very unpredictable for multivalued fields, and this was seen as a possible future solution. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #13 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi David, if you write bug 18316 instead of bug_18316, bugzilla will automagically link it. :) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |BLOCKED -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #15 from David Gustafsson <glasklas@gmail.com> --- I think this patch in it's current state needs to be refactored with regard to the changes introduces in Bug 18316 once has been pushed to master. I will do this, but will probably wait until it has been pushed (which it seems close to be). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Bug 20589 depends on bug 18316, which changed state. Bug 18316 Summary: Add weighting/relevancy options to ElasticSearch https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18316 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #18 from David Gustafsson <glasklas@gmail.com> --- This is still work in progress. Have not adjusted tests for the refactored code for example. There are parts of this patch regarding fixes for authorities search that could be a separate patch, but since removing '_all' also requires adjusting this patch such a patch would probably cause merge conflict with this one. So it could be simple to bundle theses changes as one patch. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #20 from David Gustafsson <glasklas@gmail.com> --- I will write a summary of the changes with motivations behind each one as this can be difficult to understand by just looking at the code. I'm also a little bit confused my self with regards to the correct behaviour of build_auth_query_compat for example. And would probably have to setup a Zebra instance to find out exactly how the different options are supposed to work. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #22 from David Gustafsson <glasklas@gmail.com> --- Here is a more verbose explaction of most of the changes: - Remove syspref QueryWeightFields as this seems not to be used anyway. - Replace hard coded 'biblios' and 'authorities' in mappings.pl with constants. - Add _search_fields method in QueryParser for returning weighted/unweighted search fields for OPAC/STaffClient (as optional subfield, used primarily for authorities search). Refactor authorities search. This part is a a litte bit iffy: What mainly confuses me is the motivation behind the differentiation between is/=, and "exact". With regards to elastic that operator only seems to be used in C4/Matcher.pm. It also occurs in Koha/QueryParser/Driver/PQF.pm and Koha/REST/Plugin/Query.pm but i don't believe Elastic is used as backend for those. What I can gather there where actually no difference between is/= and "exact" before my changes as is/= performed a "term" query on "<field>.phrase" and "exact" performed a "match_phrase" on the same field, which is pointless since the "phrase" subfield is not tokenized (used keyword analyser). So I simplified this and use the same query for both is/= and "exact". As "exact" queries seems to be used for matching, I also thought it would be more fitting to perform a case insensitive match instead of also removing punctuation (which could result in unexpected matches). I also noticed that the Authorities type drop-down in the authorities search was not respected, so this is not included in the query if selected. Also noted there is an $and_or option to build_authorities_query_compat that is not used, this should probably be addressed, but not fixed in this patch. Tidy up the index/fields configuration a little bit: - Rename all occurrences of misspelled "analyser" to "analyzer". - Rename "my_normalizer" to more descriptive "nfkc_cf_normalizer". - Rename "normalizer_keyword" to more descriptive "icu_folding_normalizer". - Rename subfield "lc_raw" to "ci_raw" (case insensitive). I don't think this subfield actually was case insensitive before since "my_normalizer" was used wich is only performing UTF normalization, not case folding. - Don't use "phrase" subfield for sorting, this is incorrect and will result in strange results since byte order of characters is used. There is also no need for a subfield for the sort field. I removed the subfield and changed type to "icu_collation_keyword", which will attempt to sort in order with the least conflicts between languages. Ideally there should be a syspref for setting collation language to sort by. This is a pretty trivial fix so might open a new issue for that. As a result of this "phrase" is almost not used at all, and could probably be removed with phrase queries on the search field instead in a later stage (which would decrease index size considerably). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #82544|0 |1 is obsolete| | --- Comment #23 from David Gustafsson <glasklas@gmail.com> --- Created attachment 83016 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83016&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #83016|0 |1 is obsolete| | --- Comment #25 from David Gustafsson <glasklas@gmail.com> --- Created attachment 83019 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83019&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Marjorie Barry-Vila <marjorie.barry-vila@collecto.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marjorie.barry-vila@collect | |o.ca -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #27 from Nick Clemens <nick@bywatersolutions.com> --- David, one request here would be to include the opac/staff columns to the mapping yaml file as well - changes that can be lost on a mapping reload are harder to track and keep in place -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #29 from David Gustafsson <glasklas@gmail.com> --- Just rebased this, but need to test some more and make sure tests are not broken. Should have that done by tomorrow. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #31 from David Gustafsson <glasklas@gmail.com> --- Not ready for signoff though. One test fails, I know the reason but it's not trivial to fix (association need to be used in test data and don't know if that is possible with TestBuilder). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #87358|0 |1 is obsolete| | --- Comment #32 from David Gustafsson <glasklas@gmail.com> --- Created attachment 87411 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=87411&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #34 from David Gustafsson <glasklas@gmail.com> --- Created attachment 87415 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=87415&action=edit Bug 20589: Exclude boolean fields from search fields -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 e.betemps <e.betemps@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |e.betemps@gmail.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #37 from David Gustafsson <glasklas@gmail.com> --- @Alex Arnaud Thanks for the comments. Regarding the query "(publisher:autrement OR jean)" I think it should be written as "publisher:(autrement OR jean)" to get the results you want. Also, surrounding parentheses can be left out of all the queries. They are only needed if you need to group sub-expressions. I don't think that Koha should try to reduce the field-list by parsing the query, this greatly increases complexity without any real benefit. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #87411|0 |1 is obsolete| | Attachment #87412|0 |1 is obsolete| | Attachment #87415|0 |1 is obsolete| | Attachment #87416|0 |1 is obsolete| | --- Comment #38 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 92454 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92454&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #40 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 92456 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92456&action=edit Bug 20589: Exclude boolean fields from search fields -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #42 from Nick Clemens <nick@bywatersolutions.com> --- Rebased and tested, this works excellently, I would love to move this forward, but I see a few issues in the patches: 1 - You removed QueryWeightedFields, but Zebra uses this, I obsoleted that patch 2 - There are changes to the authority searches here, things are improved, but a bit too strict, they don't relate to this bug, can you move those to a new report? 3 - Schema changes are mixed into first patch, they should be their own so RM can generate separately 4 - I did a minor rebase on top of 22258 and added a followup to allow searchig the entire record if using marc array Let me know if this makes sense or if I went too far anywhere -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #92455|0 |1 is obsolete| | --- Comment #44 from David Gustafsson <glasklas@gmail.com> --- Created attachment 93239 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93239&action=edit Bug 20589: fix QueryBuilder tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #92457|0 |1 is obsolete| | --- Comment #46 from David Gustafsson <glasklas@gmail.com> --- Created attachment 93241 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93241&action=edit Bug 20589: Add option for searching entire record if record stored as array -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93238|0 |1 is obsolete| | --- Comment #48 from David Gustafsson <glasklas@gmail.com> --- Created attachment 93247 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93247&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93240|0 |1 is obsolete| | --- Comment #50 from David Gustafsson <glasklas@gmail.com> --- Created attachment 93249 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93249&action=edit Bug 20589: Exclude boolean fields from search fields -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #52 from David Gustafsson <glasklas@gmail.com> --- Created attachment 93251 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93251&action=edit Bug 20589: DO NOT PUSH Schema changes -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93247|0 |1 is obsolete| | Attachment #93248|0 |1 is obsolete| | Attachment #93249|0 |1 is obsolete| | Attachment #93250|0 |1 is obsolete| | Attachment #93251|0 |1 is obsolete| | --- Comment #53 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 93348 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93348&action=edit Bug 20589: Add field boosting and use query_string fields parameter Generate a list of fields for the query_string query fields parameter, with possible boosts, instead of using "_all"-field. Also add "search" flag in search_marc_to_field table so that certain mappings can be excluded from searches. Add option to include/exclude fields in query_string "fields" parameter depending on searching in OPAC or staff client. Refactor code to remove all other dependencies on "_all"-field. How to test: 1) Reindex authorities and biblios. 2) Search biblios and try to verify that this works as expected. 3) Search authorities and try to verify that this works as expected. 4) Go to "Search engine configuration" 5) Change some "Boost", "Staff client", and "OPAC" settings and save. 6) Verify that those settings where saved accordingly. 7) Click the "Biblios" or "Authorities" tab and change one or more "Searchable" settings 8) Verfiy that those settings where saved accordingly. 9) Try to verify that these settings has taken effect by peforming some biblios and/or authorities searches. Sponsorded-by: Gothenburg Univesity Library Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #55 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 93350 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93350&action=edit Bug 20589: Exclude boolean fields from search fields Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #57 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 93352 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93352&action=edit Bug 20589: DO NOT PUSH Schema changes Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #59 from Nick Clemens <nick@bywatersolutions.com> --- Added minor followup Everything seems to work as expected. The authority searching changes seem to be an improvement: previously an 'exact' search matched any term, now it must match the entire field. Punctuation could be excluded in the future, but exact match should eb exact so I think good starts with searches and sorting are improved -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93349|0 |1 is obsolete| | --- Comment #61 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 93380 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93380&action=edit Bug 20589: fix QueryBuilder tests Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93351|0 |1 is obsolete| | --- Comment #63 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 93382 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93382&action=edit Bug 20589: Add option for searching entire record if record stored as array Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93353|0 |1 is obsolete| | --- Comment #65 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 93384 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93384&action=edit Bug 20589: Remove expanded_facet variable and fix tests Thi is a rebase from 14419 to remove an unused variable Also update tests to expect the new expected result Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |alex.arnaud@biblibre.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |19.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #68 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Thanks all, great enhancement -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #69 from David Gustafsson <glasklas@gmail.com> --- Wonderful, thanks everyone! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #71 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 93590 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93590&action=edit Bug 20589: (RM follow-up) Fix Koha::SearchField.t This patch removes the unit test for the weighted_fields method which this bug removed. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 --- Comment #73 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Martin Renvoize from comment #70)

Hmm, I think it's this one that causing a fair few breaks in tests on master right now..

I've narrowed down some of the issues.. for example I don't think we need the Koha::SearchFields->weighted_fields tests at all any more (you've removed the method entirely in your patch but missed this test).

But, there are some more I'm less confident about.. can you guys take a look please?

I attached a patch, I think the mappings test is probably not the best, but I think we should delete/replace it on its own bug, this gets things running again for now -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |25864 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 [Bug 25864] Case sensitivity breaks searching of some fields in ES5 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |28279 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28279 [Bug 28279] mappings.yaml no longer in sync -- You are receiving this mail because: You are watching all bug changes.