[Bug 33594] New: Sorting results by Title A-Z might use wrong title field
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Bug ID: 33594 Summary: Sorting results by Title A-Z might use wrong title field Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: katrin.fischer@bsz-bw.de CC: januszop@gmail.com, nick@bywatersolutions.com Depends on: 33206 When sorting by title we expect the sorting to use 245, but this is not always the case. title in Elasticsearch is a broadly defined index, also containing other title fields. Example: * Search for 'e' with Elasticsearch in the sample data * Result will look something like this: 14. Agesilaos and the crisis of Sparta Paul Cartledge 15. Xenophōntos Agēsilaos. Agesilaus of Xenophon / the text revised, with critical and explanatory notes, introduction, analyses and indices, by H. Hailstone. 16. Alcestis / Euripides ; edited with introduction and commentary by A.M. Dale. So what happened here? Look at the MARC of 15. - 240 was used: 240 1 0 _a Agesilaus. _l Greek. _f 1879 245 1 0 _a Xenophōntos Agēsilaos. Agesilaus of Xenophon / _c the text revised, with critical and explanatory notes, introduction, analyses and indices, by H. Hailstone. Maybe we need a specific index definition for sorting? Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33206 [Bug 33206] Bad title__sort made of multisubfield 245 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #1 from Janusz Kaczmarek <januszop@gmail.com> --- I have encountered this some time ago. For me worked customizing the mappings.yaml as follows: - facet: '' marc_field: 240 marc_type: marc21 - sort: ~ + sort: '' suggestible: '' I.e. changing ~ into '' for all MARC fields except 245. For some to-me-mysterious reason, ~ (= undef) is explicitly interpreted as "yes, include it into the sort search field" by Koha/SearchEngine/Elasticsearch.pm: # Sort is a bit special as it can be true, false, undef. # We care about "true" or "undef", # "undef" means to do the default thing, which is make it sortable. So, since this is by design and mappings.yaml is configurable, I didn't treat it as a bug. Just use a customized copy of mappings. Please, check if changing ~ into '' works for you. Please, consider also if it is worth to be a bug. If so, the solution is there. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #2 from Janusz Kaczmarek <januszop@gmail.com> --- (In reply to Janusz Kaczmarek from comment #1)
I.e. changing ~ into '' for all MARC fields except 245.
I meant, for title search field, of course. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Janusz Kaczmarek from comment #2)
(In reply to Janusz Kaczmarek from comment #1)
I.e. changing ~ into '' for all MARC fields except 245.
I meant, for title search field, of course.
Hi Janusz, thx for the insight, this is really helpful. I also found bug 32479. Would you agree that all ~ shoudl be replaced by "" or 0? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I feel it's a bug and we should fix it, but still a little confused about the different values we could use here for sort and what effect they will have. Can someone explain a little more? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #5 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Katrin Fischer from comment #4)
I feel it's a bug and we should fix it, but still a little confused about the different values we could use here for sort and what effect they will have. Can someone explain a little more?
27316 moved the options for setting the field in staff interface from "Undef/Yes/No" to simply "Yes/No" = 0/1 We did not update the mappings file, that should be done here The code continues to support ~/Undef so that we don't break existing setups - we should update the comment Janusz noted to explain that 'UNdef' should be set as '1' -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #6 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 150649 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150649&action=edit Bug 33594: Update mappings and comment -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- This changes a few other fields, but it really is just a dump from the DB, those changes are essentially how the fields are going to be treated anyways, it was generated using: perl misc/search_tools/export_elasticsearch_mappings.pl > admin/searchengine/elasticsearch/mappings.yaml -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Janusz Kaczmarek <januszop@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #8 from Janusz Kaczmarek <januszop@gmail.com> --- Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" makes the mappings.yaml to be more explicit and therefor less mysterious. But the patch does not change the way Koha with ES sorts titles, so it does not fix the problem. I think, everybody would expect that when choosing sort order by title (both in staff and OPAC interface) the sorting will by done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC. So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- 'sort: 0'. I would appreciate your feedback. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Janusz Kaczmarek from comment #8)
Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" makes the mappings.yaml to be more explicit and therefor less mysterious.
But the patch does not change the way Koha with ES sorts titles, so it does not fix the problem. I think, everybody would expect that when choosing sort order by title (both in staff and OPAC interface) the sorting will by done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC.
So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- 'sort: 0'.
I would appreciate your feedback.
I'd agree with that expectation. The other titles don't even all show, so it can get quite confusing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #10 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Janusz Kaczmarek from comment #8)
Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" makes the mappings.yaml to be more explicit and therefor less mysterious.
But the patch does not change the way Koha with ES sorts titles, so it does not fix the problem. I think, everybody would expect that when choosing sort order by title (both in staff and OPAC interface) the sorting will by done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC.
So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- 'sort: 0'.
I would appreciate your feedback.
Agreed - it looks like the Elastic default on a multi-value sort field is min (ascending) or max (descending) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #11 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 150749 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150749&action=edit Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #150649|0 |1 is obsolete| | --- Comment #12 from David Nind <david@davidnind.com> --- Created attachment 150792 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150792&action=edit Bug 33594: Update mappings and comment Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #150749|0 |1 is obsolete| | --- Comment #13 from David Nind <david@davidnind.com> --- Created attachment 150793 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150793&action=edit Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This changes the behavour release notes| |of title search and sorting | |(Title (A-Z)) in the OPAC | |and staff interface when | |using Elasticsearch or Open | |Search. The title search | |now only uses 245 (for MARC | |21) and 200 (for UNIMARC). | |Previously other title | |fields may have affected | |the search order, for | |example 240$a in MARC21. | | | |To make this change for | |existing installations, | |reset the search engine | |mappings (Administration > | |Catalog > Search engine | |configuration | |(Elasticsearch) > Reset | |mappings (scroll down to | |the bottom of the page)) | |and rebuild the search | |index (koha-elasticsearch | |--rebuild -d -b -a | |<instancename>). If you | |have customized the search | |engine configuration, | |remember to record or back | |these up BEFORE resetting | |the mappings. CC| |david@davidnind.com --- Comment #14 from David Nind <david@davidnind.com> --- Testing notes (using KTD): 1. I used the example in the bug description (#c0) to test. 2. After step 1, I needed to reset the mappings (Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (scroll down to the bottom of the page)). 3. Draft release notes. I've made an attempt at this, but this area is not something I really understand. Please update to fix what I've got wrong! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=26472 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #150793|0 |1 is obsolete| | --- Comment #15 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 150813 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150813&action=edit Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 --- Comment #16 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.05. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Needs documenting CC| |matt.blenkinsop@ptfs-europe | |.com --- Comment #17 from Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> --- Enhancement - not backporting to 22.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal --- Comment #18 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- It would be nice to have this reconsidered, but it will only affect new installations. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33594 Bug 33594 depends on bug 33206, which changed state. Bug 33206 Summary: Bad title__sort made of multisubfield 245 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33206 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org