[Bug 21331] New: [Elasticsearch] Concatenated subfields are not indexed correctly
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Bug ID: 21331 Summary: [Elasticsearch] Concatenated subfields are not indexed correctly Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: alex.arnaud@biblibre.com I have the following unimarc data for a biblio: 200 $f Hanna Johansen 700 $a Johansen, $b Hanna 701 $a Maquet, $b Yves-Marie 701 $a Behnd, $b Kathi And the following mappings in search engine configuration: author => 200f author => 700ba author => 701ba Data that results of indexing in Elasticsearch is like: "author__facet" : [ "Hanna Johansen", "JohansenHanna" "MaquetYves-Marie", "BehndKathi" ], "author" : [ "Hanna Johansen", "JohansenHanna", "MaquetYves-Marie", "BehndKathi" ], This can cause troubles when searching and with facets. What we want is: "author__facet" : [ "Hanna Johansen", "Johansen Hanna" "Maquet Yves-Marie", "Behnd Kathi" ], "author" : [ "Hanna Johansen", "Johansen Hanna", "Maquet Yves-Marie", "Behnd Kathi" ], -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 --- Comment #1 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 78560 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78560&action=edit Bug 21331: Elasticsearch - fix indexing of concatenated subfields -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |alex.arnaud@biblibre.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #78560|0 |1 is obsolete| | --- Comment #2 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 80217 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=80217&action=edit Bug 21331: Elasticsearch - fix indexing of concatenated subfields Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |martin.renvoize@ptfs-europe | |.com --- Comment #3 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This one seemed to be a good bug to cut my teeth on as it were with my new found elastic search setup for testing. Looks good, does as expected. Signing off. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |BLOCKED --- Comment #4 from Alex Arnaud <alex.arnaud@biblibre.com> --- (In reply to Martin Renvoize from comment #3)
This one seemed to be a good bug to cut my teeth on as it were with my new found elastic search setup for testing.
Looks good, does as expected. Signing off.
Thanks Martin, But we'll have to throw this patch away since bug 19893 will completely change the code, and, fix this bug i hope. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 e.betemps <e.betemps@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |e.betemps@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=19893 Depends on| |19893 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 [Bug 19893] Alternative optimized indexing for Elasticsearch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |RESOLVED Resolution|--- |DUPLICATE --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- *** This bug has been marked as a duplicate of bug 19893 *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- The functionality here indeed appears to have been folded into bug 19893 so I'm marking as RESOLVED DUPLICATE. However, it does rely on updated available configuration options which should be documented somewhere. Fields may now be properly concatenated by specifying them within brackets `()`... Marking as 'Manual' to bring a maintainers attention. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |Manual -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |caroline.cyr-la-rose@inlibr | |o.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |glasklas@gmail.com --- Comment #7 from David Gustafsson <glasklas@gmail.com> --- (In reply to Martin Renvoize from comment #6)
The functionality here indeed appears to have been folded into bug 19893 so I'm marking as RESOLVED DUPLICATE.
However, it does rely on updated available configuration options which should be documented somewhere.
Fields may now be properly concatenated by specifying them within brackets `()`... Marking as 'Manual' to bring a maintainers attention.
Thanks! This has been brought up before in this issue and agree this needs to be fixed. I failed to find any existing section for the mappings.yaml format in the Koha Online manual where this could be put, but I just took a quick glance. Another thing that I also think has been brought up is that there perhaps should be added some defaults for fields that should be concatenated, like author and full title. This could perhaps be handled in a follow up patch though. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 --- Comment #8 from David Gustafsson <glasklas@gmail.com> --- Oops, I thought I responded in bug 19893, and were referring to the discussion in that issue. Will link to this comment in that issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- (In reply to David Gustafsson from comment #7)
Another thing that I also think has been brought up is that there perhaps should be added some defaults for fields that should be concatenated, like author and full title. This could perhaps be handled in a follow up patch though.
See the last patch on bug 19365, does this cover the defaults scenario? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 --- Comment #10 from David Gustafsson <glasklas@gmail.com> --- (In reply to Martin Renvoize from comment #9)
(In reply to David Gustafsson from comment #7)
Another thing that I also think has been brought up is that there perhaps should be added some defaults for fields that should be concatenated, like author and full title. This could perhaps be handled in a follow up patch though.
See the last patch on bug 19365, does this cover the defaults scenario?
It seems to use the new syntax, but for me it seems a little bit odd to categorically concatenate all the subfield for most fields. That defeats the purpose of the special syntax (to just concatenate the subfields where it makes sense to do so, when the result will be something like full title, or full author name like in the previous examples). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 Bug 21331 depends on bug 19893, which changed state. Bug 19893 Summary: Alternative optimized indexing for Elasticsearch https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org