[Bug 41668] New: Elasticsearch reset not setting facets correctly
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 Bug ID: 41668 Summary: Elasticsearch reset not setting facets correctly Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au QA Contact: testopia@bugs.koha-community.org When resetting the Elasticsearch settings using mappings.yaml, it appears that the "Facetable" setting isn't being updated correctly. For instance, the default mappings.yaml says "publisher" is facetable, but after resetting the Elasticsearch from file, it says it's not facetable. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=35138 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Ok in main... I've reset and it is setting in the database but the UI still says "No" for Facetable for publisher, local-classification, copydate. I re-indexed with the default conf, and I see "publisher__facet" and "copydate__facet" keys in the ES document despite Facetable saying "no" for publisher and copydate, and those facets not being defined in "Facet Order". -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Ahhh if I save the mappings via the UI it sets "facet" to 0 for publisher, local-classification, and copydate. That would explain why they weren't showing up in the database when I looked previously... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Ahhh it's because "is_facetable" is based off Koha::SearchEngine::Elasticsearch::get_facet_fields which uses Koha::SearchFields->search( { facet_order => { '!=' => undef } }, { order_by => ['facet_order'] } )->as_list; Which has a certain logic to it but it's a problem. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18235 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |35138 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35138 [Bug 35138] Enable configuration of facets with Elasticsearch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- Hmm it's not enough to have that "Yes" show up in the UI for publisher, local-classification, and copydate... there's something with the save overall that is blitzing it. Jeez... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- That said... maybe it makes sense to only set "facet" if the mapping has a facet_order as well. Because if it doesn't, it's never going to show up anyway, and then we're creating indexed values that will never be used. But if that's the case maybe we should update the indexing to only index facet names that appear in Koha::SearchEngine::Elasticsearch->get_facet_fields() and to remove "facet: 1" from the default mappings.yaml for indexes that don't have a facet_order out of the box. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elasticsearch reset not |Inconsistency between |setting facets correctly |mappings.yaml and | |Elasticsearch web UI causes | |data loss -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion --- Comment #6 from David Cook <dcook@prosentient.com.au> --- What should we do here? Option 1: - Change mappings.yaml to set facet: 0 for publisher, local-classification, copydate - This will mean that mappings.yaml and the UI are more in sync. That is, people won't look at mappings.yaml and wonder why the UI says "facetable: no" when the YAML says "facet: 1". That said, they could do that to their custom YAML and be confused still. Option 2a: - Change mappings.pl to show facet: 1 for fields like publisher, local-classification, copydate even though they don't have facet_order set. - This will mean these fields are indexed into "*__facet" fields even though those facets won't be shown. That seems suboptimal Option 2b: - Do option 2a PLUS update the indexing so that "*__facet" fields are only created if there is a facet_order for that facet, so we don't create indexes that aren't used Option 3: - Something else I haven't thought of? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- Note a manifestation of this bug... if you have a default/vanilla Elasticsearch mappings, you can add the "publisher" facet immediately and have it work out of the box, because "publisher__facet" is already indexed out of the box. I had 2 similar Koha instances and one had a working publisher facet and one didn't and it's because one had an original ES config and one must've had a slightly modified one where the publisher facet had been blitzed by clicking "Save" for an unrelated configuration change. It was crazy-making for sure. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 Emmanuel Bétemps <e.betemps@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |e.betemps@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com --- Comment #8 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- I recall domm mentioned last hackfest that saving the mappings to a file with `export_elasticsearch_mappings.pl` and then applying the file using the `--reset` flag for `rebuild_elasticsearch.pl` yielded inconsistent results in the configured mappings. I bet both are related. I think there was a bug for that too. Not facet-specific but related. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |domm@plix.at -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- (In reply to Tomás Cohen Arazi (tcohen) from comment #8)
I recall domm mentioned last hackfest that saving the mappings to a file with `export_elasticsearch_mappings.pl` and then applying the file using the `--reset` flag for `rebuild_elasticsearch.pl` yielded inconsistent results in the configured mappings. I bet both are related. I think there was a bug for that too. Not facet-specific but related.
I raised a bug for that too yesterday hehe: bug 41670 I don't know that it's related though. This facetable stuff specifically relates to changes made in bug 35138. I think bug 41670 is partially a code bug and partially the default file just being inaccurate. But with this one... I don't think the answer is quite straightforward. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |lukasz.koszyk@kit.edu, | |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- In 25.11 I'm able to change the "publisher" label to "Publisher" and I can add the facet under the "Facet order" section... and it appears correctly in the Staff Interface. But in the OPAC instead of showing "Publisher", it shows "publisher_id". So annoying... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #10)
In 25.11 I'm able to change the "publisher" label to "Publisher" and I can add the facet under the "Facet order" section... and it appears correctly in the Staff Interface.
But in the OPAC instead of showing "Publisher", it shows "publisher_id". So annoying...
Although it's possible I added that bug by accident... let's see... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41668 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #11)
(In reply to David Cook from comment #10)
In 25.11 I'm able to change the "publisher" label to "Publisher" and I can add the facet under the "Facet order" section... and it appears correctly in the Staff Interface.
But in the OPAC instead of showing "Publisher", it shows "publisher_id". So annoying...
Although it's possible I added that bug by accident... let's see...
Nope it's bug 25314 it seems... but bug 42555 fixes it. But only in 26.11 and 26.05... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org