[Bug 25325] New: ElasticSearch mapping export lacks staff_client/opac fields
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Bug ID: 25325 Summary: ElasticSearch mapping export lacks staff_client/opac fields Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: frederic@tamil.fr export_elasticsearch_mappings.pl doesn't export staff_client and opac fields. Its means that when reimporting (Reset mapping in Search engine configuration) indexing rules, the searchability (staff/opac) of fields will be lost. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Depends on| |23204 Severity|enhancement |minor Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23204 [Bug 23204] Script for exporting Elasticsearch mappings to YAML -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #1 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 104004 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104004&action=edit Bug 25325 ElasticSearch mapping export lacks staff_client/opac fields Add to export_elasticsearch_mappings.pl two missing fields: staff_client and opac. Test plan: 1. Run export_elasticsearch_mappings.pl script. Write the result in mappings.yaml (the one defined in koha-conf.xml). 2. In Admin > Search engine configuration, click on Reset Mappings 3. Confirm that in Search fields tab, the Searchable column hasn't changed. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org Severity|minor |normal Assignee|koha-bugs@lists.koha-commun |frederic@tamil.fr |ity.org | Keywords| |rel_20_05_candidate -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104004|0 |1 is obsolete| | --- Comment #2 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 104063 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104063&action=edit Bug 25325 ElasticSearch mapping export lacks staff_client/opac fields Add to export_elasticsearch_mappings.pl two missing fields: staff_client and opac. Test plan: 1. Run export_elasticsearch_mappings.pl script. Write the result in mappings.yaml (the one defined in koha-conf.xml). 2. In Admin > Search engine configuration, click on Reset Mappings 3. Confirm that in Search fields tab, the Searchable column hasn't changed. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |victor@tuxayo.net --- Comment #3 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Hi, I tried to test and improvised a lot because Koha ES is new for me. First, I switch SearchEngine syspref to ES but I don't know what else to do. (I have ES running as part of koha-testing-docker)
3. Confirm that in Search fields tab, the Searchable column hasn't changed.
Without the patch, the Searchable column hasn't changed. So couldn't reproduce the issue. I suspect that it's because my test data (from koha-testing-docker) has everything to "yes". And that "yes" might be the default. So in the intranet I change few a columns to "no" and save.
An error occurred when updating Elasticsearch index mappings: Unable to update mappings for index "koha_kohadev_biblios". Reason was: "no such index". Index needs to be recreated and reindexed.
An error occurred when updating Elasticsearch index mappings: Unable to update mappings for index "koha_kohadev_authorities". Reason was: "no such index". Index needs to be recreated and reindexed.
Index 'koha_kohadev_biblios' needs to be recreated.
Index 'koha_kohadev_authorities' needs to be recreated.
Ok that doesn't look good. I must be missing some basic ES config. At least I can confirm that with the patch there is a diff on the output of export_elasticsearch_mappings.pl It's precisely on the changes I made which is encouraging. Let's try anyway. So my test plan completed was: - SearchEngine syspref to ES (don't know if that's useful) - set few fields to not searchable - scary errors... - misc/search_tools/export_elasticsearch_mappings.pl - save to mappings.yaml - for me it's: /kohadevbox/koha/admin/searchengine/elasticsearch/mappings.yaml - restart_all (don't know if that's useful) - reset mappings - changes to the fields are lost - apply patch - restart_all - set few fields to not searchable - scary errors... - misc/search_tools/export_elasticsearch_mappings.pl - save to mappings.yaml - restart_all (don't know if that's useful) - reset mappings - changes to the fields are still there! :D So in spite of the scary errors and my dubious setup. It really look like it's working :D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- *** Bug 25337 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #5 from Frédéric Demians <frederic@tamil.fr> --- Victor, your correct 'yes' is default value for 'staff_client' and 'opac' fields. It means you have to set some ES fields to 'false' to see the problem. You can also see the reverse function of exporting ES config which is: Koha::SearchEngine::ElasticSearch.pm::reset_elasticsearch_mappings http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=Koha/SearchEngine/... Thks for testing/signing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Salut Frédéric, could you add tests for that? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #7 from Frédéric Demians <frederic@tamil.fr> --- (In reply to Jonathan Druart from comment #6)
Salut Frédéric, could you add tests for that?
Yes, sorry. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #8 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 104376 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104376&action=edit Bug 25325: Test ES lack of staff_client/opac fields Run this test to see that export_elasticsearch_mappings.pl doesn't export staff_client/opac fields. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #9 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 104377 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104377&action=edit Bug 25325: ElasticSearch mapping export lacks staff_client/opac fields Run test: prove t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t Or: Run export_elasticsearch_mappings.pl script. See that opac/staff_client fields are exorted. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104063|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #10 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Should the first patch be manually tested again? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #11 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Wait, it's all automated tests now. I got also confused that the implementation changed from the one on comment 1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104376|0 |1 is obsolete| | --- Comment #12 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 104451 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104451&action=edit Bug 25325: Test ES lack of staff_client/opac fields Run this test to see that export_elasticsearch_mappings.pl doesn't export staff_client/opac fields. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104377|0 |1 is obsolete| | --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 104452 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104452&action=edit Bug 25325: ElasticSearch mapping export lacks staff_client/opac fields Run test: prove t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t Or: Run export_elasticsearch_mappings.pl script. See that opac/staff_client fields are exorted. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104451|0 |1 is obsolete| | Attachment #104452|0 |1 is obsolete| | --- Comment #14 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 104482 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104482&action=edit Bug 25325: Test ES lack of staff_client/opac fields Run this test to see that export_elasticsearch_mappings.pl doesn't export staff_client/opac fields. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #15 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 104483 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104483&action=edit Bug 25325: ElasticSearch mapping export lacks staff_client/opac fields Run test: prove t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t Or: Run export_elasticsearch_mappings.pl script. See that opac/staff_client fields are exorted. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com --- Comment #17 from Joy Nelson <joy@bywatersolutions.com> --- not backported to 19.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_20_05_candidate | CC| |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Bug 25325 depends on bug 23204, which changed state. Bug 23204 Summary: Add script for exporting Elasticsearch mappings to YAML https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23204 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #18 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- On bug 23204 we made sure that the mappings.yaml file included in the codebase was the same as the output of misc/search_tools/export_elasticsearch_mappings.pl However since this change (pushed only two months later) they differ. Is that a big deal? Should we resync them? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 --- Comment #19 from Fridolin Somers <fridolin.somers@biblibre.com> --- I say we should update mapping.yaml, it is a reference when starting to manage Elasticsearch with files. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25325 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |28279 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28279 [Bug 28279] mappings.yaml no longer in sync -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org