[Bug 14899] New: Mapping configuration page for Elastic search
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Bug ID: 14899 Summary: Mapping configuration page for Elastic search Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: enhancement Priority: P5 - low Component: Searching Assignee: jonathan.druart@bugs.koha-community.org Reporter: jonathan.druart@bugs.koha-community.org QA Contact: testopia@bugs.koha-community.org Depends on: 12478 To help the users and testers to use and understand how Elastic search works, Koha needs an interface to configure the mappings between the Elastic search indexes and the marc fields. Referenced Bugs: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12478 [Bug 12478] Elasticsearch support for Koha -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Mirko Tietgen <mirko@abunchofthings.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mirko@abunchofthings.net -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Josef Moravec <josef.moravec@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |josef.moravec@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 42882 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42882&action=edit Bug 14899: Add Koha::SearchField[s] and Koha::SearchMarcMap[s] classes -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #2 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 42883 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42883&action=edit Bug 14899: Add tableDND JS lib -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #3 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 42884 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42884&action=edit Bug 14899: DB changes This patch applies several changes to the DB structure introduced by the previous ES patch set: 1/ Add a search_field.label column => It will be easier to understand what the search_field does. Indeed, some are not user friendly: acqdate, an, pl, ff8-23, ln, etc. TODO later: Update the labels with correct values (at the moment label=name) 2/ Update the foreign key search_marc_to_field.search_marc_map_id with on delete cascase. This will permit to automatically remove the entries in search_marc_to_field when they are deleted in search_marc_map. 3/ Remove the index_name_2 unique constraint on the search_marc_map table. I don't understand how this could be useful, it was defined like (index_name, marc_type, marc_field), so it means a mapping cannot be defined twice with different values for facet, suggestible and sort. This limitation does not seem to make sense. Robin, please correct me if I am wrong :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 42885 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42885&action=edit Bug 14899: Add the mapping configuration page in the admin module This new page (admin/searchengine/elasticsearch/mappings.pl) will permit to manage the ES mappings. For the biblios and authorities indexes, the different mappings can be managed from this single page. The interface let you add, remove and update mappings. It's also possible to reorder the mappings, as the order can be important in the indexation process. Note that the table can be displayed in a different order that the one it was before saving, but the mappings are grouped by search field and the order inside the search field is preserved. Limitations: - If something went wrong during the insertion/deletion/modification, the users will loose all these changes. - Using the interface the user can defined several labels for a same field. The script will retrieve the first change and apply it, next ones will be avoided. TODO: - Add a specific permission (?) - Add some data checks client side (JS) - Use checkboxes for facet and suggestible (lazy today...) - Understand the difference between the 3 values that sortable can have and improve the value for the options in the select box. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #5 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 42886 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42886&action=edit Bug 14899: Add a link to the new page in the admin -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- To apply on top of the ES work, see the catalyst/elastic_search branch. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Peter Zhao <peterzhaonj@163.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |peterzhaonj@163.com --- Comment #7 from Peter Zhao <peterzhaonj@163.com> --- http://127.0.0.1:8080/cgi-bin/koha/admin/searchengine/elasticsearch/mappings... An error has occurred! Error 500 In Koha this typically means that the Koha team is working on new features Wait while system maintenance is being done or email the Koha administrator. Use top menu bar to navigate to another part of Koha. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #8 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Peter Zhao from comment #7)
http://127.0.0.1:8080/cgi-bin/koha/admin/searchengine/elasticsearch/mappings. pl
An error has occurred! Error 500
In Koha this typically means that the Koha team is working on new features Wait while system maintenance is being done or email the Koha administrator. Use top menu bar to navigate to another part of Koha.
What is the error in the log file? How did you test? To test: $ git remote -v # should at least return catalyst git://git.catalyst.net.nz/koha.git (fetch) catalyst git://git.catalyst.net.nz/koha.git (push) $ git fetch catalyst $ git checkout -b bug_14899 catalyst/elastic_search $ git bz apply 14899 $ perl installer/data/mysql/updatedatabase.pl -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #42882|0 |1 is obsolete| | Attachment #42883|0 |1 is obsolete| | Attachment #42884|0 |1 is obsolete| | Attachment #42885|0 |1 is obsolete| | Attachment #42886|0 |1 is obsolete| | --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43054 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43054&action=edit Bug 14899: Add Koha::SearchField[s] and Koha::SearchMarcMap[s] classes -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43055 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43055&action=edit Bug 14899: Add tableDND JS lib -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43056 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43056&action=edit Bug 14899: DB changes This patch applies several changes to the DB structure introduced by the previous ES patch set: 1/ Add a search_field.label column => It will be easier to understand what the search_field does. Indeed, some are not user friendly: acqdate, an, pl, ff8-23, ln, etc. TODO later: Update the labels with correct values (at the moment label=name) 2/ Update the foreign key search_marc_to_field.search_marc_map_id with on delete cascase. This will permit to automatically remove the entries in search_marc_to_field when they are deleted in search_marc_map. 3/ Remove the index_name_2 unique constraint on the search_marc_map table. I don't understand how this could be useful, it was defined like (index_name, marc_type, marc_field), so it means a mapping cannot be defined twice with different values for facet, suggestible and sort. This limitation does not seem to make sense. Robin, please correct me if I am wrong :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #12 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43057 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43057&action=edit Bug 14899: Add the mapping configuration page in the admin module This new page (admin/searchengine/elasticsearch/mappings.pl) will permit to manage the ES mappings. For the biblios and authorities indexes, the different mappings can be managed from this single page. The interface let you add, remove and update mappings. It's also possible to reorder the mappings, as the order can be important in the indexation process. Note that the table can be displayed in a different order that the one it was before saving, but the mappings are grouped by search field and the order inside the search field is preserved. Limitations: - If something went wrong during the insertion/deletion/modification, the users will loose all these changes. - Using the interface the user can defined several labels for a same field. The script will retrieve the first change and apply it, next ones will be avoided. TODO: - Add a specific permission (?) - Add some data checks client side (JS) - Use checkboxes for facet and suggestible (lazy today...) - Understand the difference between the 3 values that sortable can have and improve the value for the options in the select box. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #13 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43058 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43058&action=edit Bug 14899: Add a link to the new page in the admin -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #14 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Patches updated, they now apply against the up-to-date catalyst/elastic_search branch. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Robin Sheat <robin@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |robin@catalyst.net.nz --- Comment #15 from Robin Sheat <robin@catalyst.net.nz> --- (In reply to Jonathan Druart from comment #11)
3/ Remove the index_name_2 unique constraint on the search_marc_map table. I don't understand how this could be useful, it was defined like (index_name, marc_type, marc_field), so it means a mapping cannot be defined twice with different values for facet, suggestible and sort. This limitation does not seem to make sense. Robin, please correct me if I am wrong :)
You are not wrong. I just wrote a big thing about what I was intending, realised I was wrong, so deleted it :) I think I was confusing index_name and field name, or something. I'll merge this in as part of the ES branch like you suggest, as it'll solve some issues. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #16 from Robin Sheat <robin@catalyst.net.nz> --- Though, it'd be best to just put DB changes in the .sql file for now and let that load them in from there, rather than creating a whole set of transactions to manage. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #17 from Robin Sheat <robin@catalyst.net.nz> --- Created attachment 43099 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43099&action=edit Bug 14899: schema changes to make the database better -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #43054|0 |1 is obsolete| | Attachment #43055|0 |1 is obsolete| | Attachment #43056|0 |1 is obsolete| | Attachment #43057|0 |1 is obsolete| | Attachment #43058|0 |1 is obsolete| | Attachment #43099|0 |1 is obsolete| | --- Comment #18 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43353 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43353&action=edit Bug 14899: Add tableDND JS lib -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #19 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43354 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43354&action=edit Bug 14899: Add the mapping configuration page in the admin module This new page (admin/searchengine/elasticsearch/mappings.pl) will permit to manage the ES mappings. For the biblios and authorities indexes, the different mappings can be managed from this single page. The interface let you add, remove and update mappings and search fields. It's also possible to reorder the mappings, as the order can be important in the indexation process. Note that the table can be displayed in a different order that the one it was before saving, but the mappings are grouped by search field and the order inside the search field is preserved. Limitations: - If something went wrong during the insertion/deletion/modification, the users will loose all these changes. TODO: - Add a specific permission (?) - Add some data checks client side (JS) - Use checkboxes for facet and suggestible (lazy today...) - Understand the difference between the 3 values that sortable can have and improve the value for the options in the select box. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14899 --- Comment #20 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 43355 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=43355&action=edit Bug 14899: Add a link to the new page in the admin -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org