[Bug 29893] New: ElasticSearch Config UI deletes mappings
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Bug ID: 29893 Summary: ElasticSearch Config UI deletes mappings Change sponsored?: --- Product: Koha Version: 21.05 Hardware: All OS: All Status: NEW Severity: critical Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: domm@plix.at When using the Elasticsearch UI /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl to edit search mappings one can quite easily delete a lot of mappings by mistake. To reproduce: * go to /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl * click on the "Authorities" Tab * You should see ~ "67 entries" * enter "Heading-Main" into the search field, you now see ~8 entries ("filtered from 67 entries" * Click "save" Button * Go again to the "Autorities" Tab * You now see ~ "8 entries" * All the other entries have been deleted! What obviously happens is that all entries that where filtered due to the search are not submitted to the backend on save. The backend now interprets the missing entries as entries that shall be deleted. And happily deleted them! This totally breaks the search index (after the next index run, or when adding single items), as (eg in the auth-index) "authtype" is no longer indexed. I currently don't have a 21.11 or later version at hand, so maybe this has already been fixed? To fix this bug we would either have to un-hide the filtered entries before submitting the form; change the backend to not deleted missing entries (which means that the delete-action would have to be implemented a bit different); something else? I guess that the first fix is easier, and would also make sense with regard to how that form works. Greetings, domm -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 129537 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=129537&action=edit Bug 29893: Prevent ES mappings UI to remove hidden rows If you are editing ES mappings and you use the DataTable filtering option, the table won't contain all the mappings you have in the DB. If the form is submitted they will be removed and data will be lost! Test plan: Edit ES mappings, filter and submit the form. Confirm that the table are redrawn before the form submission which prevent data loss -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|21.05 |unspecified Assignee|koha-bugs@lists.koha-commun |jonathan.druart+koha@gmail. |ity.org |com CC| |jonathan.druart+koha@gmail. | |com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #2 from Thomas Klausner <domm@plix.at> --- I can confirm that the patch provided by Jonathan fixed this bug. Meta-Question: Can / should I set the status of this bug to "signed off" or RESOLVED? Or are status changes only "allowed" for proper team members? Greetings, domm -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #3 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Ideally you should submit the patch with your "signed-off-by" line, then switch the status to "Signed Off". https://wiki.koha-community.org/wiki/Sign_off_on_patches -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #4 from Thomas Klausner <domm@plix.at> --- Created attachment 129571 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=129571&action=edit sign-off Sign-off for the patch, after applying it the bug is fixed. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #129537|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |tomascohen@gmail.com CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|tomascohen@gmail.com |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #129571|0 |1 is obsolete| | --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 130237 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130237&action=edit Bug 29893: Prevent ES mappings UI to remove hidden rows If you are editing ES mappings and you use the DataTable filtering option, the table won't contain all the mappings you have in the DB. If the form is submitted they will be removed and data will be lost! Test plan: Edit ES mappings, filter and submit the form. Confirm that the table are redrawn before the form submission which prevent data loss -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Nick told me the previous patch didn't work for him. This method is using what is described on the DataTable's website https://datatables.net/examples/api/form.html and so might be a better approach. See also https://www.gyrocode.com/articles/jquery-datatables-how-to-submit-all-pages-... This seems quite ugly to me, especially the redirect, but I don't have anything else to suggest for now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |david@davidnind.com --- Comment #7 from David Nind <david@davidnind.com> --- Issues from testing: 1. The patch works for me - it stops mappings being deleted as reported. I had to clear the browser cache for this to work. 2. However, with mappings either filtered or unfiltered, if you: 2.1: Change one (for example: Authorities > Heading-Main, 155a - change the Yess to Nos), save, the change to the mapping is not saved. 2.2: Delete one, save, the deleted mapping is not deleted. 2.3: Add a new one (for example: Heading-Main, Yes for all options, 155b), the new mapping is not saved. 3. I also noted that when adding a new mapping, the options available for the sortable column are Undef, 0 and 1. For all the other columns the options are Yes and No. I'm not familiar enough with Elastic to know whether this is correct or not. Tested using Firefox Developer Edition. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130237|0 |1 is obsolete| | --- Comment #8 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 130348 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130348&action=edit Bug 29893: Prevent ES mappings UI to remove hidden rows If you are editing ES mappings and you use the DataTable filtering option, the table won't contain all the mappings you have in the DB. If the form is submitted they will be removed and data will be lost! Test plan: Edit ES mappings, filter and submit the form. Confirm that the table are redrawn before the form submission which prevent data loss -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #9 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 130349 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130349&action=edit Bug 29893: Make sure the form will be submitted after the draw calls -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #10 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Sorry David, the previous patch was terribly wrong. I kept the first approach and fix what I think was a bug. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #11 from David Nind <david@davidnind.com> --- Thanks Jonathan! However, I am now getting these messages in yellow boxes above the "Search engine configuration (Elasticsearch)" heading: "You attempted to delete all mappings for a required index, you must leave at least one mapping" "An error occurred when updating mappings: ." This occurs any time the save button is clicked. On the positive side, the mappings are no longer deleted. On the downside, you can't edit, add or delete any mappings. Consolidated test plan: 1. In the staff interface go to Administration > Catalog > Search engine configuration (Elasticsearch). 2. Select the authorities tab. 3. You should see ~67 entries. 4. Enter "Heading-Main" into the search field, you now see ~8 entries (filtered from 67 entries). 5. Cick the save button. 6. Go back to the authorities tab. 7. You will now see ~8 entries - all the other entries were deleted! 8. Apply the patch, flush_memcached, restart_all, and you may also need to clear your browser cache (or open a private window). 9. Repeat steps 1-7 - this time the expected number of entries should be visible after clicking save (~67), that is, all the entries are not deleted when you click save! 10. With mappings filtered and unfiltered, check that everything works as expected if you: . Change an entry (for example: Authorities > Heading-Main, 155a - change the Yess to Nos). . Delete an entry. . Add a new entry. (for example: Heading-Main, Yes for all options, 155b). 11. Sign off :D! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #12 from Nick Clemens <nick@bywatersolutions.com> --- When I follow the test plan all authority mappings except for heading-main are still deleted -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #13 from Michal Denar <black23@gmail.com> --- Same for me. If I changed the values, the original values remained. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #14 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 130764 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130764&action=edit Bug 29893: Redraw the tables from other tabs without paging -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #15 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I didn't recreate David or Michal problems. Nick's problem should be fixed with the last patch. Sorry guys, but what we are doing in this template is weird (we drop and redraw the table every time a tab is clicked, that's not efficient). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130348|0 |1 is obsolete| | Attachment #130349|0 |1 is obsolete| | Attachment #130764|0 |1 is obsolete| | --- Comment #16 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 130768 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130768&action=edit Bug 29893: Prevent ES mappings UI to remove hidden rows If you are editing ES mappings and you use the DataTable filtering option, the table won't contain all the mappings you have in the DB. If the form is submitted they will be removed and data will be lost! Test plan: Edit ES mappings, filter and submit the form. Confirm that the table are redrawn before the form submission which prevent data loss Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #17 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 130769 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130769&action=edit Bug 29893: Make sure the form will be submitted after the draw calls Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #18 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 130770 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130770&action=edit Bug 29893: Redraw the tables from other tabs without paging Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|nick@bywatersolutions.com | CC| |martin.renvoize@ptfs-europe | |.com --- Comment #19 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- What is explaining a critical bug stuck in NQA for more than a month? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30000 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30000 [Bug 30000] Replace the use of jQueryUI tabs on the search engine configuration page -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130768|0 |1 is obsolete| | --- Comment #20 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 132185 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132185&action=edit Bug 29893: Prevent ES mappings UI to remove hidden rows If you are editing ES mappings and you use the DataTable filtering option, the table won't contain all the mappings you have in the DB. If the form is submitted they will be removed and data will be lost! Test plan: Edit ES mappings, filter and submit the form. Confirm that the table are redrawn before the form submission which prevent data loss Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130769|0 |1 is obsolete| | --- Comment #21 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 132186 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132186&action=edit Bug 29893: Make sure the form will be submitted after the draw calls Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130770|0 |1 is obsolete| | --- Comment #22 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 132187 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132187&action=edit Bug 29893: Redraw the tables from other tabs without paging Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #23 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Apologies, not been on QA for a little while whilst I played catch up. Patch works as expected, man the logic flow is weird here. No regressions found. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 --- Comment #24 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to master for 22.05, thanks to everybody involved 🦄 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.05.00 |22.05.00,21.11.05 released in| | CC| |kyle@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #25 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to 21.11.x for 21.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29893 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |30584 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30584 [Bug 30584] Cannot add field mappings to Elasticsearch configuration -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org