[Bug 36853] New: Enhance Elasticsearch Notes/Comments MARC21 index mappings
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Bug ID: 36853 Summary: Enhance Elasticsearch Notes/Comments MARC21 index mappings Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: aleisha@catalyst.net.nz We would like to add the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: * 501a * 503a * 504a * 508a * 511a * 521a * 538a * 547a * 591a * 592a * 593a * 594a * 595a * 596a * 597a * 598a * 599ab -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Change sponsored?|--- |Sponsored -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 --- Comment #1 from wainuiwitikapark@catalyst.net.nz --- Created attachment 167204 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167204&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a, 591a, 592a, 593a, 594a, 595a, 596a, 597a, 598a, 599ab Sponsored-by: Educational Services Australia SCIS -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |wainuiwitikapark@catalyst.n |ity.org |et.nz -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #167204|0 |1 is obsolete| | --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 167300 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167300&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a, 591a, 592a, 593a, 594a, 595a, 596a, 597a, 598a, 599ab Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #3 from David Nind <david@davidnind.com> --- Testing notes using KTD: 1. Start up KTD with Elasticsearch: ktd --es8 up 2. Edit a record and add unique values as follows (I edited Programming Perl, 262): . Add a unique value to 500$a (this is currently included in the index) . Add unique values to a sample of the notes fields not currently mapped . Note: Need to edit BKS framework to make 503$a, 508$a, 511$a, 547$a, 591$a, 592$a, $593a, 594$a, 595$a, 596$a, 597$a, 598$a, 599$a and $b subfields visible in the editor for KTD) Mappings - none of these subfields are currently mapped: * 501a * 503a * 504a * 508a * 511a * 521a * 538a * 547a * 591a * 592a * 593a * 594a * 595a * 596a * 597a * 598a * 599ab 3. Do a search for the terms in the subfields using the 'Notes/Comments' index in the staff interface advanced search. ==> Result: Should get a result when using values in the subfields that are mapped, but no result where the subfields are not mapped. 4. Apply this patch and restart all services (restart_all). 5. Reset the mappings: Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (at the bottom of the page) 6. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 7. Alternative to steps 5 and 6: reset_all then repeat step 2 8. Repeat step 3 ==> Result: Should now get a result when using values in the subfields that are now mapped (that weren't previously). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |domm@plix.at CC| |domm@plix.at -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #167300|0 |1 is obsolete| | --- Comment #4 from Thomas Klausner <domm@plix.at> --- Created attachment 169067 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=169067&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a, 591a, 592a, 593a, 594a, 595a, 596a, 597a, 598a, 599ab Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #5 from David Cook <dcook@prosentient.com.au> --- At a glance, this shouldn't have the same issue as bug 36850, bug 36851, and bug 36852, because these fields aren't faceted, so I think it's OK at a glance. -- As an aside, I have a library that want to add 530 and 533 too. Part of me wonders if there's a better way we could be managing these index mappings... In Zebra, there are no explicit note mappings for 530 and 533, but they are included in the "Any" index, which doesn't exist in our Elasticsearch implementation. I don't know... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- These are custom fields that are not present in the MARC frameworks: https://www.loc.gov/marc/bibliographic/bd59x.html + - facet: '' + marc_field: '591a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '592a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '593a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '594a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '595a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '596a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '597a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '598a' + marc_type: marc21 + sort: 1 + suggestible: '' + - facet: '' + marc_field: '599ab' + marc_type: marc21 + sort: 1 + suggestible: '' I feel we should not add them to the standard mappings, as we cannot tell what information might be stored in these fields. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #6)
I feel we should not add them to the standard mappings, as we cannot tell what information might be stored in these fields.
Mmm good call. I really need to look more at how custom mappings are done for Elasticsearch on a per-instance basis... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #169067|0 |1 is obsolete| | --- Comment #8 from wainuiwitikapark@catalyst.net.nz --- Created attachment 170333 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170333&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a, 591a, 592a, 593a, 594a, 595a, 596a, 597a, 598a, 599ab Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #170333|0 |1 is obsolete| | --- Comment #9 from wainuiwitikapark@catalyst.net.nz --- Created attachment 170334 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170334&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #170334|0 |1 is obsolete| | --- Comment #10 from David Nind <david@davidnind.com> --- Created attachment 183638 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183638&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement adds these release notes| |note subfields to the | |Notes/Comments index for | |new installations using | |Elasticsearch: 501$a, | |503$a, 504$a, 508$a, 511$a, | |521$a, 538$a, and 547$a. | |This makes them notes | |findable in the staff | |interface and OPAC, and in | |the advanced search for | |when the Notes/Comments | |option is selected. | | | |Note: Existing | |installations will need to | |either manually add these, | |or reset the mappings. --- Comment #11 from David Nind <david@davidnind.com> --- Testing notes (using KTD): 1. Start up KTD with Elasticsearch: ktd --es7 up 2. Go to Koha administration > Catalog > Search engine configuration (Elasticsearch). 3. In the bibliographic records tab, search the page and note that none of 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a fields are mapped to the note search field. 4. Apply the patch and reset everything (reset_all) (alternative: reset the mappings, reindex everything (koha-elasticsearch --rebuild -d -b -a kohadev), then restart everything). 5. Repeat step 3 and note that the 501a, 503a, 504a, 508a, 511a, 521a, 538a, and 547a fields are now mapped to the note search field. 6. Bonus testing: 6.1 Update the bibliographic framework for BKS and make the 501a, 503a, 508a, 511a, and 547a fields editable in the staff interface (504a, 521a, 538a are already editable). 6.2 Add some unique text to the fields 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a - I used TEST501, TEST503 and so on. 6.3 Staff interface search: Search > Advanced search > select "Notes/Comments" and search for the unique terms you added to a record - you should only get one result each time. 6.4 OPAC search: repeat the advanced search with more options selected in the OPAC - you should get the same results as for the staff interface. 6.5 Add one of the unique terms to another record, for example TEST501 to 501$a, and you should now get two results. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Irene <i.tortorella@arts.ac.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |i.tortorella@arts.ac.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|domm@plix.at |katrin.fischer@bsz-bw.de --- Comment #12 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Thomas, stealing this one from you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |String patch Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183638|0 |1 is obsolete| | --- Comment #13 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 193050 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=193050&action=edit Bug 36853: Enhance Elasticsearch Notes/Comments MARC21 index mappings Added the following tags and subfields to the existing note index for Elasticsearch MARC21, as used by the "Notes/Comments" advanced search dropdown option: 501a, 503a, 504a, 508a, 511a, 521a, 538a, 547a Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 --- Comment #14 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I am quite a fan of improving our default mappings, thanks Wainui! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |26.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 --- Comment #15 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 26.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36853 Jacob O'Mara <jacob.omara@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Needs documenting --- Comment #16 from Jacob O'Mara <jacob.omara@openfifth.co.uk> --- This will not be backported to 25.11.x due to it being deemed an enhancement or a dependancy not being met -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org