https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 Anthony Zhu <zdypop@outlook.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |zdypop@outlook.com --- Comment #1 from Anthony Zhu <zdypop@outlook.com> --- Hi Nick, Regarding the Elasticsearch implementation, my research findings (based on the PDF documentation I shared previously) suggest that installing the plugins alone might not be enough. We likely need to define a custom analyzer in the Elasticsearch index settings to explicitly chain the tokenizer and the conversion filter. Here is the specific configuration logic derived from my research (Source: "DeepSeek" analysis in my documentation) that might be the missing link: 1. Define Custom Analyzer (Elasticsearch API Approach) It seems we need to update the index settings to include a custom analyzer that uses the ICU or SmartCN tokenizer followed by a Traditional-to-Simplified transform filter. JSON /* Concept Configuration for Elasticsearch Index Settings */ PUT /koha_biblios { "settings": { "analysis": { "filter": { "traditional_to_simplified": { "type": "icu_transform", "id": "Traditional-Simplified" /* Or use stconvert if installed: "type": "stconvert", "convert_type": "t2s" */ } }, "analyzer": { "zh_cn_search": { "tokenizer": "icu_tokenizer", /* or smartcn_tokenizer */ "filter": [ "traditional_to_simplified", "lowercase" ] } } } } } 2. Update Koha Search Mappings (search_fields.yaml) After defining the analyzer in ES, we need to tell Koha to use this zh_cn_search analyzer for the relevant fields (like Title, Author). YAML /* In Koha's search mappings */ title: type: text analyzer: zh_cn_search search_analyzer: zh_cn_search 3. Regarding the Zebra Config If Elasticsearch proves too difficult for now, the Zebra ICU configuration I verified previously is: koha-conf.xml: Enable <icu>1</icu> and <language>zh</language>. ICU Rule: Use ::zh-Hans-zh-Hant; transliteration. I hope these specific JSON snippets help clarify how to trigger the plugins! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 --- Comment #3 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- One of our partners suggested testing the transform using the analysis-icu plugin In testing this works well, however, I have to search the extra field created: title.zh_cn There are some option here: 1 - Add the ability to search additional fields specific, that would let us include the zh_cn version of any field where appropriate 2 - Add the ability to add custom types - I was thinking that we could use authorized values to specify the type code and display name - then include those in the type option dropdowns on the ES config page. These would require the library adding them in the backend index config, but would allow specifying the new types for whichever fields desired to title_zh_cn etc 3 - Add the ability to have an analyzed search - and just automatically run anything the user enters through the same conversion All are reasonable extensions of ES and things we probably want at some point, we just need to decide on which paths to take -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- I hadn't gotten too far with this before running out of time I think. It would be interesting to look at though... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 --- Comment #5 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 201780 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201780&action=edit Bug 41469: Add ability to define custom types This patch removes the ENUM limits from search_fields.type to allow for custom values. It adds a new SEARCH_FIELD_TYPES authorizsed value category for controlling the types that are available. I also edit av-build-dropbox.inc to allow for its use on Search fields configuration page and to handle disabling and adding a custom data field. Incidentally I add some chomping to keep the HTML smaller To test: 1 - Apply patch - update database 2 - Confirm you can load/edit search engine configuration as before 3 - Add a new SEARCH_FIELDS_TYPE value 'zh_cn' description 'Transliterate Chinese' 4 - Apply POC patch from earlier 5 - Reload elasticsearch mappings 6 - Add a new field 'title-transliterated' 7 - Reindex your catalog 8 - Confirm searching works as expected -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #6)

I'll add it to the list to look at :3

Sorry but what's the consolidated test plan for this one? Not sure how to test this one at present. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41469 --- Comment #9 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- (In reply to Katrin Fischer from comment #8)

(In reply to David Cook from comment #7)

...

(In reply to David Cook from comment #6)

...

I'll add it to the list to look at :3

Sorry but what's the consolidated test plan for this one? Not sure how to test this one at present.

I also had a look and got stuck on:

...

3 - Add a new SEARCH_FIELDS_TYPE value 'zh_cn' description 'Transliterate Chinese'

Can you please give us some more hints/explanation?

It's a new authorised values category - you just need to add a new authorised value that will match the one you set in your mappings file -- You are receiving this mail because: You are watching all bug changes.