https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 --- Comment #2 from Michal Denar <black23@gmail.com> --- Hi Nick, can we add some other czech and slavic letters into ICU too? <transliterate rule="{ č > c "/> <transliterate rule="{ Č > c "/> <transliterate rule="{ ď > d "/> <transliterate rule="{ Ď > d "/> <transliterate rule="{ ť > t "/> <transliterate rule="{ Ť > t "/> <transliterate rule="{ ř > r "/> <transliterate rule="{ Ř > r "/> <transliterate rule="{ š > s "/> <transliterate rule="{ Š > s "/> <transliterate rule="{ ú > u "/> <transliterate rule="{ Ú > u "/> <transliterate rule="{ ů > u "/> <transliterate rule="{ ě > e "/> <transliterate rule="{ Ě > e "/> <transliterate rule="{ é > e "/> <transliterate rule="{ É > e "/> <transliterate rule="{ á > a "/> <transliterate rule="{ Á > a "/> <transliterate rule="{ í > i "/> <transliterate rule="{ ý > y "/> I'm ready for test :-) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I wonder if adding the rules is the best way of achieving this. You can add a general rule for using the 'base letter'. We have been doing this I think. Found a hint about the rule here: https://wiki.koha-community.org/wiki/ICU_do_not_undiacritic -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- It means, the change here should not be necessary... Nick, can you please double check? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 --- Comment #8 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to David Cook from comment #7)

(In reply to Katrin Fischer from comment #6)

...

It means, the change here should not be necessary... Nick, can you please double check?

Although wouldn't that NFD change make Žižek into something like... ˇZiˇzek?

You have to look at the full example in the links I posted. 3 lines: <transform rule="NFD"/> <transform rule="[:Nonspacing Mark:] Remove"/> <transform rule="NFC"/> So yes, but then it uses that form to remove the diacritics: https://www.compart.com/en/unicode/category/Mn -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26390 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #8)

You have to look at the full example in the links I posted. 3 lines:

<transform rule="NFD"/> <transform rule="[:Nonspacing Mark:] Remove"/> <transform rule="NFC"/>

So yes, but then it uses that form to remove the diacritics: https://www.compart.com/en/unicode/category/Mn

Ahhh right. I should've been more thorough. I was thinking recently about how Zebra ICU has been seen as inferior to Elasticsearch ICU on the listserv. Looking at ftp://ftp.software.ibm.com/software/globalization/icu/3.6/icu-3_6-userguide.pdf, it looks like ICU actually originated in Java (ICU4J) and was later ported to C++ and C (ICU4C). According to https://wiki.koha-community.org/wiki/Record_Indexing_and_Retrieval_Options_f..., the Zebra use of libicu is inferior to Lucence ICU which uses ICU4J. There's no evidence given for the claim, but it seems believable (especially considering global prominence of Solr and Elasticsearch). Looking at https://lucene.apache.org/core/4_4_0/analyzers-icu/index.html, it seems that writing systems can use dictionary based algorithms (for systems like Thai script, Chinese, etc). That explains a lot. I know a bit of Chinese, and I've wondered how indexers could handle such a context-dependent language... -- You are receiving this mail because: You are watching all bug changes.