In my haste, I already sent out a series of patches that take care of the easy cases of UTF-8 conversion. On Fri, 2010-03-26 at 07:45 -0700, Galen Charlton wrote:
Hi,
2010/3/24 Lars Wirzenius <lars@catalyst.net.nz>:
C4/tests/testrecords/marc21_marc8_combining_chars.dat
This should be left alone - it is a test file for converting MARC records from the MARC-8 character set to UTF-8.
Right. My patches include a find-nonutf8 script, which ignores those files now.
etc/zebradb/etc/urx.chr etc/zebradb/lang_defs/en/sort-string-utf.chr
This can be converted to UTF-8, but not mechanically. They're Zebra configurations that are expected to be in ISO-8859-1 by default. To signal that they're using UTF-8, the following directive would have to be added to them and the results of the indexing tested:
encoding utf-8
Since I am still getting up to speed with Koha development, I am not sure I can test this reliably yet. My patches also ignore these files, which I think will be fine for now.