Re: [Koha-devel] Charsets: Moving to utf-8
Hi, I dont respond often on this list, but I thought the following information may be useful...
OK ... during the bugsquash session the issue of charsets came up. (in response to 885:; Non-ISO8859-1 charset support broken in CSS theme in 2.2). Our goal for 2.4 would be to move everything--db storage, and templates -- to utf-8. The question is, how?
[snip]
So we're looking for suggestions on how to make the switch. What do we need to consider:
HTML::Template limitations (should we be looking at Template::Toolkit) Database Issues Translation Issues Other stuff
At my work we provide a browser-based application which is translated into a number of languages. Some of the interesting points: - we only support UTF8 as the charset, as any other charset in simply insufficient (Google does this too). - all translations are stored within a database table, then have the template lookup any static text (text generated from within Perl, can be translated before being sent to the templates. - since templates lookup their own translations, you dont need a template-per-language. - since translations are stored within a database, you can update the translations at run-time, without needing to restart / reload your updated translations - web access to the database translations allow multiple people to localise the translations, at the same time. - right-to-left text is also easily supported To do some of these, we modified HTML::Template to suport custom TMPL_xxx tags, so that we could implement a tag for the static text lookup. ie: our templates look like: ... <TMPL_CATGETS "A phrase containing a template variable [_1], that gets inserted here",some_var> ... where some_var is a Perl generated TMPL_VAR value. We combine H::T with the Locale::MakePhrase module so that the output can handle singular / plural / etc based on the string/numeric value of some_var. For some extra information, go here: http://members.optusnet.com.au/mathew Hope this helps, Mathew PS. Some languages, particularily Asian languages, contain many pen-strokes within a glyph. Often these languages need the text point-size to be slightly larger than European languages... this is just a tidbit that I have found through experience, particularily if a single template is to be used for every language.
- we only support UTF8 as the charset, as any other charset in simply insufficient (Google does this too). UTF8 is OK.
- all translations are stored within a database table, then have the template lookup any static text (text generated from within Perl, can be translated before being sent to the templates. I think this is not a good idea. Read why:
- since templates lookup their own translations, you dont need a template-per-language. I'd prefer a template per language. Often the syntax is different in different languages. With one template per language you can build phrases, which mix static text with database generated text correctly for every language.
- since translations are stored within a database, you can update the translations at run-time, without needing to restart / reload your updated translations We don't have to restart/reload right now.
- web access to the database translations allow multiple people to localise the translations, at the same time. What do you mean with "localise the translations"? I think CVS does this job too.
- right-to-left text is also easily supported This is primary a point of coding. The correct formatting of such text can be controlled better by one template per language.
To do some of these, we modified HTML::Template to suport custom TMPL_xxx tags, so that we could implement a tag for the static text lookup. ie: our templates look like: You modified the original module? What about upgrades?
We combine H::T with the Locale::MakePhrase module so that the output can handle singular / plural / etc based on the string/numeric value of some_var. This could be interesting for some words or phrases, but it can't provide the flexibility of a template.
PS. Some languages, particularily Asian languages, contain many pen-strokes within a glyph. Often these languages need the text point-size to be slightly larger than European languages... this is just a tidbit that I have found through experience, particularily if a single template is to be used for every language. got it.
Dorian
participants (2)
-
Dorian Meid -
Mathew