Apache configuration for UTF-8
Hi to all, I have update the wiki page http://wiki.koha-community.org/wiki/Encoding_and_Character_Sets_in_Koha in the section about apache. Now there are those lines: AddCharset UTF-8 .utf8 AddDefaultCharset utf-8 I check the use of utf-8 instead of UTF-8 in official Apache manual (2.0, 2.2, 2.4). Bye Zeno Tajoli -- Dr. Zeno Tajoli Dipartimento Gestione delle Informazioni e della Conoscenza z.tajoli@cineca.it fax +39 02 2135520 CINECA - Sede operativa di Segrate
Hello to all, We're using the MARC library for some migration, as usual, but we encountered some new issue with some arabic title: the key code 703 0x02BF 703 MODIFIER LETTER LEFT HALF RING ? is not part of the Table db, which cause the whole subfield to disappear and causing us headaches. I tried to ignore the unrecognized code using MARC::Charset->ignore_errors(1); as specified in the comments, but it crashes immediately after: my $code = $table->lookup_by_utf8($slice); returns undef if (! $code) { warn("no mapping found at position $i in $utf8"); reset_charsets() and return unless $ignore_errors; do not return } # if it's a combining character move it around if ($code->is_combining()) crashes on undef { My questions: - Any way to generate the Table with all the 700s characters? - Or any way to just avoid the error? Thanks, Philippe
At 02:56 PM 6/4/2014 -0400, Philippe Blouin wrote:
[snip] My questions: - Any way to generate the Table with all the 700s characters?
Have you had a look at <http://search.cpan.org/~gmcharlt/MARC-Charset-1.35/lib/MARC/Charset/Table.pm>? Best - P.
Hi, On Wed, Jun 4, 2014 at 11:56 AM, Philippe Blouin < philippe.blouin@inlibro.com> wrote:
We're using the MARC library for some migration, as usual, but we encountered some new issue with some arabic title: the key code 703 0x02BF 703 MODIFIER LETTER LEFT HALF RING ʿ is not part of the Table db, which cause the whole subfield to disappear and causing us headaches.
What is the source character encoding of the records? If the records are already in UTF-8, then it is not necessary to transcode them to MARC8, then back to UTF8 for loading into Koha. Adding the following line to whatever code you're using to pre-process the records might help: MARC::Charset->assume_unicode(1); As an alternative, you could adjust change the records to use 0x02bb rather than 0x02bf. I'm assuming that the strings in question are transliterated Arabic following the ALA-LC Arabic romanization. If so, back in 1999, the mapping of the "ayn" character was changed from 0x02bf to 0x02bb. [1] [1] http://www.loc.gov/marc/marbi/2005/2005-05.html Regards, Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
Thanks Galen, Paul, I'm sorry I just noticed your answers in my inbox. Galen, I tried your solution (the record is in fact a utf8 encoded xml document), but it made no difference. But it did lead me to some nice places in the code (and documentation) where I now specify the BinaryEncoding => 'utf8' in the 'use Marc::File::XML'. It will probably break another migration, but this fixes my problem for now. Thanks again! Philippe On 06/04/2014 06:49 PM, Galen Charlton wrote:
Hi,
On Wed, Jun 4, 2014 at 11:56 AM, Philippe Blouin <philippe.blouin@inlibro.com <mailto:philippe.blouin@inlibro.com>> wrote:
We're using the MARC library for some migration, as usual, but we encountered some new issue with some arabic title: the key code 703 0x02BF 703 MODIFIER LETTER LEFT HALF RING ʿ
is not part of the Table db, which cause the whole subfield to disappear and causing us headaches.
What is the source character encoding of the records? If the records are already in UTF-8, then it is not necessary to transcode them to MARC8, then back to UTF8 for loading into Koha. Adding the following line to whatever code you're using to pre-process the records might help:
MARC::Charset->assume_unicode(1);
As an alternative, you could adjust change the records to use 0x02bb rather than 0x02bf. I'm assuming that the strings in question are transliterated Arabic following the ALA-LC Arabic romanization. If so, back in 1999, the mapping of the "ayn" character was changed from 0x02bf to 0x02bb. [1]
[1] http://www.loc.gov/marc/marbi/2005/2005-05.html
Regards,
Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com <mailto:gmc@esilibrary.com> direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
-- Philippe Blouin, Responsable du développement informatique Tél. : (888) 604-2627 philippe.blouin@inLibro.com <mailto:philippe.blouin@inLibro.com> inLibro | pour esprit libre | www.inLibro.com <http://www.inLibro.com>
participants (4)
-
Galen Charlton -
Paul A -
Philippe Blouin -
Zeno Tajoli