[Koha-bugs] [Bug 9114] exported MARC frameworks encoding should be UTF-8
bugzilla-daemon at bugs.koha-community.org
bugzilla-daemon at bugs.koha-community.org
Thu Jan 16 10:42:44 CET 2014
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9114
M. de Rooy <m.de.rooy at rijksmuseum.nl> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #22740|0 |1
is obsolete| |
--- Comment #6 from M. de Rooy <m.de.rooy at rijksmuseum.nl> ---
Created attachment 24450
-->
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=24450&action=edit
Bug 9114: import/export routines should use UTF-8
Currently both the import_export_framework.pl script and the associated libs
output data with Perl's default encoding, ISO-8859. This patch properly sets
the binmode to UTF-8 for the CGI script, and makes the
C4::ImportExportFramework
routines apply the ":encoding(UTF-8)" layer to file descriptors.
To test:
Export step test
- Use some non-ASCII character(s) in some field description in a chosen
framework.
- Export the framework at Administration > MARC frameworks
- Run this to check the file is ISO-8859 encoded:
$ file export_XXX.csv
export_XXX.csv: ISO-8859 text, with very long lines
(Note: try SQL and other output formats too. But not ODS)
- Apply the patch
- Export the framework again (change the name), and test encoding:
$ file export_XXX_2.csv
export_XXX_2.csv: UTF-8 Unicode text
Import step test
I assume you have two files, export_XXX.csv (ISO-8859 encoded) and
export_XXX_2.csv (XXX will depend on
your framework's code)
- Reset your testing branch to master
- Import export_XXX.csv
- The string with non-ASCII chars is truncated at the first non-ASCII char's
position
(Note: this is the current behaviour).
- Import export_XXX_2.csv
- The non-ASCII chars are broken, the logs show errors on non-UNICODE chars.
(Note: even thou UTF-8 is the expected encoding it is treated as ISO-8859).
- Apply the patch
- Import the good (UTF-8 as expected) file and check everything worked as
expected.
Regards
To+
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Tested export and re-import with CSV containing Latin chars and containing
Chinese chars.
--
You are receiving this mail because:
You are watching all bug changes.
More information about the Koha-bugs
mailing list