[Bug 25501] New: Wide character in print warning installing translations
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bug ID: 25501 Summary: Wide character in print warning installing translations Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: I18N/L10N Assignee: koha-bugs@lists.koha-community.org Reporter: bgkriegel@gmail.com QA Contact: testopia@bugs.koha-community.org CC: frederic@tamil.fr Install any translation, (cd misc/translator/; ./translate install fr-FR ) see example output Wide character in print at /../misc/translator/tmpl_process3.pl line 144. Wide character in printf at /.../misc/translator/tmpl_process3.pl line 141. Wide character in print at /.../misc/translator/tmpl_process3.pl line 134 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #1 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 104876 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104876&action=edit Bug 25501: use utf8 for open in tmpl_process3.pl Opening files using "":encoding(UTF-8)" removes the warning "wide character in print" installing translations To test: 1) Verify the error installing translations for any language 2) Apply the patch 3) Repeat 1), check warning is gone -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |bgkriegel@gmail.com |ity.org | Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25305 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org --- Comment #2 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- I am used to explicitly specify the encoding when opening the file, but I am fine with this approach as well. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_20_05_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #104876|0 |1 is obsolete| | --- Comment #3 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 105043 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105043&action=edit Bug 25501: use utf8 for open in tmpl_process3.pl Opening files using "":encoding(UTF-8)" removes the warning "wide character in print" installing translations To test: 1) Verify the error installing translations for any language 2) Apply the patch 3) Repeat 1), check warning is gone Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |victor@tuxayo.net -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Bernardo, there is an encoding issue in the translation process now. Translated interface in French: Périodiques -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bgkriegel@gmail.com |jonathan.druart@bugs.koha-c | |ommunity.org CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |25305 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25305 [Bug 25305] Double UTF-8 encoding on translation files -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105043|0 |1 is obsolete| | --- Comment #5 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105060 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105060&action=edit Bug 25501: Encode strings in utf8 for output - translation process This is a follow-up of bug 25305, all strings need to be utf8 encoded when printed to the file. Test plan: Test the whole translation process and translate a language you know. Test the translated interface for: XSLT (detail and result pages), syspref, and "normal" views You should not get encoding errors (or warning in logs), nowhere. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Wide character in print |Encoding issues in the |warning installing |translation process |translations | Severity|normal |critical -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105067 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105067&action=edit Bug 25501: Pass the encoding to Locale::PO -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #7 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Still not ready: misc/translator/po/fr-FR-installer-MARC21.po: GNU gettext message catalogue, ASCII text misc/translator/po/fr-FR-installer.po: GNU gettext message catalogue, ASCII text misc/translator/po/fr-FR-marc-MARC21.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-marc-NORMARC.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-marc-UNIMARC.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-messages-js.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-messages.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-opac-bootstrap.po: GNU gettext message catalogue, UTF-8 Unicode text misc/translator/po/fr-FR-pref.po: GNU gettext message catalogue, UTF-8 Unicode text, with very long lines misc/translator/po/fr-FR-staff-prog.po: GNU gettext message catalogue, UTF-8 Unicode text, with very long lines -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #8 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Actually better than before, for message-js: misc/translator/po/fr-FR-messages-js.po: GNU gettext message catalogue, ASCII text the installer PO files are not translated yet, and so does not contain utf8 chars. Ready for testing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105060|0 |1 is obsolete| | Attachment #105067|0 |1 is obsolete| | --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105073 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105073&action=edit Bug 25501: Encode strings in utf8 for output - translation process This is a follow-up of bug 25305, all strings need to be utf8 encoded when printed to the file. Test plan: Test the whole translation process and translate a language you know. Test the translated interface for: XSLT (detail and result pages), syspref, and "normal" views You should not get encoding errors (or warning in logs), nowhere. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105074 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105074&action=edit Bug 25501: Pass the encoding to Locale::PO -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #11 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- With my patch there were 624 files with double encoding after install :( With yours none One qa error, you need to remove line 380 in misc/translator/tmpl_process3.pl - local(*INPUT, *OUTPUT); * But now something new: * (cd misc/translator/; perl translate update es-ES; perl translate install es-ES) Wide character in print at /home/bgkriegel/koha/kohaclone/misc/translator/po2json line 171. this never ends... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105073|0 |1 is obsolete| | Attachment #105074|0 |1 is obsolete| | --- Comment #12 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105080 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105080&action=edit Bug 25501: Encode strings in utf8 for output - translation process This is a follow-up of bug 25305, all strings need to be utf8 encoded when printed to the file. Test plan: Test the whole translation process and translate a language you know. Test the translated interface for: XSLT (detail and result pages), syspref, and "normal" views You should not get encoding errors (or warning in logs), nowhere. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #13 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105081 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105081&action=edit Bug 25501: Pass the encoding to Locale::PO -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #14 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105084 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105084&action=edit Bug 25501: Remove local vars -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #15 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105085 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105085&action=edit Bug 25501: Remove 'Wide character in print' for po2json -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com --- Comment #16 from Julian Maurice <julian.maurice@biblibre.com> --- Here is a simpler counter patch : https://gitlab.com/jajm/Koha/-/commit/a8764d0b0add70acbf5d5b5ad6046c1571e2ec... It suppresses all warnings for me and I haven't found any encoding issues in the interface yet -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #17 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Jonathan, Without patch, no double encoding but warnings With patches, no warnings but 635 files with double encoding With Julian's approach, no warnings nor double encoding Checked encoding issues using http://paste.koha-community.org/20552 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105080|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105081|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105084|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105085|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #18 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 105095 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105095&action=edit Bug 25501: Supress warnings on installing translation To test: 1) Verify the error installing translations for any language 2) Apply the patch 3) Repeat 1), check warning is gone Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Attributed to Julian No warnings, no double encoding, no errors. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105095|0 |1 is obsolete| | --- Comment #19 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 105099 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105099&action=edit Bug 25501: Supress warnings on installing translation To test: 1) Verify the error installing translations for any language 2) Apply the patch 3) Repeat 1), check warning is gone Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Attributed to Julian No warnings, no double encoding, no errors. Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Tested full install with yml and js strings, everything works great! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|critical |normal Assignee|jonathan.druart@bugs.koha-c |julian.maurice@biblibre.com |ommunity.org | QA Contact|testopia@bugs.koha-communit |jonathan.druart@bugs.koha-c |y.org |ommunity.org --- Comment #20 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Lowering severity as it did not actually break things. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #21 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #22 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Jonathan Druart from comment #20)
Lowering severity as it did not actually break things.
I'm not sure if you mean that encoding issues are minor issues, or that there wasn't any encoding issue in the first place, but the patch did fix a double encoding issue for me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_20_05_candidate | CC| |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #23 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Julian Maurice from comment #22)
(In reply to Jonathan Druart from comment #20)
Lowering severity as it did not actually break things.
I'm not sure if you mean that encoding issues are minor issues, or that there wasn't any encoding issue in the first place, but the patch did fix a double encoding issue for me.
On master? I got encoding issues with Bernardo's attempt, but did not notice something wrong on master, did you? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #24 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I had noticed problems with XSLT not displaying and the wide character error for de-DE on master a few days ago. Can retest now with the patch. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #25 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Jonathan Druart from comment #23)
On master? I got encoding issues with Bernardo's attempt, but did not notice something wrong on master, did you?
Yes. To reproduce: 1. git reset --hard 8994ded2c47d22fd46ea46f6e7855da52826b351 # the commit before 25501 2. cd misc/translator && ./translate install fr-FR 3. Open koha-tmpl/intranet-tmpl/prog/fr-FR/xslt/MARC21Languages.xsl, you should see the string "Hébreu" 4. git reset --hard c97921bb68849a2c7dd5f2f5ad8f5b97789df5c1 5. ./translate install fr-FR 6. Reopen koha-tmpl/intranet-tmpl/prog/fr-FR/xslt/MARC21Languages.xsl and see that the string is now "Hébreu" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 --- Comment #26 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Katrin Fischer from comment #24)
I had noticed problems with XSLT not displaying and the wide character error for de-DE on master a few days ago. Can retest now with the patch.
I just retested and things see to be ok now - thx for all the work here! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |critical -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25501 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com --- Comment #27 from Joy Nelson <joy@bywatersolutions.com> --- not backported to 19.11.x due to missing dependencies -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org