[Bug 12279] New: Diacritics in system preference editor broken
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Bug ID: 12279 Summary: Diacritics in system preference editor broken Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: I18N/L10N Assignee: koha-bugs@lists.koha-community.org Reporter: katrin.fischer@bsz-bw.de QA Contact: testopia@bugs.koha-community.org CC: frederic@tamil.fr Created attachment 28316 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28316&action=edit Screenshot of German OPAC tab in sys pref editor I see some strange behaviour with diacritics in the system preference editor on current master. What I did: - cd misc/translator - perl translate update de-DE - perl translate install de-DE The strange thing is that only some pages/umlauts are broken, while others on other tabs appear ok. I see the same behaviour with French - attaching screenshots. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 28317 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28317&action=edit Screenshot of French OPAC tab in sys pref editor -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #2 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 28318 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28318&action=edit Screenshot of German Logs tab in sys pref editor And while OPAC is broken, the Logs tab displays diacritics correctly. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Nicolas Legrand <nicolas.legrand@bulac.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nicolas.legrand@bulac.fr --- Comment #3 from Nicolas Legrand <nicolas.legrand@bulac.fr> --- On a 3.15.00.052 it looks I have the same problem. I made cd misc/translator ./translate update de-DE ./translate install de-DE ./translate update fr-FR ./translate install fr-FR ./translate update tr-TR ./translate install tr-TR Some are broken, like opaccolorstylesheet : fr-FR: Inclure la feuille de style CSS supplémentaire pour outrepasser les paramètres spécifiques depuis la feuille de style par défaut (laisser vide pour désactiver). Inscrire juste le nom du fichier, un chemin local complet ou une URL complète commençant par http:// (si le fichier est sur un serveur distant). Merci de noter que si seul un nom de fichier est entré, le fichier devra se trouver dans le sous-dossier css pour chaque thème et langage actif dans le dossier des modèles de Koha. Un chemin local est attendu pour démarrer depuis votre fichier HTTP source. de-DE: Verwende zusätzlich das CSS-Stylesheet um Anweisungen im vorhandenen Stylesheet zu überschreiben (frei lassen, wenn nicht verwendet). Geben Sie nur einen Dateinamen oder einen vollständigen lokalen Pfad oder eine vollständige URL beginnend mit http:// an (falls die Datei auf einem anderen Server liegt). Wenn nur ein Dateiname angegeben wird, sollte die Datei im Verzeichnis css für das aktive Theme und die jeweilige Sprache gespeichert sein. Ein vollständiger lokaler Pfad beginnt mit dem Wurzelverzeichnis des Webservers. Some are OK, like language : fr-FR: Activer à l'interface professionnelle les langues suivantes : Deutsch(de-DE) English(en) Türkçe(tr-TR) Français(fr-FR) de-DE: Aktiviere folgende Sprachen in der Dienstoberfläche: Deutsch(de-DE) Français(fr-FR) Türkçe(tr-TR) English(en) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m --- Comment #4 from Jonathan Druart <jonathan.druart@biblibre.com> --- I reproduce with perl v5.18. Bernardo and Fridolin don't reproduce with perl v5.14. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bgkriegel@gmail.com --- Comment #5 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Please report version of YAML::Syck Have no problems on Ubuntu 12.04 (ver 1.19), or Debian 7 (ver 1.20), but with latest version (ver 1.27, from cpan) I can reproduce -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Yaml:Syck: 1.27 - It looks like we have a candidate. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #7 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 28375 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28375&action=edit Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked template object, we only use gettemplate to build the correct file name (path included) and load it using the (at functional) YAML::Syck::LoadFile method. Pros: Doesn't break anything for YAML::Syck pre-1.27 (the ones shipped with Ubuntu 14.04) Cons: something is wrong with C4::Templates encoding/decoding, or at least our default setup using C4::Templates->new doesn't work properly for all file types. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28375|0 |1 is obsolete| | --- Comment #8 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 28377 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28377&action=edit Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked template object, we only use gettemplate to build the correct file name (path included) and load it using the (at functional) YAML::Syck::LoadFile method. Pros: Doesn't break anything for YAML::Syck pre-1.27 (the ones shipped with Ubuntu 14.04) Cons: something is wrong with C4::Templates encoding/decoding, or at least our default setup using C4::Templates->new doesn't work properly for all file types. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works ok, no errors Test: 1) with YAML::Syck 1.19, no problems before, translated preferences encode diacritics, ej. '\xC3\xA8' for grave accented e Page shows correctly 2) with YAML:Syck 1.27, translated preferences have unencoded diacritics, without patch display is broken, with patch it shows correctly -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28377|0 |1 is obsolete| | --- Comment #9 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 28378 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28378&action=edit [SIGNED-OFF] Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked template object, we only use gettemplate to build the correct file name (path included) and load it using the (at functional) YAML::Syck::LoadFile method. Pros: Doesn't break anything for YAML::Syck pre-1.27 (the ones shipped with Ubuntu 14.04) Cons: something is wrong with C4::Templates encoding/decoding, or at least our default setup using C4::Templates->new doesn't work properly for all file types. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works ok, no errors Test: 1) with YAML::Syck 1.19, no problems before, translated preferences encode diacritics, ej. '\xC3\xA8' for grave accented e Page shows correctly 2) with YAML:Syck 1.27, translated preferences have unencoded diacritics, without patch display is broken, with patch it shows correctly -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28378|0 |1 is obsolete| | --- Comment #10 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 28380 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28380&action=edit Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked template object, we only use gettemplate to build the correct file name (path included) and load it using the (at functional) YAML::Syck::LoadFile method. Pros: Doesn't break anything for YAML::Syck pre-1.27 (the ones shipped with Ubuntu 14.04) Cons: something is wrong with C4::Templates encoding/decoding, or at least our default setup using C4::Templates->new doesn't work properly for all file types. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works ok, no errors Test: 1) with YAML::Syck 1.19, no problems before, translated preferences encode diacritics, ej. '\xC3\xA8' for grave accented e Page shows correctly 2) with YAML:Syck 1.27, translated preferences have unencoded diacritics, without patch display is broken, with patch it shows correctly Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #11 from Jonathan Druart <jonathan.druart@biblibre.com> --- YAML::Syck is v1.27 here and I confirm this patch fixes the issue. But I'm still wondering why the pref files contain a mix of '\xC3\xA8' and 'é'. Marked as Passed QA. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA CC| |gmcharlt@gmail.com --- Comment #12 from Galen Charlton <gmcharlt@gmail.com> --- Two comments: This causes a regression, as a few of the tabs actually use TT: circulation.pref: - '[% local_currency %] in fines.' circulation.pref: - '[% local_currency %] in fines.' opac.pref: - '[% local_currency %] in fines (leave blank to disable).' patrons.pref: - '[% local_currency %].' Failing QA on that basis. Also, I'm not sure YAML::Syck 1.27 is specifically to blame -- on a Debian Wheezy system, when I installed YAML::Syck 1.27 from CPAN, I did not see the bug. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28380|0 |1 is obsolete| | --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 28424 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28424&action=edit Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked C4::Templates->output method, it adds a new one that takes care of decoding the .pref file data for storing in memory. This might need to be re-done in the context of 11944, but at least provides a clean workaround for 3.16. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28424|0 |1 is obsolete| | --- Comment #14 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 28438 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28438&action=edit Bug 12279: Diacritics in system preference editor broken This patch changes how .pref files are read. Instead of going through all our global ultra-tweaked C4::Templates->output method, it adds a new one that takes care of decoding the .pref file data for storing in memory. This might need to be re-done in the context of 11944, but at least provides a clean workaround for 3.16. To test: - On clean master, install a translation like: cd misc/translator ; perl translate install fr-FR - Go to sysprefs, opac => FAIL: descriptions are encoding-broken - Apply the patch, reload => SUCCESS: Everything shows ok Regards To+ Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |gmcharlt@gmail.com |y.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #15 from M. de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 28517 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28517&action=edit Bug 12279: Counterpatch touching only preferences.pl Instead of a new routine in Templates.pm that actually only decodes the data, I would recommend to put this temporary fix into preferences.pl. Bug 11944 should provide the real fix.. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA CC| |m.de.rooy@rijksmuseum.nl Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | QA Contact|gmcharlt@gmail.com |m.de.rooy@rijksmuseum.nl --- Comment #16 from M. de Rooy <m.de.rooy@rijksmuseum.nl> --- Moving status to Failed QA in order to make a decision between both approaches. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off --- Comment #17 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to M. de Rooy from comment #15)
Created attachment 28517 [details] [review] Bug 12279: Counterpatch touching only preferences.pl
Instead of a new routine in Templates.pm that actually only decodes the data, I would recommend to put this temporary fix into preferences.pl. Bug 11944 should provide the real fix..
I first tried that same approach, but hesitated to implement it that way beacuse i wasn't sure about the effect of double encoding in some unknown scenario: i.e. we are messing with UTF-8 inside C4::Templates->output() and again on its output. Anyway, I tried this (again) on both YAML::Syck 1.19 and 1.27 and the patch fixes the problem. If Galen agrees on Marcel's approach I'd like to push this one soon. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28438|0 |1 is obsolete| | Attachment #28517|0 |1 is obsolete| | --- Comment #18 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 28557 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28557&action=edit Bug 12279: Diacritics in system preference editor broken Temporary fix into preferences.pl. Bug 11944 should provide the real fix.. Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28557|0 |1 is obsolete| | --- Comment #19 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 28614 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28614&action=edit Bug 12279: Diacritics in system preference editor broken Temporary fix into preferences.pl. Bug 11944 should provide the real fix.. Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #20 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patch pushed to the master branch. Thanks Marcel! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable --- Comment #21 from Galen Charlton <gmcharlt@gmail.com> --- Pushed to 3.16.x. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12279 --- Comment #22 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Jonathan Druart from comment #19)
Created attachment 28614 [details] [review] Bug 12279: Diacritics in system preference editor broken
Patch provided on bug 11944 (actually it just reverts this one). -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org