[Bug 35043] New: Handling of \t in PO files is confusing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Bug ID: 35043 Summary: Handling of \t in PO files is confusing Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: jonathan.druart+koha@gmail.com QA Contact: testopia@bugs.koha-community.org We have our own "quote_po" sub that is stating that Locale::PO::quote is buggy because it does not deal with quoting new lines correctly. However it seems that it is fixed now. Ideally we could use Locale::PO::quote everywhere, but it does not escape tab characters: $string =~ s/\\(?!t)/\\\\/g; # \t is a tab This means the following: msgid "Tabulation (\\t)" msgstr "Tabulation (\\t)" become: -msgid "Tabulation (\t)" -msgstr "Tabulation (\t)" And we are seeing the following on Weblate: https://snipboard.io/BjQmDC.jpg Note that Locale::PO has not been updated since 2014... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |34959 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34959 [Bug 34959] Translator tool generates too many changes -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 156937 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=156937&action=edit Bug 35043: Use Locale::PO->quote We have our own "quote_po" sub that is stating that Locale::PO::quote is buggy because it does not deal with quoting new lines correctly. However it seems that it is fixed now. Ideally we could use Locale::PO::quote everywhere, but it does not escape tab characters: $string =~ s/\\(?!t)/\\\\/g; # \t is a tab This means the following: msgid "Tabulation (\\t)" msgstr "Tabulation (\\t)" become: -msgid "Tabulation (\t)" -msgstr "Tabulation (\t)" And we are seeing the following on Weblate: https://snipboard.io/BjQmDC.jpg Note that Locale::PO has not been updated since 2014... The real problem behind this is that we have 2 methods to quote strings. At first glance it seems that Locale::PO::quote was not used before, but with the introduction of the koha-i18n project we will have scripts that will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote => Those scripts will be used on the translation server for post processing (security reason, marking potential XSS strings as fuzzy). Test plan: 0. Do not apply the patch 1. gulp po:update --lang LANG # Replace LANG with your favorite language code 2. git commit -a -m"init PO files" 3. Apply this patch 4. Repeate 1. 5. git diff => The change is about the "Tabulation" and "New line" strings from tools/csv-profiles.tt 6. Translate them (replace the \t and \n with %s) and remove the fuzzy flag 7. install the template: cd misc/translator && perl translate install LANG 8. Enable the language, use it and go to the "Nouveau profil CSV" view => Notice that the \t and \n are correctly displayed. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 --- Comment #2 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 156938 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=156938&action=edit Bug 35043: Do not have \n or \t appear in PO files This is a trick, so that we won't have to deal with \t in PO files. \n is not a problem, but it seems consistent to use the same method for both. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #156937|0 |1 is obsolete| | Attachment #156938|0 |1 is obsolete| | --- Comment #3 from Owen Leonard <oleonard@myacpl.org> --- Created attachment 156950 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=156950&action=edit Bug 35043: Use Locale::PO->quote We have our own "quote_po" sub that is stating that Locale::PO::quote is buggy because it does not deal with quoting new lines correctly. However it seems that it is fixed now. Ideally we could use Locale::PO::quote everywhere, but it does not escape tab characters: $string =~ s/\\(?!t)/\\\\/g; # \t is a tab This means the following: msgid "Tabulation (\\t)" msgstr "Tabulation (\\t)" become: -msgid "Tabulation (\t)" -msgstr "Tabulation (\t)" And we are seeing the following on Weblate: https://snipboard.io/BjQmDC.jpg Note that Locale::PO has not been updated since 2014... The real problem behind this is that we have 2 methods to quote strings. At first glance it seems that Locale::PO::quote was not used before, but with the introduction of the koha-i18n project we will have scripts that will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote => Those scripts will be used on the translation server for post processing (security reason, marking potential XSS strings as fuzzy). Test plan: 0. Do not apply the patch 1. gulp po:update --lang LANG # Replace LANG with your favorite language code 2. git commit -a -m"init PO files" 3. Apply this patch 4. Repeate 1. 5. git diff => The change is about the "Tabulation" and "New line" strings from tools/csv-profiles.tt 6. Translate them (replace the \t and \n with %s) and remove the fuzzy flag 7. install the template: cd misc/translator && perl translate install LANG 8. Enable the language, use it and go to the "Nouveau profil CSV" view => Notice that the \t and \n are correctly displayed. Signed-off-by: Owen Leonard <oleonard@myacpl.org> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 --- Comment #4 from Owen Leonard <oleonard@myacpl.org> --- Created attachment 156951 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=156951&action=edit Bug 35043: Do not have \n or \t appear in PO files This is a trick, so that we won't have to deal with \t in PO files. \n is not a problem, but it seems consistent to use the same method for both. Signed-off-by: Owen Leonard <oleonard@myacpl.org> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |jonathan.druart+koha@gmail. |ity.org |com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #1)
However it seems that it is fixed now.
Seems? My version on D11: our $VERSION = '0.27'; Do we know at which point this was solved? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #156950|0 |1 is obsolete| | --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157922 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157922&action=edit Bug 35043: Use Locale::PO->quote We have our own "quote_po" sub that is stating that Locale::PO::quote is buggy because it does not deal with quoting new lines correctly. However it seems that it is fixed now. Ideally we could use Locale::PO::quote everywhere, but it does not escape tab characters: $string =~ s/\\(?!t)/\\\\/g; # \t is a tab This means the following: msgid "Tabulation (\\t)" msgstr "Tabulation (\\t)" become: -msgid "Tabulation (\t)" -msgstr "Tabulation (\t)" And we are seeing the following on Weblate: https://snipboard.io/BjQmDC.jpg Note that Locale::PO has not been updated since 2014... The real problem behind this is that we have 2 methods to quote strings. At first glance it seems that Locale::PO::quote was not used before, but with the introduction of the koha-i18n project we will have scripts that will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote => Those scripts will be used on the translation server for post processing (security reason, marking potential XSS strings as fuzzy). Test plan: 0. Do not apply the patch 1. gulp po:update --lang LANG # Replace LANG with your favorite language code 2. git commit -a -m"init PO files" 3. Apply this patch 4. Repeate 1. 5. git diff => The change is about the "Tabulation" and "New line" strings from tools/csv-profiles.tt 6. Translate them (replace the \t and \n with %s) and remove the fuzzy flag 7. install the template: cd misc/translator && perl translate install LANG 8. Enable the language, use it and go to the "Nouveau profil CSV" view => Notice that the \t and \n are correctly displayed. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #156951|0 |1 is obsolete| | --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 157923 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157923&action=edit Bug 35043: Do not have \n or \t appear in PO files This is a trick, so that we won't have to deal with \t in PO files. \n is not a problem, but it seems consistent to use the same method for both. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #5)
(In reply to Jonathan Druart from comment #1)
However it seems that it is fixed now.
Seems? My version on D11: our $VERSION = '0.27';
Do we know at which point this was solved?
Well, if this is 2014, lets assume we are safe. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #10 from Fridolin Somers <fridolin.somers@biblibre.com> --- Depends on Bug 34959 not yet in 23.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Bug 35043 depends on bug 34959, which changed state. Bug 34959 Summary: Translator tool generates too many changes https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34959 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35043 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39201 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org