[Bug 11054] New: Creating a child record has problems with accented Latin characters
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Bug ID: 11054 Summary: Creating a child record has problems with accented Latin characters Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Cataloging Assignee: gmcharlt@gmail.com Reporter: colin.campbell@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl When you select New child record from the catalogue detail page. If the data that will be used to create the host field contains latin characters with diacritics from the range covered by latin-1 they are incorrectly encoded in the resulting child record. This is not the case with non-latin characters (Cyrillic, Arabic and Chinese tested) and I suspect with latin diacritic combinations that are not in the latin-1 range -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Assignee|gmcharlt@gmail.com |colin.campbell@ptfs-europe. | |com --- Comment #1 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 22005 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22005&action=edit Proposed Patch To test. Find a record with an accented character in the title e.g. a vowel with an acute or grave or an n with a tilde in the detail page select new/new child record the title will be copied into 773 $t (marc21) without the patch garbage replaces the accented characters with the patch they appear correctly. I've confirmed that the change does not have any detrimental effects when copying non-latin characters e.g. Chinese, Cyrillic, Arabic which were already being correctly copied -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22005|0 |1 is obsolete| | --- Comment #2 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 22006 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22006&action=edit [SIGNED-OFF] Bug 11054: Specify utf8 encoding when creating a child record When the only non-ascii characters in a new child record are latin + diacritic in the range covered by Latin-1 they are not being added to a new child record correctly encoded. Explicitly setting the record's encodiing ensures that they are interpreted correctly. see "Unicode Bug" in perldoc perlunicode for background on why these characters are special Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Work as described. No koha-qa errors Test search for a record with diacritics on author or title press New Child Record without patch, 773's with mangled chars with patch, all fine This has nothing to do with major problems of encoding, here we create a new record and as CPAN page states, default encoding is MARC-8, so it's the right solution. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |bgkriegel@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion CC| |paul.poulain@biblibre.com QA Contact|testopia@bugs.koha-communit |paul.poulain@biblibre.com |y.org | --- Comment #3 from Paul Poulain <paul.poulain@biblibre.com> --- QA comment * one-liner * I'm wondering if it would not be better to use C4/Charset.pm sub SetMarcUnicodeFlag($marc_record, $marc_flavour); for this : it raises the utf8 flag, but ALSO set UNIMARC field 100 if applicable or position 9 of the leader for MARC21/NORMARC Setting to "in discussion", colin, argue and/or reset status to signed-off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Signed Off --- Comment #4 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Paul SetMarcUnicodeFlag sets the MARC flag in the leader the bug is not related to this the patch is setting the utf-8 flag in the perl var containing the string. A different thing Setting back to signed off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 --- Comment #5 from Paul Poulain <paul.poulain@biblibre.com> --- (In reply to Colin Campbell from comment #4)
Paul SetMarcUnicodeFlag sets the MARC flag in the leader the bug is not related to this the patch is setting the utf-8 flag in the perl var containing the string. A different thing Setting back to signed off
Right, but it ALSO set the flag: $marc_record->encoding('UTF-8'); (3 lines after the sub prototype) I still think we should use this sub instead of a direct ->encoding -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 --- Comment #6 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- For what is worth, with patch applied, I replaced line 824 in addbiblio.pl $record->encoding( 'UTF-8' ); with SetMarcUnicodeFlag($record, $marcflavour); Same (good) result, and perhaps is more general. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22006|0 |1 is obsolete| | --- Comment #7 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 22317 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22317&action=edit Amended patch Make the change as suggested in the comments -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #8 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Switched back to needs sign off. Bernardo it should pass your test -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22317|0 |1 is obsolete| | --- Comment #9 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 22318 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22318&action=edit [SIGNED-OFF] Bug 11054: Specify utf8 encoding when creating a child record When the only non-ascii characters in a new child record are latin + diacritic in the range covered by Latin-1 they are not being added to a new child record correctly encoded. Explicitly setting the record's encodiing ensures that they are interpreted correctly. see "Unicode Bug" in perldoc perlunicode for background on why these characters are special Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works well. No koha-qa errors. Same test as before 1) with a record with diacritics in author/title, 2) press new child record, on new record 773adt with mangled accents 3) Apply patch 4) Repeat 2, now good results -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22318|0 |1 is obsolete| | --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 22449 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22449&action=edit [PASSED QA] Bug 11054: Specify utf8 encoding when creating a child record When the only non-ascii characters in a new child record are latin + diacritic in the range covered by Latin-1 they are not being added to a new child record correctly encoded. Explicitly setting the record's encodiing ensures that they are interpreted correctly. see "Unicode Bug" in perldoc perlunicode for background on why these characters are special Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works well. No koha-qa errors. Same test as before 1) with a record with diacritics in author/title, 2) press new child record, on new record 773adt with mangled accents 3) Apply patch 4) Repeat 2, now good results Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Passes all tests and QA script. Tested with a record containing umlauts in the title, $t was empty before and umlauts in $a mangled. After applying the patch fields are correctly filled and diacritics display correctly. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de QA Contact|paul.poulain@biblibre.com |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |gmcharlt@gmail.com --- Comment #11 from Galen Charlton <gmcharlt@gmail.com> --- Pushed to master. Thanks, Colin! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |tomascohen@gmail.com --- Comment #12 from Tomás Cohen Arazi <tomascohen@gmail.com> --- This patch has been pushed to 3.12.x, will be in 3.12.7. Thanks Colin! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11054 giuseppe.angilella@ct.infn.it changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |giuseppe.angilella@ct.infn. | |it -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org