[Koha-devel] Strange encoding problems that sometimes happens

[Paul Poulain]

Hello,

English natives will less concerned by this problem probably, but there
are/were some places where strange encoding problems occurs. We have a
fix for them (use utf8 decode function), but I was wondering what was
the origin of this problem.

I think Frédérick Capovilla has a good explanation that I share with you
(it's a copy/paste of but 6479)
> It's been a while since I created that patch but here is what I understand :
>
> I remember that in the NewIssue subroutine of Serials.pm, The content of 
> $serialseq is concatenated with a variable fetched from a SQL query and that's
> where the problem happen.
>
> I did some tests to check the utf-8 flag on those two variables
> $serialseq = variable from the form (is_utf8 = false)
> $recievedlist = variable from SQL (is_utf8 = true)
>
> The encoding of the data fetched from SQL differs from the encoding of the data
> received from the form. If two string with a different encoding gets
> concatenated together, the encoding of one of the string is automatically
> changed, and we get an encoding problem on one half of the string.
>
> Using decode on $serialseq adds the utf-8 flag, so we don't get an encoding
> problem when we concatenate.
>
>
> We don't get this encoding problem when the first item is added in
> $recievedlist because $recievedlist is empty and doesn't automatically get the
> utf-8 flag. I'm guessing Perl DBI automatically addes the utf-8 flag if it
> finds utf-8 characters in a string returned from a SELECT.
If there are remaining places where this problem occurs, we know why &
how to fix it then !

PS: This explanation sounds highly logical to me. But if someone
disagree or has another explanation, feel free to drop a comment on the
bug or continue this thread !

-- 
Paul POULAIN
http://www.biblibre.com
Expert en Logiciels Libres pour l'info-doc
Tel : (33) 4 91 81 35 08