[Koha-bugs] [Bug 31351] Worker dies on reindex job when operator last name/first name/branch name contains non-ASCII chars

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31351

Katrin Fischer <katrin.fischer at bsz-bw.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #140222|0                           |1
        is obsolete|                            |
 Attachment #140236|0                           |1
        is obsolete|                            |

--- Comment #10 from Katrin Fischer <katrin.fischer at bsz-bw.de> ---
Created attachment 140262
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140262&action=edit
Bug 31351: Koha::BackgroundJob: Let database connection object handle utf8
transcoding

Our database connections have been set up so that they will
automatically convert perl encoded strings to utf8 encoded strings
when sending data to database tables and decode the utf8 encoded
strings from the tables back to internal perl strings. As we can see
from a call in Koha/BackgroundJob.pm we are encoding the perl internal
string to utf8 and then decoding it back to perl internal string:

my $data_dump = decode_json encode_utf8 $self->data;

We can skip this unnecessary encode<->decode step (as the database
object has done the decoding for us) by simply calling the
JSON->new->decode() method which doesn't perform any string decoding.

Furthermore, the original code was buggy and didn't always remember to
encode the unencoded strings, in Koha::BackgroundJob::process we can
see

my $context = decode_json($self->context);

is missing the encode step. Now after this change encoding before
decoding is not necessary as we are using the methods from the JSON
module that do not perform any transcoding.

Note to those concerned whether the old data in the database is
compatible with this new code:
 Luckily our database connection object seems to be smart and
 didn't utf8 encode the utf8 returned data from the old encode_json()
 calls (probably checks the utf8 flag for the string (Encode::is_utf8($str))).

To test whether this fixes the original bug reported of not being able
to schedule background jobs with koha user having non-ASCII letters in
their surname:
 1) Change your staff users surname/lastname to "ääää"
 2) Log out and back in.
 3) Go to a biblio record detail page and click "Select all" in the
 items table
 4) Click Delete selected items and proceed with the deletion
 5) Notice the batch item deletion job has failed status
 6) Apply patch and repeat but this time the deletion job should finish.

Signed-off-by: Katrin Fischer <katrin.fischer.83 at web.de>

-- 
You are receiving this mail because:
You are watching all bug changes.