[Bug 32242] New: The job has not been sent to the message broker: (Wide character in syswrite ... )
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Bug ID: 32242 Summary: The job has not been sent to the message broker: (Wide character in syswrite ... ) Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: m.de.rooy@rijksmuseum.nl QA Contact: testopia@bugs.koha-community.org The full version: [2022/11/17 08:51:39] [WARN] The job has not been sent to the message broker: (Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl-base/IO/Handle.pm line 200.) at /usr/share/koha/Koha/BackgroundJob.pm line 147. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |31351 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31351 [Bug 31351] Worker dies on reindex job when operator last name/first name/branch name contains non-ASCII chars -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I think we should encode via Net::Stomp but I will first add a quick fix. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 143986 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=143986&action=edit Bug 32242: Resolve encoding issue when sending to Net::Stomp This is meant as a quick fix. It should be enhanced later on by letting Net::Stomp do the UTF8 conversion. Test plan: Run batch mod items. Add a Unicode char in the notes. Check if job runs and results are correct. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |major Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com, tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32243 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- FYI Tomas: The json to UTF8 conversion was handled correctly at the database layer but this bug is about sending json to RabbitMQ. That conversion is not handled properly (yet). See also bug 32243. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #143986|0 |1 is obsolete| | --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 143987 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=143987&action=edit Bug 32242: Resolve encoding issue when sending to Net::Stomp This is meant as a quick fix. It should be enhanced later on by letting Net::Stomp do the UTF8 conversion. Test plan: Run batch mod items. Add a Unicode char in the notes. Check if job runs and results are correct. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Still added a use Encode -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_22_11_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #143987|0 |1 is obsolete| | --- Comment #6 from David Nind <david@davidnind.com> --- Created attachment 143994 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=143994&action=edit Bug 32242: Resolve encoding issue when sending to Net::Stomp This is meant as a quick fix. It should be enhanced later on by letting Net::Stomp do the UTF8 conversion. Test plan: Run batch mod items. Add a Unicode char in the notes. Check if job runs and results are correct. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #7 from David Nind <david@davidnind.com> --- Testing notes (using KTD): 1. Use Batch item modification (Cataloging > Batch editing) to edit two records: - Add a Unicode character, such as 〼, in the non-public note and public note fields 2. View the details of job - no records modified 3. Check the items - no change was made 4. Check the /var/log/koha/kohadev/plack-intranet-error.log - there is a warning: [2022/11/17 11:43:13] [WARN] The job has not been sent to the message broker : (Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl-base/IO/Hand le.pm line 200. 5. Apply the patch (and restart all the things) 6. Repeat steps 1-4 - the records should be successfully modified and there should be no warning message in the logs. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #8 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- It's fixing the problem. Any good reasons to not import encode_utf8 from Encode? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #8)
It's fixing the problem.
Any good reasons to not import encode_utf8 from Encode?
I didnt need to import it. Main reason ;) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #10 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- That's not a discussion for this bug report, but I thought we agreed on the following pattern: use Module qw( sub list ); then call Module::list() of list() -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #10)
That's not a discussion for this bug report, but I thought we agreed on the following pattern:
use Module qw( sub list ); then call Module::list() of list()
It is conform that agreement with the list being empty ;) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #12 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 143998 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=143998&action=edit Bug 32242: Add selenium tests This is not testing the thing at the correct level, but at least we test the whole workflow. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #143994|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #143998|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #13 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 144016 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144016&action=edit Bug 32242: Resolve encoding issue when sending to Net::Stomp This is meant as a quick fix. It should be enhanced later on by letting Net::Stomp do the UTF8 conversion. Test plan: Run batch mod items. Add a Unicode char in the notes. Check if job runs and results are correct. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 144017 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144017&action=edit Bug 32242: Add selenium tests This is not testing the thing at the correct level, but at least we test the whole workflow. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #15 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |ASSIGNED Keywords|rel_22_11_candidate |additional_work_needed --- Comment #16 from Tomás Cohen Arazi <tomascohen@gmail.com> --- This broke tests: kohadev-koha@kohadevbox:/kohadevbox/koha$ prove t/db_dependent/Koha/BackgroundJobs.t t/db_dependent/Koha/BackgroundJobs.t .. 1/15 # Failed test 'No jobs found without userenv' # at t/db_dependent/Koha/BackgroundJobs.t line 135. # got: '2' # expected: '0' # Looks like you failed 1 test of 3. # Failed test 'search_limited' # at t/db_dependent/Koha/BackgroundJobs.t line 142. ... Reverting makes tests pass: git revert 022034a1c7b91adac539c7ed16fc7b48d566912e Please take your time to fix it for the release. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Passed QA --- Comment #17 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Tomás Cohen Arazi from comment #16)
This broke tests:
kohadev-koha@kohadevbox:/kohadevbox/koha$ prove t/db_dependent/Koha/BackgroundJobs.t t/db_dependent/Koha/BackgroundJobs.t .. 1/15 # Failed test 'No jobs found without userenv' # at t/db_dependent/Koha/BackgroundJobs.t line 135. # got: '2' # expected: '0' # Looks like you failed 1 test of 3.
# Failed test 'search_limited' # at t/db_dependent/Koha/BackgroundJobs.t line 142. ...
Reverting makes tests pass:
git revert 022034a1c7b91adac539c7ed16fc7b48d566912e
Please take your time to fix it for the release.
I was wrong. My bisect was flawed because of the reset_all I was using. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I am seeing this now Tomas, but it should not be related to this report: # Subtest: purge 1..9 ok 1 - Only the old finished type1 job would be purged not ok 2 - All finished old jobs would be purged with job_types = all # Failed test 'All finished old jobs would be purged with job_types = all' # at t/db_dependent/Koha/BackgroundJobs.t line 162. # got: '5' # expected: '2' ok 3 - All jobs still left in queue ok 4 - Only the old finished type1 job is purged ok 5 - 3 jobs still left in queue not ok 6 - The remaining old finished jobs is purged # Failed test 'The remaining old finished jobs is purged' # at t/db_dependent/Koha/BackgroundJobs.t line 183. # got: '4' # expected: '1' ok 7 - 2 jobs still left in queue ok 8 - Unfinished job still left in queue ok 9 - Recent finished job still left in queue # Looks like you failed 2 tests of 9. not ok 15 - purge -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #19 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #18)
not ok 2 - All finished old jobs would be purged with job_types = all # Failed test 'All finished old jobs would be purged with job_types = all' # at t/db_dependent/Koha/BackgroundJobs.t line 162. # got: '5' # expected: '2'
It seems that this test assumes that there are no other old jobs. So this should be corrected by removing them all first ? Opened bug 32304. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|additional_work_needed | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #20 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Tomás Cohen Arazi from comment #17)
kohadev-koha@kohadevbox:/kohadevbox/koha$ prove t/db_dependent/Koha/BackgroundJobs.t t/db_dependent/Koha/BackgroundJobs.t .. 1/15 # Failed test 'No jobs found without userenv' # at t/db_dependent/Koha/BackgroundJobs.t line 135. # got: '2' # expected: '0' # Looks like you failed 1 test of 3.
# Failed test 'search_limited' # at t/db_dependent/Koha/BackgroundJobs.t line 142.
Added a fix for this one on bug 32304 too now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |critical CC| |lucas@bywatersolutions.com --- Comment #21 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- This should have been backported into 22.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_22_05_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 --- Comment #22 from Lucas Gass <lucas@bywatersolutions.com> --- Created attachment 144772 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144772&action=edit Bug 32242: [22.05] fix selenium tests for 22.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|22.11.00 |22.11.00, 22.05.08 released in| | --- Comment #23 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for upcoming 22.05.08 release -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_22_05_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32242 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED CC| |arthur.suzuki@biblibre.com --- Comment #24 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- This one seems not to apply to 21.11 branch since it depends on 31351 (only in 22.05.x). Won't backport -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org