https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37164 Bug ID: 37164 Summary: worker-output.log file grows to incredible sizes and causing the server to crash Change sponsored?: --- Product: Koha Version: 23.05 Hardware: All OS: Linux Status: NEW Severity: critical Priority: P5 - low Component: bugs.koha-community.org Assignee: koha-bugs@lists.koha-community.org Reporter: yuriy.kotsyuk@oa.edu.ua QA Contact: testopia@bugs.koha-community.org Created attachment 168002 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168002&action=edit scrinchot worker-output.log.2 (fragment) The worker-output.log file grows to incredible sizes. While checking one of the files (5 GB), a warning entry was found and this entry was duplicated a huge number of times over a short period of time. The first two records are as follows: DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('database=koha_library;host=localhost;port=3306','koha_library',...) failed: Lost connection to MySQL server at 'reading initial communication packet', system error: 104 at /usr/share/koha/lib/Koha/Database.pm line 91. at /usr/share/koha/lib/Koha/Objects.pm line 312 DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('database=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/Koha/Database.pm line 91. at /usr/share/koha/lib/Koha/Objects.pm line 312 Line number three indicates the date and time 2024/05/20 17:35:52 In general, the record looks like this: [2024/05/20 17:35:52] [WARN] Frame not processed - malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "You must log in usin...") at /usr/share/koha/bin/workers/background_jobs_worker.pl line 120. main::catch {...} /usr/share/koha/bin/workers/background_jobs_worker.pl (122) This entry occurs 985 times in a row. Recording in the next second: [2024/05/20 17:35:53] [WARN] Frame not processed - malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "You must log in usin...") at /usr/share/koha/bin/workers/background_jobs_worker.pl line 120. main::catch {...} /usr/share/koha/bin/workers/background_jobs_worker.pl (122) occurs 1483 times in a row. Recording in the next second: [2024/05/20 17:35:54] [WARN] Frame not processed - malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "You must log in usin...") at /usr/share/koha/bin/workers/background_jobs_worker.pl line 120. main::catch {...} /usr/share/koha/bin/workers/background_jobs_worker.pl (122) occurs 6952 times A total of 16,709,173 messages of the same type were generated from 2024/05/20 17:35:52 to 2024/05/21 10:59:08 After this series of messages, other messages have already appeared: Cannot connect to the message broker, the jobs will be processed anyway (Failed to connect: Error connecting to localhost:61613: Connection refused at /usr/share/perl5/Net/Stomp.pm line 27.; giving up at /usr/share/perl5/Net/Stomp.pm line 27. ) at /usr/share/koha/bin/workers/background_jobs_worker.pl line 92. Cannot connect to the message broker, the jobs will be processed anyway (Failed to connect: Error connecting to localhost:61613: Connection refused at /usr/share/perl5/Net/Stomp.pm line 27.; giving up at /usr/share/perl5/Net/Stomp.pm line 27. ) at /usr/share/koha/bin/workers/background_jobs_worker.pl line 92. Today, worker-output.log was 108 GB in size and took up all free disk space, causing the server to crash. Perhaps someone had the same problems and was able to overcome them -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.