[Bug 35907] New: Add ability to log all custom report runs with or without query
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Bug ID: 35907 Summary: Add ability to log all custom report runs with or without query Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: kyle@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |kyle@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #1 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161427 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161427&action=edit Bug 35907: Add ability to log all custom report runs with or without query Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. Test Plan: 1) Apply this patch 2) Modify your log4perl file, add the following: log4perl.logger.reports.execute.time = INFO, REPORTTIME log4perl.appender.REPORTTIME=Log::Log4perl::Appender::File log4perl.appender.REPORTTIME.filename=/tmp/report-time.log log4perl.appender.REPORTTIME.mode=append log4perl.appender.REPORTTIME.layout=PatternLayout log4perl.appender.REPORTTIME.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTTIME.utf8=1 log4perl.logger.reports.execute.query = INFO, REPORTQUERY log4perl.appender.REPORTQUERY=Log::Log4perl::Appender::File log4perl.appender.REPORTQUERY.filename=/tmp/report-query.log log4perl.appender.REPORTQUERY.mode=append log4perl.appender.REPORTQUERY.layout=PatternLayout log4perl.appender.REPORTQUERY.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTQUERY.utf8=1 3) Restart all the things! 4) Run a report somehow: CLI: ./misc/cronjobs/runreport.pl 1 API: /cgi-bin/koha/svc/report?id=1 Web: /cgi-bin/koha/reports/guided_reports.pl?reports=1&phase=Run this report 5) Note the report runs are logged to /tmp/report-time.log and /tmp/report-query.log -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #2 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161429 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161429&action=edit Bug 35907: Tidy execute_query -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161427|0 |1 is obsolete| | Attachment #161429|0 |1 is obsolete| | --- Comment #3 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161452 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161452&action=edit Bug 35907: Add ability to log all custom report runs with or without query Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. Test Plan: 1) Apply this patch 2) Modify your log4perl file, add the following: log4perl.logger.reports.execute.time = INFO, REPORTTIME log4perl.appender.REPORTTIME=Log::Log4perl::Appender::File log4perl.appender.REPORTTIME.filename=/tmp/report-time.log log4perl.appender.REPORTTIME.mode=append log4perl.appender.REPORTTIME.layout=PatternLayout log4perl.appender.REPORTTIME.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTTIME.utf8=1 log4perl.logger.reports.execute.query = INFO, REPORTQUERY log4perl.appender.REPORTQUERY=Log::Log4perl::Appender::File log4perl.appender.REPORTQUERY.filename=/tmp/report-query.log log4perl.appender.REPORTQUERY.mode=append log4perl.appender.REPORTQUERY.layout=PatternLayout log4perl.appender.REPORTQUERY.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTQUERY.utf8=1 3) Restart all the things! 4) Run a report somehow: CLI: ./misc/cronjobs/runreport.pl 1 API: /cgi-bin/koha/svc/report?id=1 Web: /cgi-bin/koha/reports/guided_reports.pl?reports=1&phase=Run this report 5) Note the report runs are logged to /tmp/report-time.log and /tmp/report-query.log -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #4 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161453 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161453&action=edit Bug 35907: Tidy execute_query -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161452|0 |1 is obsolete| | Attachment #161453|0 |1 is obsolete| | --- Comment #5 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161458 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161458&action=edit Bug 35907: Add ability to log all custom report runs with or without query Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. Test Plan: 1) Apply this patch 2) Modify your log4perl file, add the following: log4perl.logger.reports.execute.time = INFO, REPORTTIME log4perl.appender.REPORTTIME=Log::Log4perl::Appender::File log4perl.appender.REPORTTIME.filename=/tmp/report-time.log log4perl.appender.REPORTTIME.mode=append log4perl.appender.REPORTTIME.layout=PatternLayout log4perl.appender.REPORTTIME.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTTIME.utf8=1 log4perl.logger.reports.execute.query = INFO, REPORTQUERY log4perl.appender.REPORTQUERY=Log::Log4perl::Appender::File log4perl.appender.REPORTQUERY.filename=/tmp/report-query.log log4perl.appender.REPORTQUERY.mode=append log4perl.appender.REPORTQUERY.layout=PatternLayout log4perl.appender.REPORTQUERY.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTQUERY.utf8=1 3) Restart all the things! 4) Run a report somehow: CLI: ./misc/cronjobs/runreport.pl 1 API: /cgi-bin/koha/svc/report?id=1 Web: /cgi-bin/koha/reports/guided_reports.pl?reports=1&phase=Run this report 5) Note the report runs are logged to /tmp/report-time.log and /tmp/report-query.log -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #6 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 161459 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161459&action=edit Bug 35907: Tidy execute_query -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |blawlor@clamsnet.org --- Comment #7 from Brendan Lawlor <blawlor@clamsnet.org> --- I tried testing this but couldn't find the log files /tmp/report-time.log and /tmp/report-query.log I may be missing a step. 1. Applied the patch 2. Added the provided config to /koha/etc/log4perl.conf 3. restart_all 4. Set all Logging system prefs to 'Log' 5. Ran some reports 6. Check /tmp for report-time.log and report-query.log There were files in /tmp but none name like that. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #8 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to Brendan Lawlor from comment #7)
I tried testing this but couldn't find the log files /tmp/report-time.log and /tmp/report-query.log
I may be missing a step. 1. Applied the patch 2. Added the provided config to /koha/etc/log4perl.conf 3. restart_all 4. Set all Logging system prefs to 'Log' 5. Ran some reports 6. Check /tmp for report-time.log and report-query.log
There were files in /tmp but none name like that.
I just ran through the test plan and I can confirm it's still working as intended! Here is a video demonstration using runreport.pl: https://monosnap.com/file/CcQHcG8kTlJhvOpXxOITdWc5wj5eea -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161458|0 |1 is obsolete| | --- Comment #9 from Brendan Lawlor <blawlor@clamsnet.org> --- Created attachment 161806 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161806&action=edit Bug 35907: Add ability to log all custom report runs with or without query Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. Test Plan: 1) Apply this patch 2) Modify your log4perl file, add the following: log4perl.logger.reports.execute.time = INFO, REPORTTIME log4perl.appender.REPORTTIME=Log::Log4perl::Appender::File log4perl.appender.REPORTTIME.filename=/tmp/report-time.log log4perl.appender.REPORTTIME.mode=append log4perl.appender.REPORTTIME.layout=PatternLayout log4perl.appender.REPORTTIME.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTTIME.utf8=1 log4perl.logger.reports.execute.query = INFO, REPORTQUERY log4perl.appender.REPORTQUERY=Log::Log4perl::Appender::File log4perl.appender.REPORTQUERY.filename=/tmp/report-query.log log4perl.appender.REPORTQUERY.mode=append log4perl.appender.REPORTQUERY.layout=PatternLayout log4perl.appender.REPORTQUERY.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTQUERY.utf8=1 3) Restart all the things! 4) Run a report somehow: CLI: ./misc/cronjobs/runreport.pl 1 API: /cgi-bin/koha/svc/report?id=1 Web: /cgi-bin/koha/reports/guided_reports.pl?reports=1&phase=Run this report 5) Note the report runs are logged to /tmp/report-time.log and /tmp/report-query.log Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161459|0 |1 is obsolete| | --- Comment #10 from Brendan Lawlor <blawlor@clamsnet.org> --- Created attachment 161807 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161807&action=edit Bug 35907: Tidy execute_query Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #11 from Brendan Lawlor <blawlor@clamsnet.org> --- It turned out I didn't edit the correct config file in my first test. I edited /kohadevbox/koha/etc/log4perl.conf I guess maybe that is just an example config file. Thanks for showing me where the real config file is /etc/koha/sites/kohadev/log4perl.conf -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #12 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to Brendan Lawlor from comment #11)
It turned out I didn't edit the correct config file in my first test.
I edited /kohadevbox/koha/etc/log4perl.conf I guess maybe that is just an example config file.
Thanks for showing me where the real config file is /etc/koha/sites/kohadev/log4perl.conf
Ack! Thanks for the signoff Brendan! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161806|0 |1 is obsolete| | --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 166501 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166501&action=edit Bug 35907: Add ability to log all custom report runs with or without query Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place. For custom report runs, the category is plack-intranet.C4::Reports::Guided when run from the web interface, cron.C4::Reports::Guided when run from runreport.pl, and plack-intranet.C4::Auth when run from svc/report. We should add a more standardized report run log, both with and without the full query, so that administrators can log all report runs to a centralized location. If an administrator were to need the "point of entry" for reports, it is easy to include via parameters in PatternLayout. Test Plan: 1) Apply this patch 2) Modify your log4perl file, add the following: log4perl.logger.reports.execute.time = INFO, REPORTTIME log4perl.appender.REPORTTIME=Log::Log4perl::Appender::File log4perl.appender.REPORTTIME.filename=/tmp/report-time.log log4perl.appender.REPORTTIME.mode=append log4perl.appender.REPORTTIME.layout=PatternLayout log4perl.appender.REPORTTIME.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTTIME.utf8=1 log4perl.logger.reports.execute.query = INFO, REPORTQUERY log4perl.appender.REPORTQUERY=Log::Log4perl::Appender::File log4perl.appender.REPORTQUERY.filename=/tmp/report-query.log log4perl.appender.REPORTQUERY.mode=append log4perl.appender.REPORTQUERY.layout=PatternLayout log4perl.appender.REPORTQUERY.layout.ConversionPattern=[%d] [%p] [%P] %m%n log4perl.appender.REPORTQUERY.utf8=1 3) Restart all the things! 4) Run a report somehow: CLI: ./misc/cronjobs/runreport.pl 1 API: /cgi-bin/koha/svc/report?id=1 Web: /cgi-bin/koha/reports/guided_reports.pl?reports=1&phase=Run this report 5) Note the report runs are logged to /tmp/report-time.log and /tmp/report-query.log Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161807|0 |1 is obsolete| | --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 166502 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166502&action=edit Bug 35907: Tidy execute_query Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Kyle M Hall from comment #0)
Because of the way Koha::Logger has been used to log to different categories based on the interface and caller, it can be extremely hard to log all of a particular log statement to one place.
Just change your log4perl.conf configuration? And a sysadmin could always concatenate all logs and grep on whatever he wants to find? This works as expected but feels a bit like a workaround. + my $prefix = $params->{prefix} // 1; I would rather rename this one personally. What you want here is to suppress the plack prefix, not add a custom prefix. no_plack => 1 or something like that ? If the log4perl config for reports.execute.query is not found, it seems that Logger just silently ignores logging. This costed me about 15ms per call. Probably acceptable. Passing QA since it is just a few lines. But hesitantly since our code doesnt get cleaner. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |24.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Needs documenting CC| |fridolin.somers@biblibre.co | |m --- Comment #17 from Fridolin Somers <fridolin.somers@biblibre.com> --- Not backported to 23.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35907 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |caroline.cyr-la-rose@inlibr | |o.com Status|Needs documenting |RESOLVED Resolution|--- |FIXED --- Comment #18 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- This looks like backend architecture modification, nothing to change/edit in the manual. Please reopen with more details if needed. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org