[Bug 42766] New: Add ScriptLog syspref and auto-logging for non-cron Koha scripts
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Bug ID: 42766 Summary: Add ScriptLog syspref and auto-logging for non-cron Koha scripts Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org CC: jake.deery@openfifth.co.uk, robin@catalyst.net.nz Depends on: 42714 Bug 42714 added automatic CRONJOBS action-log entries (start and end) for scripts that declare themselves as cron jobs via 'use Koha::Script -cron'. That bug deliberately excluded non-cron scripts (admin tools, migration utilities, maintenance scripts) to avoid polluting the CRONJOBS log. This bug extends the same auto-logging idea to all Koha scripts that use bare 'use Koha::Script' (without -cron), gated behind a new ScriptLog system preference that mirrors CronjobLog. Work required: 1. Add a new 'ScriptLog' system preference (type: YesNo, default: 0). 2. Extend the INIT/END blocks in Koha::Script to call a new logscriptaction() function (or reuse/extend cronlogaction()) when ScriptLog is enabled and $_cron is false. The logged module should be distinct from CRONJOBS (e.g. 'SCRIPTS') so admins can filter the two separately in the action log viewer. 3. Implement @ARGV scrubbing before logging. Some scripts accept sensitive values on the command line (passwords, tokens). A concrete example is misc/devel/create_superlibrarian.pl which accepts --password. The mechanism should redact any argument that follows a flag whose name contains 'password', 'passwd', 'secret', 'token', or similar, replacing the value with '[REDACTED]'. Affected scripts (or a central list in Koha::Script) should be able to declare which flags are sensitive. 4. Update the system preferences admin UI (.pref file) to expose ScriptLog alongside or near CronjobLog. 5. Add tests: - Unit test for the @ARGV scrubbing logic. - Verify ScriptLog=0 suppresses logging for non-cron scripts. - Verify ScriptLog=1 creates action log entries for non-cron scripts. Test plan: 1. Apply patch. 2. Enable ScriptLog in Administration > System preferences > Logging. 3. Run a non-cron script, e.g.: perl misc/devel/create_superlibrarian.pl --userid admin --password secret 4. Check Administration > Action logs (module: SCRIPTS) and confirm a Run entry exists with the password value replaced by [REDACTED]. 5. Confirm no entry appears in the CRONJOBS module for this run. 6. Run a cron script, e.g.: perl misc/cronjobs/cleanup_database.pl --confirm 7. Confirm the entry appears under CRONJOBS (not SCRIPTS). 8. Disable ScriptLog and re-run step 3; confirm no log entry is created. 9. Run the test suite: prove t/Koha/Script.t t/Koha/Script_cron.t Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42714 [Bug 42714] Koha::Script should automatically log start and end of scripts when they are run -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Sponsorship status|--- |Unsponsored Status|NEW |Needs Signoff Assignee|koha-bugs@lists.koha-commun |martin.renvoize@openfifth.c |ity.org |o.uk Patch complexity|--- |Small patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 199880 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199880&action=edit Bug 42766: Add ScriptLog system preference Adds a new YesNo system preference 'ScriptLog' in the Logging section of system preferences, adjacent to CronjobLog. When enabled, the start and end of non-cron Koha scripts (those using bare 'use Koha::Script' rather than 'use Koha::Script -cron') will be recorded in the action log under the SCRIPTS module. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #2 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 199881 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199881&action=edit Bug 42766: Add logscriptaction() to C4::Log Adds a logscriptaction() function parallel to cronlogaction() that logs to the SCRIPTS action-log module rather than CRONJOBS, gated behind the new ScriptLog system preference. Exported from C4::Log alongside the existing cronlogaction(). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 199882 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199882&action=edit Bug 42766: Add ScriptLog auto-logging and ARGV scrubbing to Koha::Script Extends the INIT/END blocks added by Bug 42714 to also log non-cron scripts (those using bare 'use Koha::Script') to the SCRIPTS action-log module when the new ScriptLog system preference is enabled. Also introduces _scrub_argv(), which redacts the value of any command- line flag whose name contains a sensitive pattern (password, secret, token, etc.) before the argument list is logged. Scrubbing applies to both cron and non-cron scripts, covering both --flag value and --flag=value forms. This prevents credentials passed on the command line (e.g. misc/devel/create_superlibrarian.pl --password ...) from appearing in the action log. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 199883 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199883&action=edit Bug 42766: Add tests for logscriptaction() and _scrub_argv() t/db_dependent/Log.t - Add subtest verifying logscriptaction() logs to the SCRIPTS module, respects the ScriptLog preference, prefixes loginfo with $0, appends the info parameter, and honours custom action values. t/Koha/Script_scrub_argv.t - New test file exercising all branches of _scrub_argv(): --flag value form, --flag=value form, non-sensitive flags, case- insensitive matching, multiple sensitive patterns (token, secret, apikey, api_key), and a flag immediately followed by another flag. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |nick@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199880|0 |1 is obsolete| | Attachment #199881|0 |1 is obsolete| | Attachment #199882|0 |1 is obsolete| | Attachment #199883|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #5 from David Nind <david@davidnind.com> --- Created attachment 199887 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199887&action=edit Bug 42766: Add ScriptLog system preference Adds a new YesNo system preference 'ScriptLog' in the Logging section of system preferences, adjacent to CronjobLog. When enabled, the start and end of non-cron Koha scripts (those using bare 'use Koha::Script' rather than 'use Koha::Script -cron') will be recorded in the action log under the SCRIPTS module. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #6 from David Nind <david@davidnind.com> --- Created attachment 199888 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199888&action=edit Bug 42766: Add logscriptaction() to C4::Log Adds a logscriptaction() function parallel to cronlogaction() that logs to the SCRIPTS action-log module rather than CRONJOBS, gated behind the new ScriptLog system preference. Exported from C4::Log alongside the existing cronlogaction(). Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #7 from David Nind <david@davidnind.com> --- Created attachment 199889 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199889&action=edit Bug 42766: Add ScriptLog auto-logging and ARGV scrubbing to Koha::Script Extends the INIT/END blocks added by Bug 42714 to also log non-cron scripts (those using bare 'use Koha::Script') to the SCRIPTS action-log module when the new ScriptLog system preference is enabled. Also introduces _scrub_argv(), which redacts the value of any command- line flag whose name contains a sensitive pattern (password, secret, token, etc.) before the argument list is logged. Scrubbing applies to both cron and non-cron scripts, covering both --flag value and --flag=value forms. This prevents credentials passed on the command line (e.g. misc/devel/create_superlibrarian.pl --password ...) from appearing in the action log. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #8 from David Nind <david@davidnind.com> --- Created attachment 199890 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199890&action=edit Bug 42766: Add tests for logscriptaction() and _scrub_argv() t/db_dependent/Log.t - Add subtest verifying logscriptaction() logs to the SCRIPTS module, respects the ScriptLog preference, prefixes loginfo with $0, appends the info parameter, and honours custom action values. t/Koha/Script_scrub_argv.t - New test file exercising all branches of _scrub_argv(): --flag value form, --flag=value form, non-sensitive flags, case- insensitive matching, multiple sensitive patterns (token, secret, apikey, api_key), and a flag immediately followed by another flag. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement allows release notes| |libraries to log when | |scripts are run (such as | |misc/devel/create_superlibr | |arian.pl) using the new | |ScriptLog system | |preference. CC| |david@davidnind.com --- Comment #9 from David Nind <david@davidnind.com> --- Two suggestions: 1. In the log viewer, module names are show in sentence case in the module column for log entries. So I would suggest changing "SCRIPTS" to "Scripts". 2. I can't see a way when selecting the module for the log viewer to filter to script logs - there is "Cron jobs" but no "Script log". Testing notes (using KTD): 1. After applying the patch: - updatedatabase - restart_all 2. Also enable the CronjobLog system preference. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199887|0 |1 is obsolete| | Attachment #199888|0 |1 is obsolete| | Attachment #199889|0 |1 is obsolete| | Attachment #199890|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #10 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200021 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200021&action=edit Bug 42766: Add ScriptLog system preference Adds a new YesNo system preference 'ScriptLog' in the Logging section of system preferences, adjacent to CronjobLog. When enabled, the start and end of non-cron Koha scripts (those using bare 'use Koha::Script' rather than 'use Koha::Script -cron') will be recorded in the action log under the SCRIPTS module. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #11 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200022 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200022&action=edit Bug 42766: Add logscriptaction() to C4::Log Adds a logscriptaction() function parallel to cronlogaction() that logs to the SCRIPTS action-log module rather than CRONJOBS, gated behind the new ScriptLog system preference. Exported from C4::Log alongside the existing cronlogaction(). Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #12 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200023 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200023&action=edit Bug 42766: Add ScriptLog auto-logging and ARGV scrubbing to Koha::Script Extends the INIT/END blocks added by Bug 42714 to also log non-cron scripts (those using bare 'use Koha::Script') to the SCRIPTS action-log module when the new ScriptLog system preference is enabled. Also introduces _scrub_argv(), which redacts the value of any command- line flag whose name contains a sensitive pattern (password, secret, token, etc.) before the argument list is logged. Scrubbing applies to both cron and non-cron scripts, covering both --flag value and --flag=value forms. This prevents credentials passed on the command line (e.g. misc/devel/create_superlibrarian.pl --password ...) from appearing in the action log. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #13 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200024 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200024&action=edit Bug 42766: Add tests for logscriptaction() and _scrub_argv() t/db_dependent/Log.t - Add subtest verifying logscriptaction() logs to the SCRIPTS module, respects the ScriptLog preference, prefixes loginfo with $0, appends the info parameter, and honours custom action values. t/Koha/Script_scrub_argv.t - New test file exercising all branches of _scrub_argv(): --flag value form, --flag=value form, non-sensitive flags, case- insensitive matching, multiple sensitive patterns (token, secret, apikey, api_key), and a flag immediately followed by another flag. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #14 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200025 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200025&action=edit Bug 42766: (QA folllow-up) Broaden sensitive patterns We are all lazy, we aren't going to type --password everytime, we should protect 'p' and 'pass' as well This could likely get broader too later TO test: 1 - perl misc/devel/create_superlibrarian.pl --branchcode=CPL --categorycode=S --cardnumber 3 --userid=ted3 --p=dog 2 - Note password is logged 3 - Apply this patch 4 - perl misc/devel/create_superlibrarian.pl --branchcode=CPL --categorycode=S --cardnumber 33 --userid=ted33 --p=dog 5 - Password redacted in log 6 - perl misc/devel/create_superlibrarian.pl --branchcode=CPL --categorycode=S --cardnumber 333 --userid=ted333 --pass=dog 7 - Password still redacted -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 --- Comment #15 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 200026 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200026&action=edit Bug 42766: (QA follow-up) Add 'SCRIPTS' to log viewer Apply this patch and confirm you can limit to 'Scripts' and that 'Scripts' is not in all caps in the results -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This enhancement allows |This enhancement allows release notes|libraries to log when |libraries to log when |scripts are run (such as |scripts are run (such as |misc/devel/create_superlibr |misc/devel/create_superlibr |arian.pl) using the new |arian.pl) using the new |ScriptLog system |ScriptLog system |preference. |preference. | | | |In the log viewer, | |'Scripts' is added as an | |option to the log viewer | |modules for filtering | |results, and any log | |entries show as the | |'Scripts' module. --- Comment #16 from David Nind <david@davidnind.com> --- Thanks for the follow-ups Nick! Things work as described in the follow-up patches. David -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42766 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org