https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #1 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 161852 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161852&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional Test plan : 1. Apply batch 2. Run the script 3. Enter the corresponding number to choose a method to run 4. Complete the Test -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #4 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 161954 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161954&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Patch doesn't apply --- Comment #7 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Hi Phan, The first patch of the 3 is the first patch that does not apply, it needs to either be rebased and resubmitted or made obsolete. If you don't want to rebase and want to submit a new one, you can make it obsolete by visiting https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161953&action=edit and clicking "edit details", tick "obsolete". Me and others are available to chat on IRC and provide help if needed. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161953|0 |1 is patch| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #8 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 162349 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162349&action=edit Bug 36027: Tidy code for QA script -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #161954|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #10 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Putting this as "Needs Signoff" to look at again later and there's no sign-off line by anyone just yet. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #14 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 162546 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162546&action=edit Bug 36027 : Tidy the code -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #15 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 163259 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=163259&action=edit Bug 36027 : (follow-up) Improvement : Add check option i add the check option so that instead of skipping certain checks, you can also choose a specific test to run -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Phan Tung Bui <phan-tung.bui@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162351|0 |1 is obsolete| | Attachment #162540|0 |1 is obsolete| | Attachment #162546|0 |1 is obsolete| | Attachment #163259|0 |1 is obsolete| | --- Comment #17 from Phan Tung Bui <phan-tung.bui@inlibro.com> --- Created attachment 163956 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=163956&action=edit Bug 36027: change interactive to pass param and add check options -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #163956|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |36039 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36039 [Bug 36039] Output of database_audit.pl should be accessible through UI -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |julian.maurice@biblibre.com |y.org | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Alexandre <alexandre.noel@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Alexandre <alexandre.noel@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #167766|0 |1 is obsolete| | --- Comment #22 from Alexandre <alexandre.noel@inlibro.com> --- Created attachment 167787 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167787&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional We can specify which sections to search for data inconsistencies using check options, or which sections to skip using skip options. TEST PLAN: Make sure there are data inconsistencies present in at least two categories to validate the results of the command. 1. Apply the patch 2. Run the command without any argument and confirm that every inconsistences are reported. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|phan-tung.bui@inlibro.com |alexandre.noel@inlibro.com CC| |david@davidnind.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #166117|0 |1 is obsolete| | --- Comment #23 from David Nind <david@davidnind.com> --- Created attachment 168953 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168953&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement updates release notes| |the data inconsistencies | |script | |(misc/maintenance/search_fo | |r_data_inconsistencies.pl) | |so that you can optionally | |run specific checks, rather | |than running all the checks | |at the same time. | | | |Examples: | |- Run all checks | |(no change): | |misc/maintenance/search_for | |_data_inconsistencies.pl | |- | |Run checks for item type | |issues and patrons with an | |invalid age for their | |category: | |misc/maintenance/search_for | |_data_inconsistencies.pl | |--check-status --check-age | |- Skip specific checks, but | |run all the others: | |misc/maintenance/search_for | |_data_inconsistencies.pl | |--skip-status --skip-title | | | |Use | |misc/maintenance/search_for | |_data_inconsistencies.pl | |--help for all the check | |and skip options. Severity|normal |enhancement --- Comment #25 from David Nind <david@davidnind.com> --- Thanks Alexandre for the patch. I've signed this off, as it does what it says it will do. I have made some comments. I will leave it up to you (or the QA Team) to decide if these need addressing. I think this is an enhancement, rather than fixing something that is wrong. So I have changed the status. I have also added a draft release note. Testing notes (using koha-testing-docker (KTD)): 1. Set up data so there are inconsistencies: 1.1 There is already one inconsistency in the sample data, for record 369 (encoding errors) 1.2 I created an inconsistency for patron age and patron category type: . koha-mysql kohadev . update borrowers set categorycode = "K" where borrowernumber=49; 2. Step 3, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --check-status --check-age 3. Step 4, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --skip-status 4. Step 5, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --check-age --skip-status Some comments: 1. The check-status option includes checking for item type issues (as described in the help description), but it also checks for invalid MARCXML. I think that the MARCXML check should have its own option (--check-marcxml Check for invalid MARCXML in bibliographic records). 1.1 I'm not sure how hard this to do (I am not a developer). 1.2 I'm not sure whether there are other checks in the script that are not included in the description and have their own options. 1.2 Instead of --check-status, --check-itemtype 2. Instead of --check-branch, use --check-library. While branch is used in the underlying code, the terminology guidelines suggest we should use library. (https://wiki.koha-community.org/wiki/Terminology#L) 3. Warning when you use --check-* and --skip-* options in the same command (step 5). I've made a wording change suggestion. The current text says to me that if you use skip and check options in the same command, the skip options are ignored. That is, it will ignore/not run the skip options: . Current warning: "Warning : skip options are ignored when check options are provided" . Suggested warning: "Warning: The --skip-* option(s) were ignored. Only the --check-* options in the command were run." 4. Description and help text. I think this needs some work with formatting and the text. I would be happy to add a follow-up patch with some suggested changes. 5. The QA script has warnings (to run the QA script when using koha-testing-docker): qa WARN misc/maintenance/search_for_data_inconsistencies.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 83, now: 206) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #27 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 169229 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=169229&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional We can specify which sections to search for data inconsistencies using check options, or which sections to skip using skip options. TEST PLAN: Make sure there are data inconsistencies present in at least two categories to validate the results of the command. 1. Apply the patch 2. Run the command without any argument and confirm that every inconsistences are reported. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #28 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I know that bug 36039 is blocked by this, but I'd like to ask some questions at least before pushing. 1) Quite a lot of untidyness added, please fix. WARN misc/maintenance/search_for_data_inconsistencies.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 83, now: 206) Can we fix this here or on the other bug? 2) New parameters (not blocker) I wonder if the chosen pattern is good as it requires that we add 2 new parameters for every check that we add to the script. We do have something similar in cleanup_database.pl for action logs and it seems more logical to me. --log-module Specify which action log modules to trim. Repeatable. --preserve-log Specify which action logs to exclude. Repeatable. --logs DAYS purge entries from action_logs older than DAYS days. Defaults to 180 days if no days specified. So instead of a unique parameter, you can use preserve-log with different codes. I am not sure what is better, just putting the question out here. 3) Script should not start without -c or any parameters... (not blocker) This is something for a separate bug. But we once said that scripts should output help when run without parameter and require a confirmation flag. As this can run quite a time, this still seems a good idea... (leaving this here for filing later) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Hammat wele <hammat.wele@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #169228|0 |1 is obsolete| | --- Comment #29 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 170847 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170847&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #31 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 170849 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170849&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Olivier Vezina <ovezina@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #170847|0 |1 is obsolete| | --- Comment #32 from Olivier Vezina <ovezina@inlibro.com> --- Created attachment 173269 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=173269&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Olivier Vezina <ovezina@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #170849|0 |1 is obsolete| | --- Comment #34 from Olivier Vezina <ovezina@inlibro.com> --- Created attachment 173271 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=173271&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #36 from David Nind <david@davidnind.com> --- Everything still seems to work as expected. However, I've Failed QA as any command run generates this line in the output, which I don't think it should: Subroutine CheckRelationshipsLoop redefined at misc/maintenance/search_for_data_inconsistencies.pl line 708. Updated testing notes (from comment #25) ===================== Testing notes (using KTD): 1. Set up data so that there are inconsistencies: 1.1 There is already one inconsistency in the sample data, for record 369 (encoding errors) 1.2 I created an inconsistency for patron age and patron category type: . koha-mysql kohadev . update borrowers set categorycode = "K" where borrowernumber=49; 2. Step 3, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --check-status --check-age 3. Step 4, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --skip-status 4. Step 5, ran check with: misc/maintenance/search_for_data_inconsistencies.pl --check-age --skip-status Updated comments (from me, comment #25) ================ I'll file separate bugs for these comments. 1. The check-status option includes checking for item type issues (as described in the help description), but it also checks for invalid MARCXML. I think that the MARCXML check should have its own option (--check-marcxml Check for invalid MARCXML in bibliographic records). 1.1 I'm not sure how hard this to do (I am not a developer). 1.2 I'm not sure whether there are other checks in the script that are not included in the description and have their own options. 1.2 Instead of --check-status, --check-itemtype 2. Instead of --check-branch, use --check-library. While branch is used in the underlying code, the terminology guidelines suggest we should use library. (https://wiki.koha-community.org/wiki/Terminology#L) 3. Warning when you use --check-* and --skip-* options in the same command (step 5). I've made a wording change suggestion. The current text says to me that if you use skip and check options in the same command, the skip options are ignored. That is, it will ignore/not run the skip options: . Current warning: "Warning : skip options are ignored when check options are provided" . Suggested warning: "Warning: The --skip-* option(s) were ignored. Only the --check-* options in the command were run." 4. Description and help text. I think this needs some work with formatting and the text. I will add a follow-up patch with some suggested changes. Additional comments (from Katrin, comment #28) =================== Just so that these don't get lost from Katrin's comments. 1. New parameters (not blocker) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I wonder if the chosen pattern is good as it requires that we add 2 new parameters for every check that we add to the script. We do have something similar in cleanup_database.pl for action logs and it seems more logical to me. --log-module Specify which action log modules to trim. Repeatable. --preserve-log Specify which action logs to exclude. Repeatable. --logs DAYS purge entries from action_logs older than DAYS days. Defaults to 180 days if no days specified. So instead of a unique parameter, you can use preserve-log with different codes. I am not sure what is better, just putting the question out here. 2. Script should not start without -c or any parameters... (not blocker) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is something for a separate bug. But we once said that scripts should output help when run without parameter and require a confirmation flag. As this can run quite a time, this still seems a good idea... (leaving this here for filing later) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #38 from Olivier Vezina <ovezina@inlibro.com> --- Created attachment 173556 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=173556&action=edit Bug 36027: (follow-up) Fixed code duplication and add mandatory options An option is now mandatory for the script to lunch. 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|36039 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36039 [Bug 36039] The output of audit_database.pl should be accessible through the UI -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #40 from Magnus Enger <magnus@libriotech.no> --- *** Bug 31093 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no Blocks| |38755 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38755 [Bug 38755] search_for_data_inconsistencies.pl: Fix list of inconsistencies -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lisette@bywatersolutions.co | |m Keywords| |no-sandbox -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Hammat wele <hammat.wele@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177270|0 |1 is obsolete| | --- Comment #43 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 177283 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177283&action=edit Bug 36027: (follow-up) Adding check-all option This patch add --check-all option, to run all the checks without specifying each one 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with 'check-all' arguments (ex: ./search_for_data_inconsistencies.pl --check-all), confirm that all the check is done. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #173269|0 |1 is obsolete| | --- Comment #44 from David Nind <david@davidnind.com> --- Created attachment 177438 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177438&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #173271|0 |1 is obsolete| | --- Comment #46 from David Nind <david@davidnind.com> --- Created attachment 177440 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177440&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177283|0 |1 is obsolete| | --- Comment #48 from David Nind <david@davidnind.com> --- Created attachment 177442 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177442&action=edit Bug 36027: (follow-up) Adding check-all option This patch add --check-all option, to run all the checks without specifying each one 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with 'check-all' arguments (ex: ./search_for_data_inconsistencies.pl --check-all), confirm that all the check is done. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|search_for_data_inconsisten |search_for_data_inconsisten |cies.pl - make each section |cies.pl - add options so |optional |you can choose what checks | |to run Text to go in the|This enhancement updates |This enhancement updates release notes|the data inconsistencies |the data inconsistencies |script |script |(misc/maintenance/search_fo |(misc/maintenance/search_fo |r_data_inconsistencies.pl) |r_data_inconsistencies.pl) |so that you can optionally |so that you can now choose |run specific checks, rather |what checks to run, instead |than running all the checks |of running all the checks |at the same time. |at once (which can take a | |lot of time for larger |Examples: |libraries). |- Run all checks | |(no change): |Running the |misc/maintenance/search_for |script without any options |_data_inconsistencies.pl |now displays the help text, |- |instead of running all the |Run checks for item type |checks (for consistency |issues and patrons with an |with other scripts). To run |invalid age for their |all the checks at the same |category: |time, use the --check-all |misc/maintenance/search_for |option. |_data_inconsistencies.pl | |--check-status --check-age |Examples: |- Skip specific checks, but |- Run |run all the others: |all checks (--check-all): |misc/maintenance/search_for |misc/maintenance/search_for |_data_inconsistencies.pl |_data_inconsistencies.pl |--skip-status --skip-title |--check-all | |- Run checks |Use |for item type issues and |misc/maintenance/search_for |patrons with an invalid age |_data_inconsistencies.pl |for their patron category: |--help for all the check |misc/maintenance/search_for |and skip options. |_data_inconsistencies.pl | |--check-status --check-age | |- Skip specific checks, but | |run all the others: | |misc/maintenance/search_for | |_data_inconsistencies.pl | |--skip-status --skip-title | | | |Use | |misc/maintenance/search_for | |_data_inconsistencies.pl | |--help for all the check | |and skip options. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #51 from David Nind <david@davidnind.com> --- Updated testing notes (from comment #25 and #36) ===================== Testing notes (using KTD): 1. Apply the patch. 2. Set up data so that there are inconsistencies: 2.1 There is already one inconsistency in the sample data, for record 369 (encoding errors) 2.2 I created an inconsistency for patron age and patron category type: . koha-mysql kohadev . update borrowers set categorycode = "K" where borrowernumber=49; 3. Run the script without any options: misc/maintenance/search_for_data_inconsistencies.pl ==> It should output the help for the script (a change in behavour - previously it ran all checks). 3. Run the script with one or more check options: misc/maintenance/search_for_data_inconsistencies.pl --check-status --check-age ==> It should output the inconsistencies (an invalid MARCXML record and an invalid age category). 4. Run the script with one or more --skip* options: misc/maintenance/search_for_data_inconsistencies.pl --skip-status ==> It should skip the checks specified. 6. Run the script with a check option and a skip option: misc/maintenance/search_for_data_inconsistencies.pl --check-age --skip-status ==> The skip option should be ignored (not run) and only the check option should be run. 7. Run the script with all options: misc/maintenance/search_for_data_inconsistencies.pl --check-all ==> It should output the results from running all the checks (in this case with KTD, the invalid MARCXML document and invalid age category). 8. Check the help to make sure it is improved, makes sense, and incorporates all the changes made by this bug: misc/maintenance/search_for_data_inconsistencies.pl --help Updated comments (from me, comment #25 and #36) =============================================== I'll file separate bugs for these comments when the bug passes QA: 1. The check-status option includes checking for item type issues (as described in the help description), but it also checks for invalid MARCXML. I think that the MARCXML check should have its own option (--check-marcxml Check for invalid MARCXML in bibliographic records). 1.1 I'm not sure how hard this to do (I am not a developer). 1.2 I'm not sure whether there are other checks in the script that are not included in the description and have their own options. 1.2 Instead of --check-status, --check-itemtype 2. Instead of --check-branch, use --check-library. While branch is used in the underlying code, the terminology guidelines suggest we should use library. (https://wiki.koha-community.org/wiki/Terminology#L) Follow-up - help and error message text ======================================= I've added a follow-up for: 1. Warning when you use --check-* and --skip-* options in the same command (step 6). The current text says (to me, anyway) that if you use skip and check options in the same command, the skip options are ignored. That is, it will ignore/not run the skip options: . Current warning: "Warning : skip options are ignored when check options are provided" . Suggested warning: "Warning: The --skip-* option(s) were ignored. Only the --check-* options in the command were run." 2. Description and help text - some formatting and text changes. Additional comments (from Katrin, comment #28) ============================================== Just so that these don't get lost. 1. New parameters (not blocker) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I wonder if the chosen pattern is good as it requires that we add 2 new parameters for every check that we add to the script. We do have something similar in cleanup_database.pl for action logs and it seems more logical to me. --log-module Specify which action log modules to trim. Repeatable. --preserve-log Specify which action logs to exclude. Repeatable. --logs DAYS purge entries from action_logs older than DAYS days. Defaults to 180 days if no days specified. So instead of a unique parameter, you can use preserve-log with different codes. I am not sure what is better, just putting the question out here. (I think that would be better, for example, two options --check and --skip with a list of codes for each check available. David) 2. Script should not start without -c or any parameters... (not blocker) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is something for a separate bug. But we once said that scripts should output help when run without parameter and require a confirmation flag. As this can run quite a time, this still seems a good idea... (leaving this here for filing later) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #53 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 177850 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177850&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional We can specify which sections to search for data inconsistencies using check options, or which sections to skip using skip options. TEST PLAN: Make sure there are data inconsistencies present in at least two categories to validate the results of the command. 1. Apply the patch 2. Run the command without any argument and confirm that every inconsistences are reported. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #55 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 177852 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177852&action=edit Bug 36027: (follow-up) Fixed code duplication and add mandatory options An option is now mandatory for the script to lunch. 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #57 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 177854 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=177854&action=edit Bug 36027: (follow-up) Update help text and error message This updates the help text and the error message when --check* and --skip* options are used in the same command. Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|alexandre.noel@inlibro.com |hammat.wele@inlibro.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Eric Bégin <eric.begin@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177849|0 |1 is obsolete| | --- Comment #60 from Eric Bégin <eric.begin@inLibro.com> --- Created attachment 180257 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=180257&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Eric Bégin <eric.begin@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177851|0 |1 is obsolete| | --- Comment #62 from Eric Bégin <eric.begin@inLibro.com> --- Created attachment 180259 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=180259&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Eric Bégin <eric.begin@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #177853|0 |1 is obsolete| | --- Comment #64 from Eric Bégin <eric.begin@inLibro.com> --- Created attachment 180261 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=180261&action=edit Bug 36027: (follow-up) Adding check-all option This patch add --check-all option, to run all the checks without specifying each one 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with 'check-all' arguments (ex: ./search_for_data_inconsistencies.pl --check-all), confirm that all the check is done. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Eric Bégin <eric.begin@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |eric.begin@inLibro.com --- Comment #66 from Eric Bégin <eric.begin@inLibro.com> --- I test David's case and you get the help info since having --skip* does nothing if you don't use the --check-all option. When you wrote the test plan, that script was doing the equivalent of --check-all when no option was set. That is not the case anymore. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Hammat wele <hammat.wele@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #180257|0 |1 is obsolete| | --- Comment #68 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 187299 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187299&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Hammat wele <hammat.wele@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #180259|0 |1 is obsolete| | --- Comment #70 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 187301 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187301&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Hammat wele <hammat.wele@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #180261|0 |1 is obsolete| | --- Comment #72 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 187303 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187303&action=edit Bug 36027: (follow-up) Adding check-all option This patch add --check-all option, to run all the checks without specifying each one 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with 'check-all' arguments (ex: ./search_for_data_inconsistencies.pl --check-all), confirm that all the check is done. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #74 from Hammat wele <hammat.wele@inlibro.com> --- Created attachment 187320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187320&action=edit Bug 36027: (follow-up) Fix the use of --skip-* options -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187300|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187302|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187304|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #76 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 199043 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199043&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #78 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 199045 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199045&action=edit Bug 36027: (follow-up) Tidy search_for_data_inconsistencies.pl Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #80 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 199047 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199047&action=edit Bug 36027: (follow-up) Adding check-all option This patch add --check-all option, to run all the checks without specifying each one 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with 'check-all' arguments (ex: ./search_for_data_inconsistencies.pl --check-all), confirm that all the check is done. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #82 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 199049 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199049&action=edit Bug 36027: (follow-up) Fix the use of --skip-* options Patch from commit 553a4da -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC|katrin.fischer@bsz-bw.de | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #83 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 202931 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202931&action=edit Bug 36027: Change interactive to pass param and tidy the code Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199044|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199046|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 Chris Mathevet <chris.mathevet@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199048|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #84 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 202932 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202932&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional We can specify which sections to search for data inconsistencies using check options, or which sections to skip using skip options. TEST PLAN: Make sure there are data inconsistencies present in at least two categories to validate the results of the command. 1. Apply the patch 2. Run the command without any argument and confirm that every inconsistences are reported. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #86 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 202934 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202934&action=edit Bug 36027: (follow-up) Fixed code duplication and add mandatory options An option is now mandatory for the script to lunch. 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #88 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 202936 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202936&action=edit Bug 36027: (follow-up) Update help text and error message This updates the help text and the error message when --check* and --skip* options are used in the same command. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #90 from Chris Mathevet <chris.mathevet@inLibro.com> --- Created attachment 202938 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202938&action=edit Bug 36027: (follow-up) removed help message misinformation Patch from commit bfa97db -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #92 from Hammat wele <hammat.wele@inlibro.com> --- (In reply to David Nind from comment #91)

I realise I'm being lazy, but can we get a consolidated test plan (since there are so many follow-ups now).

Thanks!

David

PS: Patches still apply.

PPS: I obsoleted some older patches.

Thanks David. Here is a consolidated test plan. Test plan Make sure there are data inconsistencies in at least two different categories to validate the results of the command. Using KTD: 1. There is already an invalid MARCXML record in the sample data (record 369). 2. Create an inconsistency for patron age/category: `./koha-mysql kohadev` `UPDATE borrowers SET categorycode = "K" WHERE borrowernumber = 49;` 3. Apply the patches 4. Run the script without any option: `./misc/maintenance/search_for_data_inconsistencies.pl` ==> The script should not run any checks. ==> The help information should be displayed. 5. Run the script with specific checks with `--check-*`, for example `./misc/maintenance/search_for_data_inconsistencies.pl --check-status --check-age` ==> Only the checks specified with `--check-*` should be run. ==> Other checks should not be run. 6. Run the script with a single `--skip-*` option: `./misc/maintenance/search_for_data_inconsistencies.pl --skip-age` ==> All checks except the age check should be run. ==> Other inconsistencies, such as the invalid MARCXML record, should still be reported. 7. Run the script with multiple `--skip-*` options:`./misc/maintenance/search_for_data_inconsistencies.pl --skip-age --skip-status` ==> All checks except the checks specified with `--skip-*` should be run. ==> The age and status checks should not be run. 8. Run the script with `--check-all` option: `./misc/maintenance/search_for_data_inconsistencies.pl --check-all` ==> All available checks should be run. 9. Combine `--check-*` and `--skip-*`: `./misc/maintenance/search_for_data_inconsistencies.pl --check-age --skip-status` ==> A warning should indicate that the `--skip-*` option(s) are ignored. ==> Only the check specified by `--check-age` should be run. 10. Combine `--check-all` and `--skip-*`:`./misc/maintenance/search_for_data_inconsistencies.pl --check-all --skip-status` ==> A warning should indicate that the `--skip-*` option(s) are ignored. ==> All checks should be run. 11. Run the script with `--help` option :`./misc/maintenance/search_for_data_inconsistencies.pl --help` ==> The help message should be displayed. 12. Verify each individual `--check-*` option. For each available `--check-*` option, run it individually, for example: ./misc/maintenance/search_for_data_inconsistencies.pl --check-age ./misc/maintenance/search_for_data_inconsistencies.pl --check-status ==> Each command should run only its corresponding check. ==> Only inconsistencies detected by that check should be reported. 13. Verify each individual `--skip-*` option. For each available `--skip-*` option, run it individually, for example: ./misc/maintenance/search_for_data_inconsistencies.pl --skip-status ./misc/maintenance/search_for_data_inconsistencies.pl --skip-age ==> All checks except the specified one should be executed. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #202931|0 |1 is obsolete| | Attachment #202932|0 |1 is obsolete| | Attachment #202933|0 |1 is obsolete| | Attachment #202936|0 |1 is obsolete| | Attachment #202937|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #94 from David Nind <david@davidnind.com> --- Created attachment 205098 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205098&action=edit Bug 36027: search_for_data_inconsistencies.pl - make each section optional We can specify which sections to search for data inconsistencies using check options, or which sections to skip using skip options. TEST PLAN: Make sure there are data inconsistencies present in at least two categories to validate the results of the command. 1. Apply the patch 2. Run the command without any argument and confirm that every inconsistences are reported. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #96 from David Nind <david@davidnind.com> --- Created attachment 205100 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205100&action=edit Bug 36027: (follow-up) Fix code duplication and add mandatory options An option is now mandatory for the script to lunch. 1. Apply the patch 2. Run the command without any argument and confirm it blocks you and shows the help message. 3. Run the command with one or more check arguments (ex: ./search_for_data_inconsistencies.pl --check-title --check-loop), confirm that it displayed the specified inconsistences. 4. Run the command with one or more skip arguments (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title), confirm that it displayed the inconsistences unsless the skip ones. 5. Run the command with check and skip options (ex: ./search_for_data_inconsistencies.pl --skip-auth --skip-title --check-loop), confirm that it displayed the specified check options and that a warning message indicate that the skip options have been ignored. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #98 from David Nind <david@davidnind.com> --- Created attachment 205102 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205102&action=edit Bug 36027: (follow-up) Update help text and error message This updates the help text and the error message when --check* and --skip* options are used in the same command. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Eric Bégin <eric.begin@inlibro.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #100 from David Nind <david@davidnind.com> --- Created attachment 205104 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=205104&action=edit Bug 36027: (follow-up) Remove help message misinformation Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #202935|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36027 --- Comment #101 from David Nind <david@davidnind.com> --- (In reply to Hammat wele from comment #92)

Thanks David. Here is a consolidated test plan.

Thanks Hammat! I've signed off based on the consolidated test plan. I also fixed some the of the patch subject lines to be a bit more consistent with the commit message guidelines (https://wiki.koha-community.org/wiki/Commit_messages) David -- You are receiving this mail because: You are watching all bug changes.