https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #2 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182309 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182309&action=edit Bug 39877: Remove files that have been deleted Ignore deleted or renamed files -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #4 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182311 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182311&action=edit Bug 39877: Apply to other slow tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #6 from Jonathan Druart <jonathan.druart@gmail.com> --- An important note I forgot in the commit message. You can use the incremental run without the report (the report will be only for Jenkins). So it will help RM as xt tests will run (way) faster. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #182308|0 |1 is obsolete| | Attachment #182309|0 |1 is obsolete| | Attachment #182310|0 |1 is obsolete| | Attachment #182311|0 |1 is obsolete| | --- Comment #8 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182347 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182347&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #10 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182349 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182349&action=edit Bug 39877: Merge vue and js tidy tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #182347|0 |1 is obsolete| | --- Comment #12 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182355 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182355&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #182349|0 |1 is obsolete| | --- Comment #14 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 182357 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=182357&action=edit Bug 39877: Merge vue and js tidy tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #16 from Julian Maurice <julian.maurice@biblibre.com> ---

xt/pl_valid.t 2502354 ms We can also save some time by not checking *.t files, since they are executed anyway. *.t files represent approximately 25% of the total time of pl_valid.t for me.

(In reply to Jonathan Druart from comment #7)

Could also happen with pl_valid I guess if an external lib introduces changes that breaks our code. But they really are edge cases we cannot avoid and that should be caught by the qa script anyway. If an exported subroutine is removed/renamed, code that still use/import it will fail to compile, but this won't get caught if we are only checking modified files. Will it be caught by the QA script ?

-- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #182355|0 |1 is obsolete| | --- Comment #18 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 183795 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183795&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #182357|0 |1 is obsolete| | --- Comment #20 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 183797 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183797&action=edit Bug 39877: Merge vue and js tidy tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net --- Comment #22 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Rebased and fixed conflicts. Double-checked with a diff between the old and new patches to confirm there are no unexpected changes introduced. I'll attempt signing off. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #24 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Victor Grousset/tuxayo from comment #23)

(In reply to Jonathan Druart from comment #5)

...

Maybe later we will also use it for stable branches.

Which could have their own branch in koha-ci-results right? So it shouldn't cause race conditions.

We should not need separate branches. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #25 from Victor Grousset/tuxayo <victor@tuxayo.net> --- It seems the initial clone expects an https:// prefix. kohadev-koha@kohadevbox:koha(testing-SO-QA)$ KOHA_CI_INCREMENTAL_RUN_REPO_URL=gitlab.com/tuxayo/koha-ci-results.git KOHA_CI_INCREMENTAL_RUNS=1 KOHA_CI_INCREMENTAL_RUNS_REPORT=1 KOHA_CI_INCREMENTAL_RUNS_TOKEN=XXXXXX prove xt/perltidy.t xt/perltidy.t .. fatal: repository 'gitlab.com/tuxayo/koha-ci-results.git' does not exist fatal: cannot change to '/tmp/koha-ci-results': No such file or directory xt/perltidy.t .. 3/3031 ^C (cleanup) So it goes further with it. But then (the push?) ends up with https://https:// kohadev-koha@kohadevbox:koha(testing-SO-QA)$ KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/tuxayo/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=XXXXXXXXXXX \ prove xt/perltidy.t xt/perltidy.t .. Cloning into '/tmp/koha-ci-results'... xt/perltidy.t .. 3007/3031 fatal: unable to access 'https://https://gitlab.com/tuxayo/koha-ci-results.git/': Could not resolve host: https xt/perltidy.t .. ok All tests successful. Files=1, Tests=3031, 216 wallclock secs ( 0.41 usr 0.04 sys + 207.40 cusr 1.50 csys = 209.35 CPU) Result: PASS -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #27 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 184463 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184463&action=edit Bug 39877: Remove files that have been deleted Ignore deleted or renamed files -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #29 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 184465 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184465&action=edit Bug 39877: Apply to other slow tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #31 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Victor Grousset/tuxayo from comment #25)

It seems the initial clone expects an https:// prefix.

Indeed, I mixed up things it seems. Should be fixed now. I've amended the commit message of the first patch and added a fix ("Fix confusion with URL prefix") -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184463|0 |1 is obsolete| | --- Comment #33 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 184780 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184780&action=edit Bug 39877: Remove files that have been deleted Ignore deleted or renamed files -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184465|0 |1 is obsolete| | --- Comment #35 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 184782 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184782&action=edit Bug 39877: Apply to other slow tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #37 from Victor Grousset/tuxayo <victor@tuxayo.net> ---

* Create an empty project on gitlab eg. https://gitlab.com/me/koha-ci-results

Let it create a readme, don't untick the readme creation. Otherwise, there will be no branch to unprotect and there won't be an actual repo. (clone will fail) ---- rebased and fixed conflicts (diffs compared to confirm no unwanted change introduced) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #39 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 184785 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184785&action=edit Bug 39877: Prevent files to be processed twice If the file had a failure and still has, we should not list it twice. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184779|0 |1 is obsolete| | --- Comment #41 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 184875 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184875&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab (keep the default readme) eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184781|0 |1 is obsolete| | --- Comment #43 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 184877 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184877&action=edit Bug 39877: Merge vue and js tidy tests Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184783|0 |1 is obsolete| | --- Comment #45 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 184879 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184879&action=edit Bug 39877: Fix confusion with URL prefix git clone needs https://, but we need to remove it to build the push url Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #47 from Victor Grousset/tuxayo <victor@tuxayo.net> ---

...

Let it create a readme, don't untick the readme creation. Otherwise, there will be no branch to unprotect and there won't be an actual repo. (clone will fail)

I will deal with the setup once this is pushed to prevent such problems.

oops, I forgot to say it was a testing note and that I amended the test plan. Anyway, it works! It super cool, thanks! :D -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #48 from Victor Grousset/tuxayo <victor@tuxayo.net> --- @QA: Redirecting `git log --pretty=format:"%h"` in a file takes 0.2 sec for the 58000 commits of the koha repo, on my slow laptop. And it's already large and will take like 15 years to double. The ls in koha-ci-results/$test_name shouldn't become slow with time. Cloning the repo from scratch might eventually be a bit slow. But then the same approach could be done as for cloning the Koha repo. Is there anything else that could become slow as time passes and the history of tested commits grows? Nothing else found on my side. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com QA Contact|testopia@bugs.koha-communit |kyle@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #51 from Jonathan Druart <jonathan.druart@gmail.com> --- Hi Kyle, Make sure you are using the test plan from the last version of the patches KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t You are missing "https://" in the REPO_URL. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184875|0 |1 is obsolete| | --- Comment #52 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Created attachment 186320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186320&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab (keep the default readme) eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184877|0 |1 is obsolete| | --- Comment #54 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Created attachment 186322 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186322&action=edit Bug 39877: Merge vue and js tidy tests Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #184879|0 |1 is obsolete| | --- Comment #56 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Created attachment 186324 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186324&action=edit Bug 39877: Fix confusion with URL prefix git clone needs https://, but we need to remove it to build the push url Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |paul.derscheid@lmscloud.de --- Comment #58 from Paul Derscheid <paul.derscheid@lmscloud.de> --- This is awesome! I would pass QA, but we need more eyes on this. For the next QA reviewer, you can skip running the tests in the QA script. They all pass. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186320|0 |1 is obsolete| | --- Comment #60 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 186327 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186327&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab (keep the default readme) eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: John Doe <you@example.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186322|0 |1 is obsolete| | --- Comment #62 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 186329 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186329&action=edit Bug 39877: Merge vue and js tidy tests Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: John Doe <you@example.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186324|0 |1 is obsolete| | --- Comment #64 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 186331 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186331&action=edit Bug 39877: Fix confusion with URL prefix git clone needs https://, but we need to remove it to build the push url Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: John Doe <you@example.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #66 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Comment on attachment 186320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186320 Bug 39877: Incremental test runs Review of attachment 186320: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=39877&attachment=186320) ----------------------------------------------------------------- Other than this one line the only question I have is should we be using something like Git.pm for git actions instead of system commands? Perl has to spawn a shell for each git command, but it's only a handful of commands at startup so I imagine the difference wouldn't be very noticeable if at all. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #68 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Hmm, the graceful handling of an error in git dir init makes sense. We should probably abort then so that we don't overlook any problems that this might mask if we just initialized? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #70 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Kyle M Hall (khall) from comment #66)

Comment on attachment 186320 [details] [review] Bug 39877: Incremental test runs

Review of attachment 186320 [details] [review]: -----------------------------------------------------------------

Other than this one line the only question I have is should we be using something like Git.pm for git actions instead of system commands? Perl has to spawn a shell for each git command, but it's only a handful of commands at startup so I imagine the difference wouldn't be very noticeable if at all.

Hum, I have initially started with Git::Wrapper (that we already use in the auto_rebase.pl script), but I think I found a blocker and decided to go with straight qx calls. But to be honest I don't remember what was the problem. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #72 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 186363 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186363&action=edit Bug 39877: Remove files that have been deleted Ignore deleted or renamed files Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #74 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 186365 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186365&action=edit Bug 39877: Apply to other slow tests Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #76 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 186367 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186367&action=edit Bug 39877: Prevent files to be processed twice If the file had a failure and still has, we should not list it twice. Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #78 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Kyle M Hall (khall) from comment #65)

Signed-off-by: John Doe <you@example.com>

replaced with Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> And added a QA check https://gitlab.com/koha-community/qa-test-tools/-/issues/97 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #80 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Kyle M Hall (khall) from comment #67)

...

Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches.

Should each running specify a name, and maybe a branch? So koha-ci-results/perltidy would be koha-ci-results/$branch/$runner/perltidy/$rev

Yes, we need to adjust that now, before this gets pushed. Actually it is a problem and we want to catch stuff like bug 40680 (xt/pl_valid.t crashes on U24 only), and the codespell problem described in comment 7. However I don't think it should be $runner. Maybe we need $os: remember that we want RM and RMaints to run those slow tests before they push, and we don't want them to specify the jenkins's job name, we could pass the "OS code" automatically from ktd (D12, D13, U24).

Branch knowability would be a possible issue, `git branch -vv` lists the tracked branch iirc.

Regarding the branch I think we can go with a single "main" branch for now, and list all commit ids. It's a problem we can fix later if it needs to be fixed (and we will create branches on koha-ci-results). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #82 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Jonathan Druart from comment #81)

Created attachment 186370 [details] [review] Bug 39877: Use KOHA_IMAGE to separate OSes

We could do this. Would that be enough? KOHA_IMAGE is exported by jenkins, then pass to the container. It's available in the root env var list, not for the kohadev user (more tweak needed here, but it would work for jenkins as a first step). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #84 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 186622 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=186622&action=edit Bug 39877: Use OS codename to separate OSes -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #86 from Jonathan Druart <jonathan.druart@gmail.com> --- (Tomas is going to install lsb-release in ktd) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #87 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Jonathan Druart from comment #84)

Created attachment 186622 [details] [review] Bug 39877: Use OS codename to separate OSes

So we don't need to add a new variable in ktd, or to pass from the jenkins job. And everything is seamless for the users. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #186362|0 |1 is obsolete| | Attachment #186363|0 |1 is obsolete| | Attachment #186364|0 |1 is obsolete| | Attachment #186365|0 |1 is obsolete| | Attachment #186366|0 |1 is obsolete| | Attachment #186367|0 |1 is obsolete| | Attachment #186368|0 |1 is obsolete| | Attachment #186370|0 |1 is obsolete| | Attachment #186622|0 |1 is obsolete| | --- Comment #89 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187387 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187387&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab (keep the default readme) eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #91 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187389 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187389&action=edit Bug 39877: Merge vue and js tidy tests Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #93 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187391 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187391&action=edit Bug 39877: Fix confusion with URL prefix git clone needs https://, but we need to remove it to build the push url Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #95 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187393 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187393&action=edit Bug 39877: Do not clone if env var is not set If we don't want the incremental run we should return early in ->new to prevent the git clone. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187394|0 |1 is obsolete| | --- Comment #97 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187395 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187395&action=edit Bug 39877: Use OS codename to separate OSes -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #187387|0 |1 is obsolete| | Attachment #187388|0 |1 is obsolete| | Attachment #187389|0 |1 is obsolete| | Attachment #187390|0 |1 is obsolete| | Attachment #187391|0 |1 is obsolete| | Attachment #187392|0 |1 is obsolete| | Attachment #187393|0 |1 is obsolete| | Attachment #187395|0 |1 is obsolete| | --- Comment #98 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 187399 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187399&action=edit Bug 39877: Incremental test runs The idea of this patchset is to allow increment runs for some of our tests. The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15). One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).

From a previous run (last week) I noted this top 5:

xt/vue_tidy.t 105446 ms xt/perltidy.t 175970 ms xt/js_tidy.t 239615 ms xt/author/codespell.t 467041 ms xt/pl_valid.t 2502354 ms We are going to merge js_tidy.t and vue_tidy.t as it does no longer make sense to have a difference between them (we had at the beginning of the vue work, because of differences in the options, now everything is handled in the .prettierrc config file) Once this is pushed, we will be able to create a new project on gitlab and pass some env variables from Jenkins. Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results When everything will be setup, jenkins will run the tests one the whole test suite, then the next run will be "incremental": we are going to run the tests on the files that were failing and the files modified by the last push. Example: * "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm * koha-ci-results/perltidy/abc is created and contains ['Koha.pm'] * "commit def" has a modification on about.pl * xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy * koha-ci-results/perltidy/def is created and contains [] Test plan: * Create an empty project on gitlab (keep the default readme) eg. https://gitlab.com/me/koha-ci-results * Generate an access token with the developer role (https://gitlab.com/me/koha-ci-results/-/settings/access_tokens) Copy the token (in the following noted as TOKEN_REPLACE_ME) * Unprotect the "main" branch * Run the following prove command: ``` KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \ KOHA_CI_INCREMENTAL_RUNS=1 \ KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \ KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \ prove xt/perltidy.t ``` => a new commit is created on your repo * Add some mess in one of your perl file, commit using `--no-verify` to bypass the git hook * Run the prove command again => only the file you have modified is tested => a new commit is created and the file is reported as failing * Add another commit, fix the failure, add other mess => confirm that what you are seeing makes sense :) * Ideally redo with the different tests modified by this patchset * Note that you can use the incremental run without the report (the report will be only for Jenkins). It will help RM as xt tests will run (way) faster. Note that you can also export the following env vars to set git author detail: * GIT_COMMITTER_NAME * GIT_COMMITTER_EMAIL * GIT_AUTHOR_NAME * GIT_AUTHOR_EMAIL Question: What are we going to use on Jenkins? Discussion for QA: How should we handle concurrency? It's crucial to prevent multiple tests from pushing simultaneously. To minimize the risk of the repository becoming outdated, we might need to perform a pull right before each push. Is that enough? Keep in mind that only one job will be run at the same time (Koha_Main). Maybe later we will also use it for stable branches. Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #100 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 187401 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187401&action=edit Bug 39877: Merge vue and js tidy tests Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #102 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 187403 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187403&action=edit Bug 39877: Fix confusion with URL prefix git clone needs https://, but we need to remove it to build the push url Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #104 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 187405 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187405&action=edit Bug 39877: Do not clone if env var is not set If we don't want the incremental run we should return early in ->new to prevent the git clone. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #106 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 187407 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187407&action=edit Bug 39877: Tidy POD Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Text to go in the| |25.11.00 release notes| | --- Comment #108 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Pushed to main for 25.11.00. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #110 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 187977 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=187977&action=edit Bug 39877: Force commit id length to 10 The size of the commit id can vary, depending on the git repo. For instance Jenkins had stored results using length of 9 (e9a07fe6a) because it's a shallow clone. But 9 is not enough for our repo: % git rev-list --all | sort | uniq -w 9 -D 29dfcf49f28bb8e75ab4b71a2d68f194928d3de9 29dfcf49f68a2718c0d217921678cd1a0be38ab5 We need to force the length for both commands we use, so commits matched during the comparison. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 --- Comment #111 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- follow-up pushed to main -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41274 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41274 [Bug 41274] Incremental test runs not properly skipping Schema files -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39877 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz Status|Pushed to main |Needs documenting --- Comment #112 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Enhancement - Not backporting to 25.05 -- You are receiving this mail because: You are watching all bug changes.