[Bug 42849] New: Retry failed Cypress specs in CI to reduce failures from flaky tests
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 Bug ID: 42849 Summary: Retry failed Cypress specs in CI to reduce failures from flaky tests Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: kyle@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Target Milestone: --- When we run cypress tests, they sometimes fail but pass when re-run. This appears to simply be a timing issue where the reponse is too slow. Allowing for a couple retries should previous spurious failures! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |minor Assignee|koha-bugs@lists.koha-commun |kyle@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 --- Comment #1 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 200481 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200481&action=edit Bug 42849: Retry failed Cypress specs in CI to reduce failures from flaky tests When we run cypress tests, they sometimes fail but pass when re-run. This appears to simply be a timing issue where the response is too slow. Allowing for a couple retries should previous spurious failures! Test plan: 1) Run cypress tests 2) Tests (should) pass 3) Apply this patch 4) Run tests again 5) Note they (still) pass! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #200481|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 200713 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200713&action=edit Bug 42849: Retry failed Cypress specs in CI to reduce failures from flaky tests When we run cypress tests, they sometimes fail but pass when re-run. This appears to simply be a timing issue where the response is too slow. Allowing for a couple retries should previous spurious failures! Test plan: 1) Run cypress tests 2) Tests (should) pass 3) Apply this patch 4) Run tests again 5) Note they (still) pass! Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #3 from David Nind <david@davidnind.com> --- bookingsModalDatePicker_spec.ts fails before and after the patch. With the patch, it attempts the test 3 times before moving onto the next test. The failing test is fixed by Bug 42862. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com --- Comment #4 from Jonathan Druart <jonathan.druart@gmail.com> --- Which flaky tests are we trying to fix? Usually the tests that fail are not correctly written. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42849 --- Comment #5 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #4)
Which flaky tests are we trying to fix? Usually the tests that fail are not correctly written.
Good question. This was my first fix attempt for bookingsModalDatePicker_spec.ts, with the solution being bug 42862. I think this is still a good patch which addresses issues related to the hardware the tests are being run on rather than our code, but it doesn't address a specific failure at this time. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org