Hello, Thank you for your answer Chris ! So there are 3 kinds of failure causes: 1/ C4::XISBN::get_xisbns fails because the connection to an external data source cannot be established. 2/ data is not created by the unit tests and does not exist in the sample files. 3/ a test "normally" fails For 1, is it consistent to mock the connection ? For 2, I think it could be resolved using the idea from bug 10337: reset the DB data before launching tests. It would be great to receive mails from Jenkins saying the master is unstable only if is a code issue (and not a connection issue or a lack of data). Moreover, we have some unit test files which are never launched. So if a regression is introduced on routines covered by these unit tests, we are not alerted. The prove command should be launched with the recursive flag. Regards, Jonathan 2013/10/30 Chris Cormack <chrisc@catalyst.net.nz>:
* Jonathan Druart (jonathan.druart@biblibre.com) wrote:
Your question is certainly valid. There are still too many tests depending on some data to exist. Finding out why it no longer fails requires some (extensive?) digging into which patches were pushed in the meantime.
Jenkins is usually back to stable without any patch pushed. So an intervention is done on the Jenkins DB on adding/updating/removing data.
Not usually no, often the test is failing due to depending on external data sources. XISBN.t is the prime example, it can fail because it can't connect to the external data source.
The current failing test is Serials http://jenkins.koha-community.org/job/Koha_master/lastCompletedBuild/testRep...
01:10:42.908 # Failed test 'test getting history from sub-scription' 01:10:42.909 # at t/db_dependent/Serials.t line 59. 01:10:42.915 01:10:42.915 # Failed test 'Subscription has at least one serial' 01:10:42.916 # at t/db_dependent/Serials.t line 62.
This one will be to do with data, and changes it started failing http://jenkins.koha-community.org/job/Koha_master/1482/
When all the serials changes were pushed.
It fails on my local machine too Test Summary Report ------------------- t/db_dependent/Serials.t (Wstat: 768 Tests: 35 Failed: 3) Failed tests: 6, 8-9 Non-zero exit status: 3 Files=1, Tests=35, 0 wallclock secs ( 0.04 usr 0.00 sys + 0.43 cusr 0.02 csys = 0.49 CPU) Result: FAIL
Chris -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand