[Koha-devel] Good enough?

[Tomas Cohen Arazi]

El lun, 5 dic 2022 a las 14:24, Julian Maurice (<julian.maurice at biblibre.com>)
escribió:

> Yes it is expensive. But manual testing is much more expensive. I'd
> rather pay for some CPU time than do manually what a bot can do better
> and faster than me. Free services exist too (CircleCI, github actions, ...)
>

When you develop, you usually add some tests and run them, along with
related ones that could be broken because of the changes. Running the full
test suite is not mandatory.
Pushing to a branch somewhere there's CI setup to run the full test suite
sounds like a good idea.


>
> Pushing to a temporary master branch is not a bad idea (a never-broken
> master branch sounds nice), but I think it would happen too late.
> Patches would have already been rebased multiple times, tested and
> reviewed before we notice a test failure.
>

Agreed!


> koha-testing-docker feels more like a symptom of the difficulty to run
> tests than a satisfying solution. It is probably necessary in order to
> run complicated end-to-end tests, but it should not be mandatory to run
> simple unit tests.
> And ktd is not that easy to use. It can break, and it's not easy to
> debug for someone not familiar with docker.
>

I don't think it is mandatory, but it is the most convenient way to run in
a controlled environment that matches as much as possible the conditions in
production. That said, I'm pretty sure we could rely on carton or
local::lib to generate an environment to work locally.


> Don't you think it would be a lot easier if we could run `prove t` (or
> `npm test` or whatever) on any dev setup and have the same
> failures/successes as Jenkins (minus the complicated tests like those
> that require selenium) ?
> Of course making that happen is not an easy task, but it should be a
> long term goal IMO.
>
> Also I think no one wants to be the person that refuse a patch because a
> comment is misaligned, so if that kind of thing is not automated, "not
> good enough code" will continue to be pushed.
>
>
> Le 05/12/2022 à 14:55, Jonathan Druart a écrit :
> > I don't think we should run the whole test suite everytime we attach
> > patches, that would be very expensive in terms of resources.
> > However it would be interesting to have a temporary 'master' branch
> > that would become 'master' only if jenkins is happy.
> > "master" would never be broken :D
>

That would only save us some reverts if the authors fail to provide fixes.
But I don't think reverts happen so often we need  to change our workflow
to avoid them. And jenkins to fail is the goal of it. We don't need it
green all day, we need it to catch the errors (lots of errors are not
covered by tests, and that's something we need to invest time on) and devs
to take responsibility of fixing their breakages. I will take the blame
there, because Jenkins notifications are not reaching the devs inboxes, and
we need it fixed ASAP.

-- 
Tomás Cohen Arazi
Theke Solutions (http://theke.io)
✆ +54 9351 3513384
GPG: B2F3C15F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20221205/457361ea/attachment.htm>