Le 27/01/2011 17:49, Ian Walls a écrit :
I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges. i'm very happy to read that we agree on the problem (congrats you've summarized in 3 lines what I needed 50+ to express ;-) ) So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years. That's where we disagree, but if we agree on the problem, if we all put our hands in it, we will find a solution that appears to all to be the good one ! I think a better solution is make it easier to test and signoff on work. There are several components to this:
* Testing plans for larger developments/bugfixes * A robust testing data set made readily available * Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.
Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?". well, in my opinion, the one that does this testing plan must not be the one who wrote the feature. Because, of course, the feature has been tested, and the one who tested will have missed a use-case, or forgotten something,... ("given enough eyes, all bugs will be found"). So we're back to the question: who can dedicate time? Maybe we're back to the question of someone being (collectively) paid just for this role? As of today, I see that no-one find time to do it ! The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail.
To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan. Do you really think? I don't think so: If that were the case, then someone should/would have said "well seems interesting, but I need more information/help/..." We had setup sandbox for all our branches, organized a meeting where I were alone (only kf jumped-in to say "sorry, I can't be here"). I was so disappointed that ... I forgot I had planned a 2nd meeting, a few days later, was not here as expected, and no-one complained, so I concluded no-one has been attending as well. I ask and will continue to ask on #irc when someone passes around, but I feel like someone crying in the desert (frenchism suspected here)
So, I fully agree that the workflow is theorically a good one. But we lack the ressource to make it happen. I'm very scared about that, but i'd like to see things going on anyway.
I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08