[Koha-devel] Development workflow

Mon Jan 31 09:11:16 CET 2011

Just my late response on this thread:

1)    If I include a test plan but should forget steps/test areas etc., and a non-developer tests the patch: aren’t we decreasing the value of signoff? In that case we could still end up with not signing patches in the long run.. Still, something is perhaps better than nothing at all.

2)    A patch should not be too big.

3)    Much depends on how the QA role is filled in [Ian’s plans sounds good]. If the QA could give more guidance on which patches should be tested next, testers are not just cherry picking patches.. Why should QA not mail once a week or so the first 10 or 20 patches that he wants to be signed? Could prevent older/larger/less interesting sounding patches waiting and newer (smaller/..) patches getting signed and pushed. Would make the process even more ‘fair’ to everyone.

Marcel

Van: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel-bounces at lists.koha-community.org] Namens Ian Walls
Verzonden: donderdag 27 januari 2011 17:50
Aan: Paul Poulain
CC: koha-devel at lists.koha-community.org
Onderwerp: Re: [Koha-devel] Development workflow

Paul,

I hear you.  There are lots of really great features that are in limbo, waiting to be tested.  The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things.  This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers.  As a result, lots of good stuff sits, waits, and diverges.

So what can be done?  I don't think that changing the signoff procedure will have the desired effect.  That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years.

I think a better solution is make it easier to test and signoff on work.  There are several components to this:

  *   Testing plans for larger developments/bugfixes
  *   A robust testing data set made readily available
  *   Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works.  It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.

Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?".

The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide.  Make it something that librarians can do by following a list of steps.  Lowering the threshold of experience required to test things will allow us to harness the Long Tail.

To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6.  My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above.  In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments.  Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan.

I would very much like to discuss this at the next IRC meeting.  It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :)

Cheers,

-Ian

On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain at biblibre.com<mailto:paul.poulain at biblibre.com>> wrote:
Hello koha-devel,

The more time passes, the more I think we have a workflow for Koha
development that does not work correctly/efficiently/fluently.
The rule we have is:
* create a branch for the feature
* file a bug for it
* submit your branch to koha-patches ML
* the RM checks that everything apply smoothly on master & ask for QA
* someone signs-off the branch/patch
* the RM merge the branch.

For the maintenance branch, this workflow is perfect and must be secured
more and more to avoid any regression. As I see it, there is always
someone that can find time to sign-off a patch or 2. As ppl have
system live with it i'm sure it's cricital for many ppl to check &
double check !

But for the next release/unstable branch this workflow does not work well.

Here is a single line of something owen said on the chat today:
<owen> paul_p: I would much prefer to be testing your stuff today, but
other stuff is taking precedence.

God knows i'll be forever thankfull to owen for the work he does. But this
sentence reveals something: for most of us, there is always "other stuff
taking precedence" :\

BibLibre has submitted many new features 3 months ago. I tried to
motivate ppl to get feedback & sign-off. I even have organised an IRC
meeting ... where no one came. Those patches includes
new features, and requires a lot of work to be tested.
2 weeks ago, I did the same for a bunch of small improvements (and I
have many many more pending). Once again, no feedback, except for one
very easy-to-test (new images) : signed-off by Nicole & pushed by chris
in less than 24H!
Those 2 minor events (with owen & nicole) made me realize that it's technically and
functionnaly hard & long to test: rebase, drop database, install, file
test cases, check the new features,... At KohaCon, liz tested some of
those features and said "wow, great, I want this feature for my library
!" (I don't remember what it was exactly). The problem is that liz is
not a techie woman, and can't test patches by her own. For our images,
it was easy for Nicole to test, no need to updatedatabase or anything
complex, just reach the itemtypes.pl<http://itemtypes.pl> page check the images are here.
Easy for a librarian !

Other point: dependancies between new features. Let me take an example:
Templates on those devs relies on html::template. Chris is working on
moving everything to Template::Toolkit.
Once he consider it's ready, he should submit for QA, as for every new features.
Suppose no-one sign-off.
And suppose someone sign-off, there are 2 options:
* move master to T::T when it's signed-off. What happends with other branches
not merged ? they can't be merged anymore, they have to be rewritten.
wow, I can't imagine that something we've submitted for QA 3 months ago
has to be rewritten because no-one took care of it !
* wait until those branches are merged. Two major problems here: the
release date won't be respected, and we have switched again to feature
based release. Or 3.4 will be released on time, but won't contain many
things all those great features that are live in all our libraries since
months. The second problem is that other devs are on the way. They still
rely on H::T as master is based on that. Switching to T::T will cause
problems for those branches later.
well, I think that none of those solutions is a correct one.

I can take an other example: the analytics record feature may/will
probably overlap functionnaly with the new features we have submitted
for cataloguing (not sure, but i think so). Suppose our cataloguing
branch is integrated in 2 weeks. it has been submitted 3 months ago. It
means our indian colleagues will have to rebase & face problems to get
their analytical record branch working. If our branch had been merged
quickly, they would have had 3 more months to deal with it.
Our actual workflow, causes a lot of overhead ! And -worst- I don't
think it improves the stability & security: there are merge problems
that are difficults to detect. It means that any test done on analytical
records would have to be done again if our cataloguing branch is merged
: more work, more pain. And the later a branch is merged, the smaller
the time to find a problem.

My main conclusion is that we are not a large enough community to deal
with testing/validating/merging new features in a short timeline. So I
think we must change the way we integrate new features. The general idea
being: remove the sign-off lock on the workflow.

Here is a proposal:
Let's go back to the timeline: we plan to release a version every 6 months.
We could have a window of, say 4 months. During those 4 months a feature
can be included if :
1- the submitter provides something that applies on master (ie: it's technically valid)
2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the branch is submitted)

It put more pressure on others to test quickly, and don't put pain on
parallels/parallelizing developments. Once a branch is merged if there
is a problem, then everybody should see it, and it will be fixed !

Then, for 1 month, any new feature can be applied only if a newly
submitted branch is approved/signed-off (less than 2 months to detect a
problem in a new feature is a must-have)
Then, feature freeze, 1 month debugging and translating.

I know it is a big change in the workflow, but the actual one is not
working well, so, we have to find another one.
I humbly request to have this question being put on the next irc agenda
(which does not mean you should not also start a thread here, of course)

PS: pls don't say i'm wrong and the workflow works well. Having branches submitted 3 months ago, getting no feedback, and planning to have a release in less than 3 months, can't be considered as something working well.

Friendly.

--
Paul POULAIN
http://www.biblibre.com
Expert en Logiciels Libres pour l'info-doc
Tel : (33) 4 91 81 35 08

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org<mailto:Koha-devel at lists.koha-community.org>
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

--
Ian Walls
Lead Development Specialist
ByWater Solutions
Phone # (888) 900-8944
http://bywatersolutions.com
ian.walls at bywatersolutions.com<mailto:ian.walls at bywatersolutions.com>
Twitter: @sekjal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20110131/cac834e3/attachment-0001.htm>