[Koha-devel] Development workflow

Thu Jan 27 17:49:53 CET 2011

Paul,

I hear you.  There are lots of really great features that are in limbo,
waiting to be tested.  The unfortunate situation of life is that we're all
incredibly busy, and it's hard to find time to test things.  This is
compounded when testing certain features requires coding or systems
experience, since that limits the pool of potential testers.  As a result,
lots of good stuff sits, waits, and diverges.

So what can be done?  I don't think that changing the signoff procedure will
have the desired effect.  That'll just let more unreviewed code slip in,
potentially introducing bugs we won't find for weeks/months/years.

I think a better solution is make it easier to test and signoff on work.
There are several components to this:

   - Testing plans for larger developments/bugfixes
   - A robust testing data set made readily available
   - Teaching people how to test and signoff on code

By including testing plans with developments or complex bugfixes, the
developer is communicating to everyone how they can prove their code works.
It lays out the intention of the development (it should do x, y and z), and
a series of tests to show how to get x, y and z without losing a, b and c.

Combine this testing plan (written in language librarians can understand,
not coder jargon) with the necessary data set to do the testing (an SQL dump
you just load into a DB), and you've lowered the barrier for testing so that
anyone who can afford a little time to run through a series of listed
procedures can answer the question "does this work?".

The third step to this is to lay out the procedure for running through the
test plan in a clear, simple manner, and distribute that information far and
wide.  Make it something that librarians can do by following a list of
steps.  Lowering the threshold of experience required to test things will
allow us to harness the Long Tail.

To this end, I'm throwing my hat in the ring for Quality Assurance Manager
for Koha 3.6.  My proposal can be found on the wiki (
http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much
of it is explained above.  In addition to this, I would also serve as a
coordinator for testing work submitted, and provide regular reports to the
community on the status of these developments.  Branches that are not
receiving active testing feedback would receive attention towards creating a
simpler, easier to follow testing plan.

I would very much like to discuss this at the next IRC meeting.  It'll be
pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand
:)

Cheers,

-Ian

On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain at biblibre.com>wrote:

> Hello koha-devel,
>
> The more time passes, the more I think we have a workflow for Koha
> development that does not work correctly/efficiently/fluently.
> The rule we have is:
> * create a branch for the feature
> * file a bug for it
> * submit your branch to koha-patches ML
> * the RM checks that everything apply smoothly on master & ask for QA
> * someone signs-off the branch/patch
> * the RM merge the branch.
>
> For the maintenance branch, this workflow is perfect and must be secured
> more and more to avoid any regression. As I see it, there is always
> someone that can find time to sign-off a patch or 2. As ppl have
> system live with it i'm sure it's cricital for many ppl to check &
> double check !
>
> But for the next release/unstable branch this workflow does not work well.
>
> Here is a single line of something owen said on the chat today:
> <owen> paul_p: I would much prefer to be testing your stuff today, but
> other stuff is taking precedence.
>
> God knows i'll be forever thankfull to owen for the work he does. But this
> sentence reveals something: for most of us, there is always "other stuff
> taking precedence" :\
>
> BibLibre has submitted many new features 3 months ago. I tried to
> motivate ppl to get feedback & sign-off. I even have organised an IRC
> meeting ... where no one came. Those patches includes
> new features, and requires a lot of work to be tested.
> 2 weeks ago, I did the same for a bunch of small improvements (and I
> have many many more pending). Once again, no feedback, except for one
> very easy-to-test (new images) : signed-off by Nicole & pushed by chris
> in less than 24H!
> Those 2 minor events (with owen & nicole) made me realize that it's
> technically and
> functionnaly hard & long to test: rebase, drop database, install, file
> test cases, check the new features,... At KohaCon, liz tested some of
> those features and said "wow, great, I want this feature for my library
> !" (I don't remember what it was exactly). The problem is that liz is
> not a techie woman, and can't test patches by her own. For our images,
> it was easy for Nicole to test, no need to updatedatabase or anything
> complex, just reach the itemtypes.pl page check the images are here.
> Easy for a librarian !
>
> Other point: dependancies between new features. Let me take an example:
> Templates on those devs relies on html::template. Chris is working on
> moving everything to Template::Toolkit.
> Once he consider it's ready, he should submit for QA, as for every new
> features.
> Suppose no-one sign-off.
> And suppose someone sign-off, there are 2 options:
> * move master to T::T when it's signed-off. What happends with other
> branches
> not merged ? they can't be merged anymore, they have to be rewritten.
> wow, I can't imagine that something we've submitted for QA 3 months ago
> has to be rewritten because no-one took care of it !
> * wait until those branches are merged. Two major problems here: the
> release date won't be respected, and we have switched again to feature
> based release. Or 3.4 will be released on time, but won't contain many
> things all those great features that are live in all our libraries since
> months. The second problem is that other devs are on the way. They still
> rely on H::T as master is based on that. Switching to T::T will cause
> problems for those branches later.
> well, I think that none of those solutions is a correct one.
>
> I can take an other example: the analytics record feature may/will
> probably overlap functionnaly with the new features we have submitted
> for cataloguing (not sure, but i think so). Suppose our cataloguing
> branch is integrated in 2 weeks. it has been submitted 3 months ago. It
> means our indian colleagues will have to rebase & face problems to get
> their analytical record branch working. If our branch had been merged
> quickly, they would have had 3 more months to deal with it.
> Our actual workflow, causes a lot of overhead ! And -worst- I don't
> think it improves the stability & security: there are merge problems
> that are difficults to detect. It means that any test done on analytical
> records would have to be done again if our cataloguing branch is merged
> : more work, more pain. And the later a branch is merged, the smaller
> the time to find a problem.
>
> My main conclusion is that we are not a large enough community to deal
> with testing/validating/merging new features in a short timeline. So I
> think we must change the way we integrate new features. The general idea
> being: remove the sign-off lock on the workflow.
>
> Here is a proposal:
> Let's go back to the timeline: we plan to release a version every 6 months.
> We could have a window of, say 4 months. During those 4 months a feature
> can be included if :
> 1- the submitter provides something that applies on master (ie: it's
> technically valid)
> 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the
> branch is submitted)
>
> It put more pressure on others to test quickly, and don't put pain on
> parallels/parallelizing developments. Once a branch is merged if there
> is a problem, then everybody should see it, and it will be fixed !
>
> Then, for 1 month, any new feature can be applied only if a newly
> submitted branch is approved/signed-off (less than 2 months to detect a
> problem in a new feature is a must-have)
> Then, feature freeze, 1 month debugging and translating.
>
> I know it is a big change in the workflow, but the actual one is not
> working well, so, we have to find another one.
> I humbly request to have this question being put on the next irc agenda
> (which does not mean you should not also start a thread here, of course)
>
> PS: pls don't say i'm wrong and the workflow works well. Having branches
> submitted 3 months ago, getting no feedback, and planning to have a release
> in less than 3 months, can't be considered as something working well.
>
> Friendly.
>
> --
> Paul POULAIN
> http://www.biblibre.com
> Expert en Logiciels Libres pour l'info-doc
> Tel : (33) 4 91 81 35 08
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>

-- 
Ian Walls
Lead Development Specialist
ByWater Solutions
Phone # (888) 900-8944
http://bywatersolutions.com
ian.walls at bywatersolutions.com
Twitter: @sekjal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20110127/3aca53d8/attachment-0001.htm>