[Koha-devel] Development workflow

Thu Jan 27 18:06:37 CET 2011

+1 to Ian's thoughts.

Kind Regards,
Chris


2011/1/27 Ian Walls <ian.walls at bywatersolutions.com>

> Paul,
>
>
> I hear you.  There are lots of really great features that are in limbo,
> waiting to be tested.  The unfortunate situation of life is that we're all
> incredibly busy, and it's hard to find time to test things.  This is
> compounded when testing certain features requires coding or systems
> experience, since that limits the pool of potential testers.  As a result,
> lots of good stuff sits, waits, and diverges.
>
> So what can be done?  I don't think that changing the signoff procedure
> will have the desired effect.  That'll just let more unreviewed code slip
> in, potentially introducing bugs we won't find for weeks/months/years.
>
> I think a better solution is make it easier to test and signoff on work.
> There are several components to this:
>
>    - Testing plans for larger developments/bugfixes
>    - A robust testing data set made readily available
>    - Teaching people how to test and signoff on code
>
> By including testing plans with developments or complex bugfixes, the
> developer is communicating to everyone how they can prove their code works.
> It lays out the intention of the development (it should do x, y and z), and
> a series of tests to show how to get x, y and z without losing a, b and c.
>
> Combine this testing plan (written in language librarians can understand,
> not coder jargon) with the necessary data set to do the testing (an SQL dump
> you just load into a DB), and you've lowered the barrier for testing so that
> anyone who can afford a little time to run through a series of listed
> procedures can answer the question "does this work?".
>
> The third step to this is to lay out the procedure for running through the
> test plan in a clear, simple manner, and distribute that information far and
> wide.  Make it something that librarians can do by following a list of
> steps.  Lowering the threshold of experience required to test things will
> allow us to harness the Long Tail.
>
> To this end, I'm throwing my hat in the ring for Quality Assurance Manager
> for Koha 3.6.  My proposal can be found on the wiki (
> http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much
> of it is explained above.  In addition to this, I would also serve as a
> coordinator for testing work submitted, and provide regular reports to the
> community on the status of these developments.  Branches that are not
> receiving active testing feedback would receive attention towards creating a
> simpler, easier to follow testing plan.
>
> I would very much like to discuss this at the next IRC meeting.  It'll be
> pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand
> :)
>
> Cheers,
>
>
> -Ian
>
>
>
>
> On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain at biblibre.com>wrote:
>
>> Hello koha-devel,
>>
>> The more time passes, the more I think we have a workflow for Koha
>> development that does not work correctly/efficiently/fluently.
>> The rule we have is:
>> * create a branch for the feature
>> * file a bug for it
>> * submit your branch to koha-patches ML
>> * the RM checks that everything apply smoothly on master & ask for QA
>> * someone signs-off the branch/patch
>> * the RM merge the branch.
>>
>> For the maintenance branch, this workflow is perfect and must be secured
>> more and more to avoid any regression. As I see it, there is always
>> someone that can find time to sign-off a patch or 2. As ppl have
>> system live with it i'm sure it's cricital for many ppl to check &
>> double check !
>>
>> But for the next release/unstable branch this workflow does not work well.
>>
>> Here is a single line of something owen said on the chat today:
>> <owen> paul_p: I would much prefer to be testing your stuff today, but
>> other stuff is taking precedence.
>>
>> God knows i'll be forever thankfull to owen for the work he does. But this
>> sentence reveals something: for most of us, there is always "other stuff
>> taking precedence" :\
>>
>> BibLibre has submitted many new features 3 months ago. I tried to
>> motivate ppl to get feedback & sign-off. I even have organised an IRC
>> meeting ... where no one came. Those patches includes
>> new features, and requires a lot of work to be tested.
>> 2 weeks ago, I did the same for a bunch of small improvements (and I
>> have many many more pending). Once again, no feedback, except for one
>> very easy-to-test (new images) : signed-off by Nicole & pushed by chris
>> in less than 24H!
>> Those 2 minor events (with owen & nicole) made me realize that it's
>> technically and
>> functionnaly hard & long to test: rebase, drop database, install, file
>> test cases, check the new features,... At KohaCon, liz tested some of
>> those features and said "wow, great, I want this feature for my library
>> !" (I don't remember what it was exactly). The problem is that liz is
>> not a techie woman, and can't test patches by her own. For our images,
>> it was easy for Nicole to test, no need to updatedatabase or anything
>> complex, just reach the itemtypes.pl page check the images are here.
>> Easy for a librarian !
>>
>> Other point: dependancies between new features. Let me take an example:
>> Templates on those devs relies on html::template. Chris is working on
>> moving everything to Template::Toolkit.
>> Once he consider it's ready, he should submit for QA, as for every new
>> features.
>> Suppose no-one sign-off.
>> And suppose someone sign-off, there are 2 options:
>> * move master to T::T when it's signed-off. What happends with other
>> branches
>> not merged ? they can't be merged anymore, they have to be rewritten.
>> wow, I can't imagine that something we've submitted for QA 3 months ago
>> has to be rewritten because no-one took care of it !
>> * wait until those branches are merged. Two major problems here: the
>> release date won't be respected, and we have switched again to feature
>> based release. Or 3.4 will be released on time, but won't contain many
>> things all those great features that are live in all our libraries since
>> months. The second problem is that other devs are on the way. They still
>> rely on H::T as master is based on that. Switching to T::T will cause
>> problems for those branches later.
>> well, I think that none of those solutions is a correct one.
>>
>> I can take an other example: the analytics record feature may/will
>> probably overlap functionnaly with the new features we have submitted
>> for cataloguing (not sure, but i think so). Suppose our cataloguing
>> branch is integrated in 2 weeks. it has been submitted 3 months ago. It
>> means our indian colleagues will have to rebase & face problems to get
>> their analytical record branch working. If our branch had been merged
>> quickly, they would have had 3 more months to deal with it.
>> Our actual workflow, causes a lot of overhead ! And -worst- I don't
>> think it improves the stability & security: there are merge problems
>> that are difficults to detect. It means that any test done on analytical
>> records would have to be done again if our cataloguing branch is merged
>> : more work, more pain. And the later a branch is merged, the smaller
>> the time to find a problem.
>>
>> My main conclusion is that we are not a large enough community to deal
>> with testing/validating/merging new features in a short timeline. So I
>> think we must change the way we integrate new features. The general idea
>> being: remove the sign-off lock on the workflow.
>>
>> Here is a proposal:
>> Let's go back to the timeline: we plan to release a version every 6
>> months.
>> We could have a window of, say 4 months. During those 4 months a feature
>> can be included if :
>> 1- the submitter provides something that applies on master (ie: it's
>> technically valid)
>> 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the
>> branch is submitted)
>>
>> It put more pressure on others to test quickly, and don't put pain on
>> parallels/parallelizing developments. Once a branch is merged if there
>> is a problem, then everybody should see it, and it will be fixed !
>>
>> Then, for 1 month, any new feature can be applied only if a newly
>> submitted branch is approved/signed-off (less than 2 months to detect a
>> problem in a new feature is a must-have)
>> Then, feature freeze, 1 month debugging and translating.
>>
>> I know it is a big change in the workflow, but the actual one is not
>> working well, so, we have to find another one.
>> I humbly request to have this question being put on the next irc agenda
>> (which does not mean you should not also start a thread here, of course)
>>
>> PS: pls don't say i'm wrong and the workflow works well. Having branches
>> submitted 3 months ago, getting no feedback, and planning to have a release
>> in less than 3 months, can't be considered as something working well.
>>
>> Friendly.
>>
>> --
>> Paul POULAIN
>> http://www.biblibre.com
>> Expert en Logiciels Libres pour l'info-doc
>> Tel : (33) 4 91 81 35 08
>>
>> _______________________________________________
>> Koha-devel mailing list
>> Koha-devel at lists.koha-community.org
>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : http://www.koha-community.org/
>> git : http://git.koha-community.org/
>> bugs : http://bugs.koha-community.org/
>>
>
>
>
> --
> Ian Walls
> Lead Development Specialist
> ByWater Solutions
> Phone # (888) 900-8944
> http://bywatersolutions.com
> ian.walls at bywatersolutions.com
> Twitter: @sekjal
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20110127/755e6d5d/attachment.htm>