Hello koha-devel, The more time passes, the more I think we have a workflow for Koha development that does not work correctly/efficiently/fluently. The rule we have is: * create a branch for the feature * file a bug for it * submit your branch to koha-patches ML * the RM checks that everything apply smoothly on master & ask for QA * someone signs-off the branch/patch * the RM merge the branch. For the maintenance branch, this workflow is perfect and must be secured more and more to avoid any regression. As I see it, there is always someone that can find time to sign-off a patch or 2. As ppl have system live with it i'm sure it's cricital for many ppl to check & double check ! But for the next release/unstable branch this workflow does not work well. Here is a single line of something owen said on the chat today: <owen> paul_p: I would much prefer to be testing your stuff today, but other stuff is taking precedence. God knows i'll be forever thankfull to owen for the work he does. But this sentence reveals something: for most of us, there is always "other stuff taking precedence" :\ BibLibre has submitted many new features 3 months ago. I tried to motivate ppl to get feedback & sign-off. I even have organised an IRC meeting ... where no one came. Those patches includes new features, and requires a lot of work to be tested. 2 weeks ago, I did the same for a bunch of small improvements (and I have many many more pending). Once again, no feedback, except for one very easy-to-test (new images) : signed-off by Nicole & pushed by chris in less than 24H! Those 2 minor events (with owen & nicole) made me realize that it's technically and functionnaly hard & long to test: rebase, drop database, install, file test cases, check the new features,... At KohaCon, liz tested some of those features and said "wow, great, I want this feature for my library !" (I don't remember what it was exactly). The problem is that liz is not a techie woman, and can't test patches by her own. For our images, it was easy for Nicole to test, no need to updatedatabase or anything complex, just reach the itemtypes.pl page check the images are here. Easy for a librarian ! Other point: dependancies between new features. Let me take an example: Templates on those devs relies on html::template. Chris is working on moving everything to Template::Toolkit. Once he consider it's ready, he should submit for QA, as for every new features. Suppose no-one sign-off. And suppose someone sign-off, there are 2 options: * move master to T::T when it's signed-off. What happends with other branches not merged ? they can't be merged anymore, they have to be rewritten. wow, I can't imagine that something we've submitted for QA 3 months ago has to be rewritten because no-one took care of it ! * wait until those branches are merged. Two major problems here: the release date won't be respected, and we have switched again to feature based release. Or 3.4 will be released on time, but won't contain many things all those great features that are live in all our libraries since months. The second problem is that other devs are on the way. They still rely on H::T as master is based on that. Switching to T::T will cause problems for those branches later. well, I think that none of those solutions is a correct one. I can take an other example: the analytics record feature may/will probably overlap functionnaly with the new features we have submitted for cataloguing (not sure, but i think so). Suppose our cataloguing branch is integrated in 2 weeks. it has been submitted 3 months ago. It means our indian colleagues will have to rebase & face problems to get their analytical record branch working. If our branch had been merged quickly, they would have had 3 more months to deal with it. Our actual workflow, causes a lot of overhead ! And -worst- I don't think it improves the stability & security: there are merge problems that are difficults to detect. It means that any test done on analytical records would have to be done again if our cataloguing branch is merged : more work, more pain. And the later a branch is merged, the smaller the time to find a problem. My main conclusion is that we are not a large enough community to deal with testing/validating/merging new features in a short timeline. So I think we must change the way we integrate new features. The general idea being: remove the sign-off lock on the workflow. Here is a proposal: Let's go back to the timeline: we plan to release a version every 6 months. We could have a window of, say 4 months. During those 4 months a feature can be included if : 1- the submitter provides something that applies on master (ie: it's technically valid) 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the branch is submitted) It put more pressure on others to test quickly, and don't put pain on parallels/parallelizing developments. Once a branch is merged if there is a problem, then everybody should see it, and it will be fixed ! Then, for 1 month, any new feature can be applied only if a newly submitted branch is approved/signed-off (less than 2 months to detect a problem in a new feature is a must-have) Then, feature freeze, 1 month debugging and translating. I know it is a big change in the workflow, but the actual one is not working well, so, we have to find another one. I humbly request to have this question being put on the next irc agenda (which does not mean you should not also start a thread here, of course) PS: pls don't say i'm wrong and the workflow works well. Having branches submitted 3 months ago, getting no feedback, and planning to have a release in less than 3 months, can't be considered as something working well. Friendly. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
One of the reasons it's taking so long to test bug 5575, and one of the reasons why it's hard to get others to test branches like it is that the branch includes so many changes at once. It would be so much simpler to be testing a branch that introduced a single feature or bugfix. I could look at that one thing, confirm it works, and sign off. As it is I have to poke around looking in different places, trying different things, looking for things which might have changed. It is so important that qa branches/patches are feature- or bug-specific. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org
Le 27/01/2011 17:23, Owen Leonard a écrit :
One of the reasons it's taking so long to test bug 5575, and one of the reasons why it's hard to get others to test branches like it is that the branch includes so many changes at once. It would be so much simpler to be testing a branch that introduced a single feature or bugfix. I could look at that one thing, confirm it works, and sign off. As it is I have to poke around looking in different places, trying different things, looking for things which might have changed.
It is so important that qa branches/patches are feature- or bug-specific. Yes, I agree, (and this point about too large branches has already been rised)
But: * when you add a major new feature, there are always many dependancies, making the branch not easy to test * it's not only a problem for this specific branch. I'm not comfortable with the fact that smaller patches/branches stays pending without feedback. No-one has given feedback on other branches as well. Including new ones that are very small and should/could be validated easily The problem is not that it takes so long to test, it's that no-one test ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Paul, I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges. So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years. I think a better solution is make it easier to test and signoff on work. There are several components to this: - Testing plans for larger developments/bugfixes - A robust testing data set made readily available - Teaching people how to test and signoff on code By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c. Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?". The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail. To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki ( http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan. I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :) Cheers, -Ian On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Hello koha-devel,
The more time passes, the more I think we have a workflow for Koha development that does not work correctly/efficiently/fluently. The rule we have is: * create a branch for the feature * file a bug for it * submit your branch to koha-patches ML * the RM checks that everything apply smoothly on master & ask for QA * someone signs-off the branch/patch * the RM merge the branch.
For the maintenance branch, this workflow is perfect and must be secured more and more to avoid any regression. As I see it, there is always someone that can find time to sign-off a patch or 2. As ppl have system live with it i'm sure it's cricital for many ppl to check & double check !
But for the next release/unstable branch this workflow does not work well.
Here is a single line of something owen said on the chat today: <owen> paul_p: I would much prefer to be testing your stuff today, but other stuff is taking precedence.
God knows i'll be forever thankfull to owen for the work he does. But this sentence reveals something: for most of us, there is always "other stuff taking precedence" :\
BibLibre has submitted many new features 3 months ago. I tried to motivate ppl to get feedback & sign-off. I even have organised an IRC meeting ... where no one came. Those patches includes new features, and requires a lot of work to be tested. 2 weeks ago, I did the same for a bunch of small improvements (and I have many many more pending). Once again, no feedback, except for one very easy-to-test (new images) : signed-off by Nicole & pushed by chris in less than 24H! Those 2 minor events (with owen & nicole) made me realize that it's technically and functionnaly hard & long to test: rebase, drop database, install, file test cases, check the new features,... At KohaCon, liz tested some of those features and said "wow, great, I want this feature for my library !" (I don't remember what it was exactly). The problem is that liz is not a techie woman, and can't test patches by her own. For our images, it was easy for Nicole to test, no need to updatedatabase or anything complex, just reach the itemtypes.pl page check the images are here. Easy for a librarian !
Other point: dependancies between new features. Let me take an example: Templates on those devs relies on html::template. Chris is working on moving everything to Template::Toolkit. Once he consider it's ready, he should submit for QA, as for every new features. Suppose no-one sign-off. And suppose someone sign-off, there are 2 options: * move master to T::T when it's signed-off. What happends with other branches not merged ? they can't be merged anymore, they have to be rewritten. wow, I can't imagine that something we've submitted for QA 3 months ago has to be rewritten because no-one took care of it ! * wait until those branches are merged. Two major problems here: the release date won't be respected, and we have switched again to feature based release. Or 3.4 will be released on time, but won't contain many things all those great features that are live in all our libraries since months. The second problem is that other devs are on the way. They still rely on H::T as master is based on that. Switching to T::T will cause problems for those branches later. well, I think that none of those solutions is a correct one.
I can take an other example: the analytics record feature may/will probably overlap functionnaly with the new features we have submitted for cataloguing (not sure, but i think so). Suppose our cataloguing branch is integrated in 2 weeks. it has been submitted 3 months ago. It means our indian colleagues will have to rebase & face problems to get their analytical record branch working. If our branch had been merged quickly, they would have had 3 more months to deal with it. Our actual workflow, causes a lot of overhead ! And -worst- I don't think it improves the stability & security: there are merge problems that are difficults to detect. It means that any test done on analytical records would have to be done again if our cataloguing branch is merged : more work, more pain. And the later a branch is merged, the smaller the time to find a problem.
My main conclusion is that we are not a large enough community to deal with testing/validating/merging new features in a short timeline. So I think we must change the way we integrate new features. The general idea being: remove the sign-off lock on the workflow.
Here is a proposal: Let's go back to the timeline: we plan to release a version every 6 months. We could have a window of, say 4 months. During those 4 months a feature can be included if : 1- the submitter provides something that applies on master (ie: it's technically valid) 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the branch is submitted)
It put more pressure on others to test quickly, and don't put pain on parallels/parallelizing developments. Once a branch is merged if there is a problem, then everybody should see it, and it will be fixed !
Then, for 1 month, any new feature can be applied only if a newly submitted branch is approved/signed-off (less than 2 months to detect a problem in a new feature is a must-have) Then, feature freeze, 1 month debugging and translating.
I know it is a big change in the workflow, but the actual one is not working well, so, we have to find another one. I humbly request to have this question being put on the next irc agenda (which does not mean you should not also start a thread here, of course)
PS: pls don't say i'm wrong and the workflow works well. Having branches submitted 3 months ago, getting no feedback, and planning to have a release in less than 3 months, can't be considered as something working well.
Friendly.
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
2011/1/28 Ian Walls <ian.walls@bywatersolutions.com>:
Paul,
I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges.
So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years.
I think a better solution is make it easier to test and signoff on work.
I agree whole heartedly
There are several components to this: l> Testing plans for larger developments/bugfixes A robust testing data set made readily available Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.
Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?".
The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail.
To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan.
That sounds like a great idea to me. I think that this is indeed missing, a champion of testing if you will, someone to pester people :)
I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :)
I think Ian has covered the sentiment of what I wanted to say, the problem isn't that stuff needs to be tested, I think we all agree things should be tested and signed off before going in master. But that testing is hard. Testing is made much harder by not following the one feature per branch rule. I just wanted to touch on one point Paul mentioned about the T::T work. We have been working to create a script to automatically convert H::T::P templates to T::T. Just so that we don't get into the scenario Paul mentions of having people having to rewrite their changes. Hundreds of hours of work have gone into making it so that the burden won't fall on the people submitting the code. Eventually we will stop accepting patches in H::T::P but it wont be write away, and certainly anything already submitted we wont be trying to make the authors rewrite. Chris
+1 to Ian's thoughts. Kind Regards, Chris 2011/1/27 Ian Walls <ian.walls@bywatersolutions.com>
Paul,
I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges.
So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years.
I think a better solution is make it easier to test and signoff on work. There are several components to this:
- Testing plans for larger developments/bugfixes - A robust testing data set made readily available - Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.
Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?".
The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail.
To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki ( http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan.
I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :)
Cheers,
-Ian
On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Hello koha-devel,
The more time passes, the more I think we have a workflow for Koha development that does not work correctly/efficiently/fluently. The rule we have is: * create a branch for the feature * file a bug for it * submit your branch to koha-patches ML * the RM checks that everything apply smoothly on master & ask for QA * someone signs-off the branch/patch * the RM merge the branch.
For the maintenance branch, this workflow is perfect and must be secured more and more to avoid any regression. As I see it, there is always someone that can find time to sign-off a patch or 2. As ppl have system live with it i'm sure it's cricital for many ppl to check & double check !
But for the next release/unstable branch this workflow does not work well.
Here is a single line of something owen said on the chat today: <owen> paul_p: I would much prefer to be testing your stuff today, but other stuff is taking precedence.
God knows i'll be forever thankfull to owen for the work he does. But this sentence reveals something: for most of us, there is always "other stuff taking precedence" :\
BibLibre has submitted many new features 3 months ago. I tried to motivate ppl to get feedback & sign-off. I even have organised an IRC meeting ... where no one came. Those patches includes new features, and requires a lot of work to be tested. 2 weeks ago, I did the same for a bunch of small improvements (and I have many many more pending). Once again, no feedback, except for one very easy-to-test (new images) : signed-off by Nicole & pushed by chris in less than 24H! Those 2 minor events (with owen & nicole) made me realize that it's technically and functionnaly hard & long to test: rebase, drop database, install, file test cases, check the new features,... At KohaCon, liz tested some of those features and said "wow, great, I want this feature for my library !" (I don't remember what it was exactly). The problem is that liz is not a techie woman, and can't test patches by her own. For our images, it was easy for Nicole to test, no need to updatedatabase or anything complex, just reach the itemtypes.pl page check the images are here. Easy for a librarian !
Other point: dependancies between new features. Let me take an example: Templates on those devs relies on html::template. Chris is working on moving everything to Template::Toolkit. Once he consider it's ready, he should submit for QA, as for every new features. Suppose no-one sign-off. And suppose someone sign-off, there are 2 options: * move master to T::T when it's signed-off. What happends with other branches not merged ? they can't be merged anymore, they have to be rewritten. wow, I can't imagine that something we've submitted for QA 3 months ago has to be rewritten because no-one took care of it ! * wait until those branches are merged. Two major problems here: the release date won't be respected, and we have switched again to feature based release. Or 3.4 will be released on time, but won't contain many things all those great features that are live in all our libraries since months. The second problem is that other devs are on the way. They still rely on H::T as master is based on that. Switching to T::T will cause problems for those branches later. well, I think that none of those solutions is a correct one.
I can take an other example: the analytics record feature may/will probably overlap functionnaly with the new features we have submitted for cataloguing (not sure, but i think so). Suppose our cataloguing branch is integrated in 2 weeks. it has been submitted 3 months ago. It means our indian colleagues will have to rebase & face problems to get their analytical record branch working. If our branch had been merged quickly, they would have had 3 more months to deal with it. Our actual workflow, causes a lot of overhead ! And -worst- I don't think it improves the stability & security: there are merge problems that are difficults to detect. It means that any test done on analytical records would have to be done again if our cataloguing branch is merged : more work, more pain. And the later a branch is merged, the smaller the time to find a problem.
My main conclusion is that we are not a large enough community to deal with testing/validating/merging new features in a short timeline. So I think we must change the way we integrate new features. The general idea being: remove the sign-off lock on the workflow.
Here is a proposal: Let's go back to the timeline: we plan to release a version every 6 months. We could have a window of, say 4 months. During those 4 months a feature can be included if : 1- the submitter provides something that applies on master (ie: it's technically valid) 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the branch is submitted)
It put more pressure on others to test quickly, and don't put pain on parallels/parallelizing developments. Once a branch is merged if there is a problem, then everybody should see it, and it will be fixed !
Then, for 1 month, any new feature can be applied only if a newly submitted branch is approved/signed-off (less than 2 months to detect a problem in a new feature is a must-have) Then, feature freeze, 1 month debugging and translating.
I know it is a big change in the workflow, but the actual one is not working well, so, we have to find another one. I humbly request to have this question being put on the next irc agenda (which does not mean you should not also start a thread here, of course)
PS: pls don't say i'm wrong and the workflow works well. Having branches submitted 3 months ago, getting no feedback, and planning to have a release in less than 3 months, can't be considered as something working well.
Friendly.
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Le 27/01/2011 17:49, Ian Walls a écrit :
I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges. i'm very happy to read that we agree on the problem (congrats you've summarized in 3 lines what I needed 50+ to express ;-) ) So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years. That's where we disagree, but if we agree on the problem, if we all put our hands in it, we will find a solution that appears to all to be the good one ! I think a better solution is make it easier to test and signoff on work. There are several components to this:
* Testing plans for larger developments/bugfixes * A robust testing data set made readily available * Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.
Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?". well, in my opinion, the one that does this testing plan must not be the one who wrote the feature. Because, of course, the feature has been tested, and the one who tested will have missed a use-case, or forgotten something,... ("given enough eyes, all bugs will be found"). So we're back to the question: who can dedicate time? Maybe we're back to the question of someone being (collectively) paid just for this role? As of today, I see that no-one find time to do it ! The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail.
To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan. Do you really think? I don't think so: If that were the case, then someone should/would have said "well seems interesting, but I need more information/help/..." We had setup sandbox for all our branches, organized a meeting where I were alone (only kf jumped-in to say "sorry, I can't be here"). I was so disappointed that ... I forgot I had planned a 2nd meeting, a few days later, was not here as expected, and no-one complained, so I concluded no-one has been attending as well. I ask and will continue to ask on #irc when someone passes around, but I feel like someone crying in the desert (frenchism suspected here)
So, I fully agree that the workflow is theorically a good one. But we lack the ressource to make it happen. I'm very scared about that, but i'd like to see things going on anyway.
I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On Thu, Jan 27, 2011 at 12:19 PM, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 27/01/2011 17:49, Ian Walls a écrit :
Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan.
Do you really think? I don't think so: If that were the case, then someone should/would have said "well seems interesting, but I need more information/help/..."
This is problematic for the same reason it's problematic to organize a QA session on IRC: It's hard to find the right people at the right time. It's daunting to approach a QA branch which has an unknown scope, and hard to ask questions if you don't know what you're looking for. If there is a branch to test, I want to know as much as possible about what is to be tested We have the right infrastructure with the workflow we have if these conditions are met: 1. The branch covers one feature/fix. 2. The feature/fix is described in a bug report. If these things are true I've got everything I need in order to test. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org
On 28 January 2011 06:19, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 27/01/2011 17:49, Ian Walls a écrit :
I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges. i'm very happy to read that we agree on the problem (congrats you've summarized in 3 lines what I needed 50+ to express ;-) ) So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years. That's where we disagree, but if we agree on the problem, if we all put our hands in it, we will find a solution that appears to all to be the good one ! I think a better solution is make it easier to test and signoff on work. There are several components to this:
* Testing plans for larger developments/bugfixes * A robust testing data set made readily available * Teaching people how to test and signoff on code
By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c.
Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?". well, in my opinion, the one that does this testing plan must not be the one who wrote the feature. Because, of course, the feature has been tested, and the one who tested will have missed a use-case, or forgotten something,... ("given enough eyes, all bugs will be found"). So we're back to the question: who can dedicate time? Maybe we're back to the question of someone being (collectively) paid just for this role? As of today, I see that no-one find time to do it ! The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail.
To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan. Do you really think? I don't think so: If that were the case, then someone should/would have said "well seems interesting, but I need more information/help/..." We had setup sandbox for all our branches, organized a meeting where I were alone (only kf jumped-in to say "sorry, I can't be here"). I was so disappointed that ... I forgot I had planned a 2nd meeting, a few days later, was not here as expected, and no-one complained, so I concluded no-one has been attending as well. I ask and will continue to ask on #irc when someone passes around, but I feel like someone crying in the desert (frenchism suspected here)
So, I fully agree that the workflow is theorically a good one. But we lack the ressource to make it happen. I'm very scared about that, but i'd like to see things going on anyway.
I think the workflow is good in practice too. For nearly every change the workflow is * create a branch per feature * file a bug for it * submit your patches to the ML * bug is marked needs signed off * someone takes the patch signs off it applies and works, and submits signed off patch * bug is marked signed off * RM applies signed off patch to a branch based on master * RM tests, if it is ok * RM Merges * bug is marked patch pushed * Feature/bug is tested and bug updated * bug is marked resolved The problem is, and I don't mean to harp on about this but it is important, is that the work that is waiting to be merged has not followed the workflow. The branches are not one per feature. This causes workflow to deal with then to become: * massive bunch of changes submitted in one branch * RM merge into qa ready branches testing they merge cleanly to master * try to find people to test a huge pile of changes * changes signed off * RM tests * Merged Now this a legacy problem, but one we need to make sure doesn't happen again, this is the only reason I keep repeating, One feature per branch. What I see is we have 2 things that need to happen, the Biblibre patches need to get tested, and in general the testing step of the workflow could be improved. I don't think either of these issues will be solved by removing the sign off step. Biblibre have acknowledged that they made a mistake in having one massive branch with all the changes on it and I congratulate them on that. To butcher an old saying 'To make a mistake is Human, to admit it in public is divine'. To their credit they have also tried to cut the work into smaller branches. But they are still big branches with more than one feature in them, and this I think is the major in fact could be the only reason they have not all been fully tested and merged yet. This is a separate problem, lets call it, how do we deal with work that has been submitted that doesn't follow the workflow, and work on a solution for that. Paul himself said that for patches that do follow the workflow (the image patch Nicole tested and signed off) the workflow works. So lets not change that, lets work out how to deal with patches that have been submitted that dont follow the workflow, ie, step 1, one branch per feature. It's not an easy problem but it is one we are all motivated to solve. No one. least of all me, wants to see anyones work go to waster. I want Biblibre's code in master before 3.4, lets try to work out how we can do it. All the while keeping vigilant and making sure all work follows the one branch per feature/bug rule for the future. Combine that with a better way of testing, and someone championing that, I think we are looking good for the future. Chris PS please forgive typos, I was woken at 5am by my son, and haven't had enough caffeine yet.
Nicole would like to chip in here on lots of points. Paul brings me up as an example of someone who signs off on the easy to test features and Ian brings up training/teaching people how to test/sign off. I think these two things are very important to note. I would sign off on more complex patches if I wasn't terrified of screwing up my one and only Koha installation/database. I know I can backup and restore and all that jazz, but I've never done it and to that end it's scary. I've screwed up my git repo a few times just trying to test the 'easy' patches! :) I would love to be properly trained in doing this so I can test further. I am on the road a lot and do most of my sign offs while alone in hotel rooms - that's the perfect quiet time to test!!! So if someone wants to do a tutorial video or write up detailed instructions I could do more (and I'm sure others like me could too). Thanks Nicole
On 1/27/2011 12:43 PM, Nicole Engard wrote:
Nicole would like to chip in here on lots of points. Paul brings me up as an example of someone who signs off on the easy to test features and Ian brings up training/teaching people how to test/sign off. I think these two things are very important to note. I would sign off on more complex patches if I wasn't terrified of screwing up my one and only Koha installation/database. I know I can backup and restore and all that jazz, but I've never done it and to that end it's scary. I've screwed up my git repo a few times just trying to test the 'easy' patches! :)
I would love to be properly trained in doing this so I can test further. I am on the road a lot and do most of my sign offs while alone in hotel rooms - that's the perfect quiet time to test!!! So if someone wants to do a tutorial video or write up detailed instructions I could do more (and I'm sure others like me could too).
Thanks Nicole
What would be Really Useful would be a virtual machine (e.g. Amazon EC2 or VMware) with a test environment to "lower the bar" for technical skills and computer resource availability. That could enable a larger pool of people who could test branches. As others pointed out, the worst case is some legacy large patch branches... having a EC2 test instance might break the logjam on them but should not be necessary for smaller patches. Amazon has very good starter terms: <http://aws.amazon.com/ec2/pricing/> --------------------------------------------------------------------- Free Tier* As part of AWS’s Free Usage Tier, new AWS customers can get started with Amazon EC2 for free. Upon sign-up, new AWS customers receive the following EC2 services each month for one year: * 750 hours of EC2 running Linux/Unix Micro instance usage * 750 hours of Elastic Load Balancing plus 15 GB data processing * 10 GB of Amazon Elastic Block Storage (EBS) plus 1 million IOs, 1 GB snapshot storage, 10,000 snapshot Get Requests and 1,000 snapshot Put Requests * 15 GB of bandwidth in and 15 GB of bandwidth out aggregated across all AWS services --------------------------------------------------------------------- Best regards, gvb
On Thu, Jan 27, 2011 at 12:37 PM, Chris Cormack <chris@bigballofwax.co.nz>wrote: <snip>
This is a separate problem, lets call it, how do we deal with work that has been submitted that doesn't follow the workflow, and work on a solution for that. Paul himself said that for patches that do follow the workflow (the image patch Nicole tested and signed off) the workflow works. So lets not change that, lets work out how to deal with patches that have been submitted that dont follow the workflow, ie, step 1, one branch per feature.
It's not an easy problem but it is one we are all motivated to solve. No one. least of all me, wants to see anyones work go to waster. I want Biblibre's code in master before 3.4, lets try to work out how we can do it. All the while keeping vigilant and making sure all work follows the one branch per feature/bug rule for the future. Combine that with a better way of testing, and someone championing that, I think we are looking good for the future.
This is the heart of the problem as I see it. The current workflow is fine (and could be embellished with Ian's suggestions/proposal.) The issue of the outstanding Biblibre code is a "left-over" problem which we need to setup up to, fix, and get behind us as soon as possible. Perhaps we should schedule a "Test Fest" similar to a "Hack Fest" and get a group of people together to pound on the Biblibre test servers one day? Kind Regards, Chris
On 28 January 2011 07:21, Chris Nighswonger <cnighswonger@foundations.edu> wrote:
On Thu, Jan 27, 2011 at 12:37 PM, Chris Cormack <chris@bigballofwax.co.nz> wrote:
<snip>
This is a separate problem, lets call it, how do we deal with work that has been submitted that doesn't follow the workflow, and work on a solution for that. Paul himself said that for patches that do follow the workflow (the image patch Nicole tested and signed off) the workflow works. So lets not change that, lets work out how to deal with patches that have been submitted that dont follow the workflow, ie, step 1, one branch per feature.
It's not an easy problem but it is one we are all motivated to solve. No one. least of all me, wants to see anyones work go to waster. I want Biblibre's code in master before 3.4, lets try to work out how we can do it. All the while keeping vigilant and making sure all work follows the one branch per feature/bug rule for the future. Combine that with a better way of testing, and someone championing that, I think we are looking good for the future.
This is the heart of the problem as I see it. The current workflow is fine (and could be embellished with Ian's suggestions/proposal.) The issue of the outstanding Biblibre code is a "left-over" problem which we need to setup up to, fix, and get behind us as soon as possible.
Perhaps we should schedule a "Test Fest" similar to a "Hack Fest" and get a group of people together to pound on the Biblibre test servers one day?
I could commit to a couple of hours for that, and I think I could get a few other catalyst employees too. Chris
Le 27/01/2011 19:21, Chris Nighswonger a écrit :
This is the heart of the problem as I see it. The current workflow is fine (and could be embellished with Ian's suggestions/proposal.) The issue of the outstanding Biblibre code is a "left-over" problem which we need to setup up to, fix, and get behind us as soon as possible. ++ Perhaps we should schedule a "Test Fest" similar to a "Hack Fest" and get a group of people together to pound on the Biblibre test servers one day? ++
How should we organise this test fest ? Should I start a doodle to get a time ? Or say "it open dayX and dayY, i'll be on the channel" ? i'm listening to your suggestions ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Just my late response on this thread: 1) If I include a test plan but should forget steps/test areas etc., and a non-developer tests the patch: aren’t we decreasing the value of signoff? In that case we could still end up with not signing patches in the long run.. Still, something is perhaps better than nothing at all. 2) A patch should not be too big. 3) Much depends on how the QA role is filled in [Ian’s plans sounds good]. If the QA could give more guidance on which patches should be tested next, testers are not just cherry picking patches.. Why should QA not mail once a week or so the first 10 or 20 patches that he wants to be signed? Could prevent older/larger/less interesting sounding patches waiting and newer (smaller/..) patches getting signed and pushed. Would make the process even more ‘fair’ to everyone. Marcel Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Ian Walls Verzonden: donderdag 27 januari 2011 17:50 Aan: Paul Poulain CC: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] Development workflow Paul, I hear you. There are lots of really great features that are in limbo, waiting to be tested. The unfortunate situation of life is that we're all incredibly busy, and it's hard to find time to test things. This is compounded when testing certain features requires coding or systems experience, since that limits the pool of potential testers. As a result, lots of good stuff sits, waits, and diverges. So what can be done? I don't think that changing the signoff procedure will have the desired effect. That'll just let more unreviewed code slip in, potentially introducing bugs we won't find for weeks/months/years. I think a better solution is make it easier to test and signoff on work. There are several components to this: * Testing plans for larger developments/bugfixes * A robust testing data set made readily available * Teaching people how to test and signoff on code By including testing plans with developments or complex bugfixes, the developer is communicating to everyone how they can prove their code works. It lays out the intention of the development (it should do x, y and z), and a series of tests to show how to get x, y and z without losing a, b and c. Combine this testing plan (written in language librarians can understand, not coder jargon) with the necessary data set to do the testing (an SQL dump you just load into a DB), and you've lowered the barrier for testing so that anyone who can afford a little time to run through a series of listed procedures can answer the question "does this work?". The third step to this is to lay out the procedure for running through the test plan in a clear, simple manner, and distribute that information far and wide. Make it something that librarians can do by following a list of steps. Lowering the threshold of experience required to test things will allow us to harness the Long Tail. To this end, I'm throwing my hat in the ring for Quality Assurance Manager for Koha 3.6. My proposal can be found on the wiki (http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much of it is explained above. In addition to this, I would also serve as a coordinator for testing work submitted, and provide regular reports to the community on the status of these developments. Branches that are not receiving active testing feedback would receive attention towards creating a simpler, easier to follow testing plan. I would very much like to discuss this at the next IRC meeting. It'll be pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand :) Cheers, -Ian On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain <paul.poulain@biblibre.com<mailto:paul.poulain@biblibre.com>> wrote: Hello koha-devel, The more time passes, the more I think we have a workflow for Koha development that does not work correctly/efficiently/fluently. The rule we have is: * create a branch for the feature * file a bug for it * submit your branch to koha-patches ML * the RM checks that everything apply smoothly on master & ask for QA * someone signs-off the branch/patch * the RM merge the branch. For the maintenance branch, this workflow is perfect and must be secured more and more to avoid any regression. As I see it, there is always someone that can find time to sign-off a patch or 2. As ppl have system live with it i'm sure it's cricital for many ppl to check & double check ! But for the next release/unstable branch this workflow does not work well. Here is a single line of something owen said on the chat today: <owen> paul_p: I would much prefer to be testing your stuff today, but other stuff is taking precedence. God knows i'll be forever thankfull to owen for the work he does. But this sentence reveals something: for most of us, there is always "other stuff taking precedence" :\ BibLibre has submitted many new features 3 months ago. I tried to motivate ppl to get feedback & sign-off. I even have organised an IRC meeting ... where no one came. Those patches includes new features, and requires a lot of work to be tested. 2 weeks ago, I did the same for a bunch of small improvements (and I have many many more pending). Once again, no feedback, except for one very easy-to-test (new images) : signed-off by Nicole & pushed by chris in less than 24H! Those 2 minor events (with owen & nicole) made me realize that it's technically and functionnaly hard & long to test: rebase, drop database, install, file test cases, check the new features,... At KohaCon, liz tested some of those features and said "wow, great, I want this feature for my library !" (I don't remember what it was exactly). The problem is that liz is not a techie woman, and can't test patches by her own. For our images, it was easy for Nicole to test, no need to updatedatabase or anything complex, just reach the itemtypes.pl<http://itemtypes.pl> page check the images are here. Easy for a librarian ! Other point: dependancies between new features. Let me take an example: Templates on those devs relies on html::template. Chris is working on moving everything to Template::Toolkit. Once he consider it's ready, he should submit for QA, as for every new features. Suppose no-one sign-off. And suppose someone sign-off, there are 2 options: * move master to T::T when it's signed-off. What happends with other branches not merged ? they can't be merged anymore, they have to be rewritten. wow, I can't imagine that something we've submitted for QA 3 months ago has to be rewritten because no-one took care of it ! * wait until those branches are merged. Two major problems here: the release date won't be respected, and we have switched again to feature based release. Or 3.4 will be released on time, but won't contain many things all those great features that are live in all our libraries since months. The second problem is that other devs are on the way. They still rely on H::T as master is based on that. Switching to T::T will cause problems for those branches later. well, I think that none of those solutions is a correct one. I can take an other example: the analytics record feature may/will probably overlap functionnaly with the new features we have submitted for cataloguing (not sure, but i think so). Suppose our cataloguing branch is integrated in 2 weeks. it has been submitted 3 months ago. It means our indian colleagues will have to rebase & face problems to get their analytical record branch working. If our branch had been merged quickly, they would have had 3 more months to deal with it. Our actual workflow, causes a lot of overhead ! And -worst- I don't think it improves the stability & security: there are merge problems that are difficults to detect. It means that any test done on analytical records would have to be done again if our cataloguing branch is merged : more work, more pain. And the later a branch is merged, the smaller the time to find a problem. My main conclusion is that we are not a large enough community to deal with testing/validating/merging new features in a short timeline. So I think we must change the way we integrate new features. The general idea being: remove the sign-off lock on the workflow. Here is a proposal: Let's go back to the timeline: we plan to release a version every 6 months. We could have a window of, say 4 months. During those 4 months a feature can be included if : 1- the submitter provides something that applies on master (ie: it's technically valid) 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the branch is submitted) It put more pressure on others to test quickly, and don't put pain on parallels/parallelizing developments. Once a branch is merged if there is a problem, then everybody should see it, and it will be fixed ! Then, for 1 month, any new feature can be applied only if a newly submitted branch is approved/signed-off (less than 2 months to detect a problem in a new feature is a must-have) Then, feature freeze, 1 month debugging and translating. I know it is a big change in the workflow, but the actual one is not working well, so, we have to find another one. I humbly request to have this question being put on the next irc agenda (which does not mean you should not also start a thread here, of course) PS: pls don't say i'm wrong and the workflow works well. Having branches submitted 3 months ago, getting no feedback, and planning to have a release in less than 3 months, can't be considered as something working well. Friendly. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org<mailto:Koha-devel@lists.koha-community.org> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ -- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com<mailto:ian.walls@bywatersolutions.com> Twitter: @sekjal
Le 31/01/2011 09:11, Marcel de Rooy a écrit :
Just my late response on this thread:
3) Much depends on how the QA role is filled in [Ian’s plans sounds good]. If the QA could give more guidance on which patches should be tested next, testers are not just cherry picking patches.. Why should QA not mail once a week or so the first 10 or 20 patches that he wants to be signed? Could prevent older/larger/less interesting sounding patches waiting and newer (smaller/..) patches getting signed and pushed. Would make the process even more ‘fair’ to everyone.
Sounds like a very good idea. I've investigated a little big bugzilla to see if we could have a query showing what is to sign-off. It seems that we have ... 164 bugs waiting for sign-off : http://bugs.koha-community.org/bugzilla3/buglist.cgi?field0-0-0=cf_patch_status&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&type0-0-0=equals&value0-0-0=Needs%20Signoff I'm highly surprised by this number. Did I make a wrong query ? Is bugzilla uptodate ? I can't imagine there are so many patches awaiting QA, but maybe i'm wrong ;\ -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On 31 January 2011 21:43, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 31/01/2011 09:11, Marcel de Rooy a écrit :
Just my late response on this thread:
3) Much depends on how the QA role is filled in [Ian’s plans sounds good]. If the QA could give more guidance on which patches should be tested next, testers are not just cherry picking patches.. Why should QA not mail once a week or so the first 10 or 20 patches that he wants to be signed? Could prevent older/larger/less interesting sounding patches waiting and newer (smaller/..) patches getting signed and pushed. Would make the process even more ‘fair’ to everyone.
Sounds like a very good idea. I've investigated a little big bugzilla to see if we could have a query showing what is to sign-off.
It seems that we have ... 164 bugs waiting for sign-off : http://bugs.koha-community.org/bugzilla3/buglist.cgi?field0-0-0=cf_patch_status&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&type0-0-0=equals&value0-0-0=Needs%20Signoff
I'm highly surprised by this number. Did I make a wrong query ? Is bugzilla uptodate ? I can't imagine there are so many patches awaiting QA, but maybe i'm wrong ;\
No that's right, I think I've pointed this out before, but I have links to the searches on my release wiki http://koha-releasemanagement.branchable.com/ (links up the top) Of course anyone can sign off on these, if we look at the statistics for the 3.2.3 release http://blog.bigballofwax.co.nz/2011/01/23/koha-3-2-3-statistics/ We can see who did signoffs for that release, http://blog.bigballofwax.co.nz/2010/12/22/koha-3-2-2-statistics/ for 3.2.2. Id love to see more names in there, Ive managed to get my list of patches awaiting testing from me down to 2, so time to start filling that list back up again. Get signing off people :) Chris
Already for some weeks this number is around 160. They include bug reports pertaining to specific rel 3.0. If the patch no longer applies and assuming that nothing happens any more in 3.0, I would set these reports to Not apply. Or even close them? Note that I closed a 3.0 bug in the past, but got negative response on doing so.. Just a tip: Save your query in Bugzilla for reports to be signed. -----Oorspronkelijk bericht----- Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Chris Cormack Verzonden: maandag 31 januari 2011 09:51 Aan: Paul Poulain CC: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] Development workflow On 31 January 2011 21:43, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 31/01/2011 09:11, Marcel de Rooy a écrit :
Just my late response on this thread:
3) Much depends on how the QA role is filled in [Ian’s plans sounds good]. If the QA could give more guidance on which patches should be tested next, testers are not just cherry picking patches.. Why should QA not mail once a week or so the first 10 or 20 patches that he wants to be signed? Could prevent older/larger/less interesting sounding patches waiting and newer (smaller/..) patches getting signed and pushed. Would make the process even more ‘fair’ to everyone.
Sounds like a very good idea. I've investigated a little big bugzilla to see if we could have a query showing what is to sign-off.
It seems that we have ... 164 bugs waiting for sign-off : http://bugs.koha-community.org/bugzilla3/buglist.cgi?field0-0-0=cf_patch_status&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&type0-0-0=equals&value0-0-0=Needs%20Signoff
I'm highly surprised by this number. Did I make a wrong query ? Is bugzilla uptodate ? I can't imagine there are so many patches awaiting QA, but maybe i'm wrong ;\
No that's right, I think I've pointed this out before, but I have links to the searches on my release wiki http://koha-releasemanagement.branchable.com/ (links up the top) Of course anyone can sign off on these, if we look at the statistics for the 3.2.3 release http://blog.bigballofwax.co.nz/2011/01/23/koha-3-2-3-statistics/ We can see who did signoffs for that release, http://blog.bigballofwax.co.nz/2010/12/22/koha-3-2-2-statistics/ for 3.2.2. Id love to see more names in there, Ive managed to get my list of patches awaiting testing from me down to 2, so time to start filling that list back up again. Get signing off people :) Chris _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
On 31 January 2011 22:01, Marcel de Rooy <M.de.Rooy@rijksmuseum.nl> wrote:
Already for some weeks this number is around 160. They include bug reports pertaining to specific rel 3.0. If the patch no longer applies and assuming that nothing happens any more in 3.0, I would set these reports to Not apply. Or even close them? Note that I closed a 3.0 bug in the past, but got negative response on doing so..
The thing to do, is test if the bug still exists in master/3.2.x, if it does update the bug accordingly (change the version). If the patch doesn't apply you can change it from needs signoff to 'Does not apply'. I wouldn't close the bug unless I was sure it isn't present in either master or 3.2.x Chris
Le 31/01/2011 10:07, Chris Cormack a écrit :
The thing to do, is test if the bug still exists in master/3.2.x, if it does update the bug accordingly (change the version). If the patch doesn't apply you can change it from needs signoff to 'Does not apply'.
I wouldn't close the bug unless I was sure it isn't present in either master or 3.2.x it isn't present in either or both master/3.2 ?
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On 31 January 2011 22:20, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 31/01/2011 10:07, Chris Cormack a écrit :
The thing to do, is test if the bug still exists in master/3.2.x, if it does update the bug accordingly (change the version). If the patch doesn't apply you can change it from needs signoff to 'Does not apply'.
I wouldn't close the bug unless I was sure it isn't present in either master or 3.2.x it isn't present in either or both master/3.2 ?
One of the foibles of English, is that they both mean the same thing. To be more clear, make sure the bug doesn't exist in 3.2.x and doesn't exist in master Chris
participants (8)
-
Chris Cormack -
Chris Nighswonger -
Ian Walls -
Jerry Van Baren -
Marcel de Rooy -
Nicole Engard -
Owen Leonard -
Paul Poulain