Proposed changes to our Patch Submission and QA workflows
Everyone, In an effort to make patch submission and quality assurance easier on everyone for this next release cycle, I've got some proposals to run by you all. 1. In Bugzilla, move the Koha Bugs List from being the Default QA Contact to a Global Watcher. This has two advantages: 1. It frees up the QA Contact field for use by the QA Team 2. It guarantees that the Bugs List always gets included on the changes; with the current set up, someone could in theory remove the Bugs List as QA contact, preventing the list from getting updates. 2. To reduce email traffic, I propose we stop sending SIGNED OFF patches to the patches list. We've got Bugzilla lists (which can be connected to RSS and daily croned emails) which will inform the QA team on what patches are ready for QA. I did not use the patches list much this last release cycle to direct my QA work; it was all based of the Bugzilla statuses 2a. Ideally, I'd like to set up Bugzilla to automatically send the patches to the listserv for us, or to be able to accept patches sent to the listserv as attachments. This is going to take some custom coding, so it's probably not practical for right now. 3. Now that I'm not the only QA person for the release cycle, I'd like to formalize our rules about making sure that any submitted patch doesn't get processed solely by one company. I felt uncomfortable every time I passed a patch both written and signed off by ByWater teammates, because while I was sure it was good code, I knew I had a bias (often times, the code was fixing an issue one of our partners had reported). My strategy for 3.6 was to acknowledge the bias, and ask if anyone had any objections. With 3.8, I can pass anything that I write, or that is written/signed-off by ByWater alone, to Marcel or Jonathan for QA testing. Here're the rules I'd like to see in place: 1. A patch may not be written and signed off by the same person (existing rule) 2. The QA team consists of the Release Manager, the Quality Assurance Manager, and the Quality Assurance Assistants. This team has the right to move a patch through QA. 3. The QAM delegates the QA work between himself and the QAAs, in accordance with the rules. 4. A patch may not be written and QA'ed by the same person 5. A patch may not be written, signed off and QA'ed by any group of people in the same institution 6. A patch MAY be signed off and QA'ed by the same member of the QA team, but the RM may challenge this if he feels the patch needs more testing. Rule 5 still applies in this case. Comments? Ideas? Counter-proposals? -Ian -- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Le 24/10/2011 20:30, Ian Walls a écrit :
Everyone,
Everyalone, (french joke suspected ;-) )
In an effort to make patch submission and quality assurance easier on everyone for this next release cycle, I've got some proposals to run by you all.
1. In Bugzilla, move the Koha Bugs List from being the Default QA Contact to a Global Watcher. This has two advantages: 1. It frees up the QA Contact field for use by the QA Team 2. It guarantees that the Bugs List always gets included on the changes; with the current set up, someone could in theory remove the Bugs List as QA contact, preventing the list from getting updates.
+1
2. To reduce email traffic, I propose we stop sending SIGNED OFF patches to the patches list. We've got Bugzilla lists (which can be connected to RSS and daily croned emails) which will inform the QA team on what patches are ready for QA. I did not use the patches list much this last release cycle to direct my QA work; it was all based of the Bugzilla statuses
+1 (and already decided I think. At least i've updated the wiki that way 2 hours ago !)
2a. Ideally, I'd like to set up Bugzilla to automatically send the patches to the listserv for us, or to be able to accept patches sent to the listserv as attachments. This is going to take some custom coding, so it's probably not practical for right now.
+1 (and same comment : it would be great, but I don't know how to do it for instance)
3. Now that I'm not the only QA person for the release cycle, I'd like to formalize our rules about making sure that any submitted patch doesn't get processed solely by one company. I felt uncomfortable every time I passed a patch both written and signed off by ByWater teammates, because while I was sure it was good code, I knew I had a bias (often times, the code was fixing an issue one of our partners had reported). My strategy for 3.6 was to acknowledge the bias, and ask if anyone had any objections. With 3.8, I can pass anything that I write, or that is written/signed-off by ByWater alone, to Marcel or Jonathan for QA testing.
Here're the rules I'd like to see in place:
1. A patch may not be written and signed off by the same person (existing rule) OK
2. The QA team consists of the Release Manager, the Quality Assurance Manager, and the Quality Assurance Assistants. This team has the right to move a patch through QA. OK
3. The QAM delegates the QA work between himself and the QAAs, in accordance with the rules. OK, but how will you 3 decide who-QA-what ? module by module ? 1st who shoot ? regular meeting to assign QA ? other ?
4. A patch may not be written and QA'ed by the same person
+1, no doubt
5. A patch may not be written, signed off and QA'ed by any group of people in the same institution
You mean the 3 (writter-signoffer-QAer) can't be from the same institution ? So 2 can be OK ? I agree then.
6. A patch MAY be signed off and QA'ed by the same member of the QA team, but the RM may challenge this if he feels the patch needs more testing. Rule 5 still applies in this case.
+1 And it's not only a matter of "sign-off & QA by the same member". for major patches, I may/will request more testing than just 1 sign-off. in a few cases, Chris asked for more testing (when it was MARC sensible, for example), I'll continue that way (and maybe be harder than him) it may be worth adding a line about that.
Comments? Ideas? Counter-proposals?
A few comments: * for trivial patches (like template changes, comment/doc), I wouldn't object to have a short circuit (just 1 of QA/sign-off for example) * I'm OK to be a member of the QA team (as RM, see #2). And I'll introduce a little change in RM stuff: I may/could push a patch without testing it myself. I'm not sure yet, but I feel that if a patch has been written by Nicole, signed-off by Katrin, QAed by Marcel, in some situations, i will trust them. For now I don't know if this possibility will arise, but it could. My idea here is to have more time to communicate/organize/improve team working/... I'll see and let you know if it happens. * can we say "RM has the final cut" about accepting a patch ? -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Le 24/10/2011 20:30, Ian Walls a écrit :
Comments? Ideas? Counter-proposals?
("send" button hit too quickly...) * For developers who have their own publit git server, we could also let them publish their bug in a branch of their repo (and not attach the patch to the bug). Is it compatible with the signoff procedure (not sure). Is it accessible to ppls knowing git only a little (not sure) so it's just an idea I throw. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
participants (2)
-
Ian Walls -
Paul Poulain