Hi all, We already agreed some time ago that the patch writer and the patch sign offer should not be from the same company/library/party. I would really like to extend that rule to also apply for patch writer and QA team member. For 3.12 we got a big QA team, which is great, because with all the ongoing work we are going to need it. And it also gives us more options and flexibility when it comes to these kinds of questions. I would like to get your opinions on this, can we agree on this new rule? Katrin
On 29 November 2012 10:34, Katrin Fischer <Katrin.Fischer.83@web.de> wrote:
Hi all,
We already agreed some time ago that the patch writer and the patch sign offer should not be from the same company/library/party. I would really like to extend that rule to also apply for patch writer and QA team member. For 3.12 we got a big QA team, which is great, because with all the ongoing work we are going to need it. And it also gives us more options and flexibility when it comes to these kinds of questions.
I would like to get your opinions on this, can we agree on this new rule?
Not being part of the QA team, but speaking as Release Maintainer, I think this is a great idea. First up people who are from the same organisation could comment on the patch, I shouldn't QA this, can someone else. If no one does step up, or everyone says no I can't. I think as a last resort we could allow it. But only as a last resort. Hope this makes sense, and to continue in my habit of quoting Tolkien for no good reason. "Short cuts make for long delays." ― J.R.R. Tolkien, The Hobbit Chris
On Wed, Nov 28, 2012 at 4:37 PM, Chris Cormack <chris@bigballofwax.co.nz>wrote:
Not being part of the QA team, but speaking as Release Maintainer, I think this is a great idea. First up people who are from the same organisation could comment on the patch, I shouldn't QA this, can someone else. If no one does step up, or everyone says no I can't. I think as a last resort we could allow it. But only as a last resort.
+1 for keeping conflicts of interest at a minimum. However, I would also agree with the "If no one does step up..." clause. Too much work stands to be lost if no one will pick up the QA on a patch, and we need to have some sort of mechanism to keep that from happening. (Not to mention that some work is a real chore to keep rebased against the current master.) On a second thought, perhaps we should entertain some sort of time/date sequential procedure whereby the QA team would QA in order of severity sub-sorted by date of general sign-off or some such in order to ensure that no signed-off patch is left out. Perhaps if no member of the QA team feels "qualified" to QA a given patch, the QAM could "deputize" another disinterested community member to do the work. Just thinking out loud here. Kind Regards, Chris
Hi all, rereading this thread again I think we can agree on the following workflow: If a patch is not passed QA for a longer time, the developer or party can leave a note on the bug or send mail to the mailing list as a gentle nudge to the other QA team members. If a patch is not passed QA for one month, then the patch can be QA'd by the same party. This doesn't mean of course that the RM can still ask for a second sign-off for a big feature or something he worries about. I am going to add this to the wiki. Katrin
+1 for gentle nudges :) If A@B writes a patch and C@D signs off, I would not mind E@D to do qa. I think we should prevent: A@B writes and C@B signs off and D@B wants to do qa. (Reading back, the B is completely incidental :) Marcel -----Oorspronkelijk bericht----- Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Katrin Fischer Verzonden: maandag 3 december 2012 8:09 Aan: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] [QA] QA process Hi all, rereading this thread again I think we can agree on the following workflow: If a patch is not passed QA for a longer time, the developer or party can leave a note on the bug or send mail to the mailing list as a gentle nudge to the other QA team members. If a patch is not passed QA for one month, then the patch can be QA'd by the same party. This doesn't mean of course that the RM can still ask for a second sign-off for a big feature or something he worries about. I am going to add this to the wiki. Katrin _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Le 28/11/2012 22:34, Katrin Fischer a écrit :
Hi all, Hi QAM,
We already agreed some time ago that the patch writer and the patch sign offer should not be from the same company/library/party. I would really like to extend that rule to also apply for patch writer and QA team member. For 3.12 we got a big QA team, which is great, because with all the ongoing work we are going to need it. And it also gives us more options and flexibility when it comes to these kinds of questions.
I would like to get your opinions on this, can we agree on this new rule?
Am I right if I say that BibLibre will be the company mostly concerned impacted by this rule ? Checking numbers. Some numbers: there are 131 patches waiting for signoff or QA. 52 are from BibLibre (40%) Displaying by "date of last change" (ie= there's been no activity on this bug since...) * August= 9 patches: 5 BibLibre, 1 OSSlab, 1 ByWater, 3 others * September = 8 patches: 4 BibLibre, 3 ByWater, 1 other * October = 27 patches: 14 BibLibre, 3 ACPL, 6 ByWater, 4 others (All of them are Enhancements or New Feature) At the end, my opinion is that it's a little bit unfair (and I restart a long-standing discussion/complain...) BibLibre does a huge of effort to submit all his development, and be quick when there is feedback. The very bad side effect of delays is that we loose a lot of energy in rebasing, just because there's no feedback from anyone, and when someone step-in the patch does not apply any more. We fully accept the rule for not self-signing the patch, because, functionally speaking, I agree that it's good to have an external eye. (and that's more important with large features) But the QA is a *technical* check of the code. If something is wrong (ie don't respect our guidelines, has a bad side effect, ...), as QAer, I'll do *exactly* the same thing whoever the patch come from. If I were suspicious, I could even say that you imply that, when Jonathan or me QA a patch from BibLibre, we're biased, and I could be upset by your suspicion (I'm not, I'm just very sad that this discussion started again, while I thought it was solved) I never "promote" BibLibre patches, I always QA by date of last change, ie: I QA the older patch without activity. Yesterday, I QAed something like 10 patches, iirc, 2 from BibLibre failed QA (including one just because there was some PODDOC missing !) Chris-es are proposing what could be a fair rule imo = "if no one step up". What could be considered as "no one step up" being the next question... -- Paul POULAIN - BibLibre http://www.biblibre.com Free & Open Source Softwares for libraries Koha, Drupal, Piwik, Jasper
Nice to see that QA gets so much attention these days :) I think that Katrin's proposal on itself is a valid one. We never wrote it out in our workflow document on the wiki. That document (that apparently has community consensus) says only: "Preferably, patch writer and patch signer should not be from the same company or institution." But just adding the clause that if patch writer and signer are from the same company, the QAer should be (preferably) independent, is in the same line of thought, just a logical extrapolation. IMO we could add what Katrin stated to this workflow document, and keep situations where writer, signer and QAer are in the same company to an absolute minimum (very close to zero). Just noting also that Paul's numbers relate to Needs Signoff and not to Signed off. The oldest patch waiting for QA from Biblibre are four from October 19. QA should indeed now pick them up first. Before I have QAed several patches already that were written and signed Biblibre only. I do not think that QAing Biblibre patches is really an issue now. Marcel ________________________________________ Van: koha-devel-bounces@lists.koha-community.org [koha-devel-bounces@lists.koha-community.org] namens Paul Poulain [paul.poulain@biblibre.com] Verzonden: donderdag 29 november 2012 9:52 To: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] [QA] QA process Le 28/11/2012 22:34, Katrin Fischer a écrit :
Hi all, Hi QAM,
We already agreed some time ago that the patch writer and the patch sign offer should not be from the same company/library/party. I would really like to extend that rule to also apply for patch writer and QA team member. For 3.12 we got a big QA team, which is great, because with all the ongoing work we are going to need it. And it also gives us more options and flexibility when it comes to these kinds of questions.
I would like to get your opinions on this, can we agree on this new rule?
Am I right if I say that BibLibre will be the company mostly concerned impacted by this rule ? Checking numbers. Some numbers: there are 131 patches waiting for signoff or QA. 52 are from BibLibre (40%) Displaying by "date of last change" (ie= there's been no activity on this bug since...) * August= 9 patches: 5 BibLibre, 1 OSSlab, 1 ByWater, 3 others * September = 8 patches: 4 BibLibre, 3 ByWater, 1 other * October = 27 patches: 14 BibLibre, 3 ACPL, 6 ByWater, 4 others (All of them are Enhancements or New Feature) At the end, my opinion is that it's a little bit unfair (and I restart a long-standing discussion/complain...) BibLibre does a huge of effort to submit all his development, and be quick when there is feedback. The very bad side effect of delays is that we loose a lot of energy in rebasing, just because there's no feedback from anyone, and when someone step-in the patch does not apply any more. We fully accept the rule for not self-signing the patch, because, functionally speaking, I agree that it's good to have an external eye. (and that's more important with large features) But the QA is a *technical* check of the code. If something is wrong (ie don't respect our guidelines, has a bad side effect, ...), as QAer, I'll do *exactly* the same thing whoever the patch come from. If I were suspicious, I could even say that you imply that, when Jonathan or me QA a patch from BibLibre, we're biased, and I could be upset by your suspicion (I'm not, I'm just very sad that this discussion started again, while I thought it was solved) I never "promote" BibLibre patches, I always QA by date of last change, ie: I QA the older patch without activity. Yesterday, I QAed something like 10 patches, iirc, 2 from BibLibre failed QA (including one just because there was some PODDOC missing !) Chris-es are proposing what could be a fair rule imo = "if no one step up". What could be considered as "no one step up" being the next question... -- Paul POULAIN - BibLibre http://www.biblibre.com Free & Open Source Softwares for libraries Koha, Drupal, Piwik, Jasper _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Paul, Am I right if I say that BibLibre will be the company mostly concerned
impacted by this rule ? Checking numbers. Some numbers: there are 131 patches waiting for signoff or QA. 52 are from BibLibre (40%)
Yes, because I already informed Elliott that under no circumstances would I accept him QAing a patch by ByWater.
Displaying by "date of last change" (ie= there's been no activity on this bug since...) * August= 9 patches: 5 BibLibre, 1 OSSlab, 1 ByWater, 3 others * September = 8 patches: 4 BibLibre, 3 ByWater, 1 other * October = 27 patches: 14 BibLibre, 3 ACPL, 6 ByWater, 4 others (All of them are Enhancements or New Feature)
At the end, my opinion is that it's a little bit unfair (and I restart a long-standing discussion/complain...) BibLibre does a huge of effort to submit all his development, and be quick when there is feedback.
The very bad side effect of delays is that we loose a lot of energy in rebasing, just because there's no feedback from anyone, and when someone step-in the patch does not apply any more.
We fully accept the rule for not self-signing the patch, because, functionally speaking, I agree that it's good to have an external eye. (and that's more important with large features)
But the QA is a *technical* check of the code. If something is wrong (ie don't respect our guidelines, has a bad side effect, ...), as QAer, I'll do *exactly* the same thing whoever the patch come from.
If I were suspicious, I could even say that you imply that, when Jonathan or me QA a patch from BibLibre, we're biased, and I could be upset by your suspicion (I'm not, I'm just very sad that this discussion started again, while I thought it was solved)
I never "promote" BibLibre patches, I always QA by date of last change, ie: I QA the older patch without activity. Yesterday, I QAed something like 10 patches, iirc, 2 from BibLibre failed QA (including one just because there was some PODDOC missing !)
Chris-es are proposing what could be a fair rule imo = "if no one step up". What could be considered as "no one step up" being the next question...
Agreed. At some point there may be no choice. I'm pretty sure we have had a backlog in the Signed Off queue for a long time, possibly since always (Chris has a nice graph somewhere, but I don't know where). In light of that, I think two months would be reasonable. Also, it's worth pointing out that we have a QA *team*. If there's a BibLibre patch that's been signed off that isn't getting QAed, it is absolutely reasonable for you to e-mail the koha-devel list a message something like: "[QA] Does anyone have time to review poor benighted bug XXXX?" There's no guarantee someone will, but a list of forty bugs clamoring for attention is daunting, and with no particular reason for choosing one over the other, the chance of another QAA choosing to review your particular bug is very slight. On the other hand, if you ask, someone just might. I've been known to repeatedly overlook patches that provide functionality that I want when looking for a patch to sign off, either because the list is just too long or because I don't recognize what the patch does from the description (for example, at one point I was going to try testing the non-linear updates, but thought it was already signed off because I didn't see the patch in the "Needs Signoff" list... and that went on for months; it was only when chris_n signed off on the patch that I realized the bug was called "updatedatabase improvements" and not "non-linear updates" or the like). Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
Paul,
Am I right if I say that BibLibre will be the company mostly concerned impacted by this rule ? Checking numbers. Some numbers: there are 131 patches waiting for signoff or QA. 52 are from BibLibre (40%) Yes, because I already informed Elliott that under no circumstances would I accept him QAing a patch by ByWater. You're the RM, so you decide, but I think it's not a good decision (and I've explained many times why I think that: QA is technical-oriented. For now, I trust Elliott to do QA exactly in the same way for ByWater
Le 29/11/2012 16:59, Jared Camins-Esakov a écrit : patches than he will do for others). However, I can promise Jonathan and me will quickly take care of what's waiting for QA from outside BibLibre.
Agreed. At some point there may be no choice. I'm pretty sure we have had a backlog in the Signed Off queue for a long time, possibly since always (Chris has a nice graph somewhere, but I don't know where). Not since always, the "need QA" queue has been empty for weeks before my summer holiday.
In light of that, I think two months would be reasonable. 2 months ? That's really a long period, considering we've a 6 months release cycle, with Feature & String Freeze. That let only a short time for New Features and Enhancements I was thinking of 2 ... weeks. Let's cut the fruit in two (frenchism ?) and say one month ?
Also, it's worth pointing out that we have a QA *team*. If there's a BibLibre patch that's been signed off that isn't getting QAed, it is absolutely reasonable for you to e-mail the koha-devel list a message something like: "[QA] Does anyone have time to review poor benighted bug XXXX?" By default (at least for me), the "signed off" list is organised by date. I've the assignee & changed column in the result list, it's trivial to see who did what, and when it has been signed-off. I can ask on the list, and will do, but I think it should not be necessary. (and for now, i'll head to QA non-biblibre old patches) -- Paul POULAIN - Associé-gérant Tel : (33) 4 91 81 35 08 http://www.biblibre.com Logiciels Libres pour les bibliothèques et les centres de documentation
On Thu, Nov 29, 2012 at 11:20 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
light of that, I think two months would be reasonable. In 2 months ? That's really a long period, considering we've a 6 months release cycle, with Feature & String Freeze. That let only a short time for New Features and Enhancements I was thinking of 2 ... weeks. Let's cut the fruit in two (frenchism ?) and say one month ?
+1 for one month. Large change sets become hard to rebase quickly.
Also, it's worth pointing out that we have a QA *team*. If there's a BibLibre patch that's been signed off that isn't getting QAed, it is absolutely reasonable for you to e-mail the koha-devel list a message something like: "[QA] Does anyone have time to review poor benighted bug XXXX?" By default (at least for me), the "signed off" list is organised by date. I've the assignee & changed column in the result list, it's trivial to see who did what, and when it has been signed-off.
Here again, I would propose some sort of QA guideline which requires the oldest, highest severity, signed-off patchs to be QA'd first. This would seem to lay to rest the entire issue of patches becoming poor and/or benighted. :-) Kind Regards, Chris
Paul,
Agreed. At some point there may be no choice. I'm pretty sure we have
had a backlog in the Signed Off queue for a long time, possibly since always (Chris has a nice graph somewhere, but I don't know where). Not since always, the "need QA" queue has been empty for weeks before my summer holiday.
Would that it were so now!
In
light of that, I think two months would be reasonable. 2 months ? That's really a long period, considering we've a 6 months release cycle, with Feature & String Freeze. That let only a short time for New Features and Enhancements I was thinking of 2 ... weeks. Let's cut the fruit in two (frenchism ?) and say one month ?
Fair enough. 1 month sounds reasonable to me. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
Le 29/11/2012 17:37, Jared Camins-Esakov a écrit :
Not since always, the "need QA" queue has been empty Would that it were so now!
I'm on it ;-) -- Paul POULAIN - Associé-gérant Tel : (33) 4 91 81 35 08 http://www.biblibre.com Logiciels Libres pour les bibliothèques et les centres de documentation
participants (6)
-
Chris Cormack -
Chris Nighswonger -
Jared Camins-Esakov -
Katrin Fischer -
Marcel de Rooy -
Paul Poulain