Hello koha-devel, I have some points to rise about our release process. Koha 3.8 has been released 2.5 months ago, and some libraries have reported major problems with 3.8.0. The main origin of those problems came from the fact that a large feature (hourly loans) has been pushed close to the release. It works well, but introduced many side effect: changing the issuing date field to a datetime field broke many other features because some comparison became wrong. When Koha 3.4 has been released, we faced a similar problem, with the switch to T::T that caused many problems too (badly scoped variables in loops, translations problems) Those problems are solved quickly, but libraries that installed Koha 3.X.0 in production faced troubles. So I think we must refine our workflow. There are 2 options here: * stop pushing enhancements that are potentially harmful sooner (2 months before the release ?). What could be harmful is the RM decision. That would make 4 months for pushing ENH, and 2 months for testing. * Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. I must say that I prefer, the 2nd option: contributors are concentrated on stabilizing 3.8, and, on the previous release, a lot of ENH have been signed off close to the release. It look like we had 2 periods: the 1st one where our strength are dedicated to stable, the 2nd one being dedicated to the next release. What is also good with the 2nd option is that it does not change a lot the current timeline and workflow. It's just a matter of announcing things. Your opinion ? PS: I must also add that, independently from this community decision, BibLibre plan to deploy most customers in 3.X.2, and I've been told by Brendan that bywatersolutions will probably wait for .3 or .4. Both of our companies having a few libraries accepting to heavily test releases early. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal what one could already suspect about a 3.X.0 release.. No real change in workflow.. (3.X.1 being stable might be nice too :-)
Marcel
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal what one could already suspect about a 3.X.0 release.. No real change in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4. But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide. I think this is also a good time to start thinking about 3.12.x http://wiki.koha-community.org/wiki/Roles_for_3.12 Chris -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
Hi all, I agree with Chris on option 1). We should always aim for the most stable release possible. Part of that is to add more automated tests and to get our human testing better organized. I think releasing a beta will mean that we will end up pushing risky things to that version, because there will still be time to fix it. So it will take even longer, until we reach a stable release. Also, if something is named "beta" or "RC", I think we are unlikely to get libraries to use and test it. So it looks to me like we would be losing time and losing 2 releases that could be great. Going into feature freeze earlier like suggested as option 1) is only about announcing too - it requires no change at all, not even a change in naming. And whenever we announce freezes - it will always be too early for some features and things we wanted to do for this next version. Katrin
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel- bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Thursday, July 12, 2012 12:22 PM To: Marcel de Rooy Cc: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] About Release process
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal +what one could already suspect about a 3.X.0 release.. No real change +in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4.
But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide.
I think this is also a good time to start thinking about 3.12.x
http://wiki.koha-community.org/wiki/Roles_for_3.12
Chris
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
I'm mostly in favour of solution 1 (in that it's close to what I think we should to, and solution 2 is not good in my opinion). But I think there is a little more to it. There are several factors in our current workflow and environment that are causing these bugs and regressions: - *Lack of robust testing suites: * All testing is done at the discretion of the sign-offer and QA team, to the best of their ability at the time. But, any 1 person is bound to miss some aspects of even an only mildly-complex patch, either by not testing a fringe case, or not thinking of another workflow that's not common to their experience (a quote123 problem) - *Enigmatic codebase:* Koha has grown organically for the last 12 years. In that time, we've had not only numerous release managers and sponsoring libraries, but a significant change in the technologies underlying web services. We are now stumbling over the anachronisms and shortcuts that have gotten us this far - *Limited personnel*: There are only a few dozen people in the world who really, really know the Koha codebase. These folks are the best suited to do testing and catch bugs, but they also tend to be the ones doing the development, migrations and support for libraries throughout the world. It won't matter how clear and clean our procedures are if there aren't enough people to process them. - *Pressure to "leave no patch behind"*: we don't want to drop patches, but sometimes when a patch (particularly a large one) has gone untested or un-QAed for a long while, it's swept forward anyway in an attempt to clean out the queue. Even if someone's code is structurally sound in and of itself, the way it interacts with the rest of the code can result in bugs. Or, more insidiously, it introduces yet another complexity to the code that makes maintenance that much harder down the road. - *A "we can fix it later" mentality*: several large feature get pushed through the door without being completely ready to go. The idea is that we just need to get them in, make them part of master, and then we can fix them later. And that's exactly what we're doing. Rather than adjusting our numbering and labeling practices, or adjusting the dates on feature freezes, I think we need to focus on resolving the above issues. Treat the problem, not the symptoms, as it were. It's not going to be as simple as the proposed solutions earlier in this thread, but it will lead us to a more solid, stable and extensible ILS long into the future. Cheers, -Ian On Thu, Jul 12, 2012 at 6:58 AM, Fischer, Katrin <Katrin.Fischer@bsz-bw.de>wrote:
Hi all,
I agree with Chris on option 1).
We should always aim for the most stable release possible. Part of that is to add more automated tests and to get our human testing better organized.
I think releasing a beta will mean that we will end up pushing risky things to that version, because there will still be time to fix it. So it will take even longer, until we reach a stable release. Also, if something is named "beta" or "RC", I think we are unlikely to get libraries to use and test it. So it looks to me like we would be losing time and losing 2 releases that could be great.
Going into feature freeze earlier like suggested as option 1) is only about announcing too - it requires no change at all, not even a change in naming. And whenever we announce freezes - it will always be too early for some features and things we wanted to do for this next version.
Katrin
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel- bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Thursday, July 12, 2012 12:22 PM To: Marcel de Rooy Cc: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] About Release process
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal +what one could already suspect about a 3.X.0 release.. No real change +in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4.
But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide.
I think this is also a good time to start thinking about 3.12.x
http://wiki.koha-community.org/wiki/Roles_for_3.12
Chris
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Nothing to say against that, of course :) Option 1 is/has been our goal, but what actually happened was option 2.. Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Ian Walls Verzonden: donderdag 12 juli 2012 13:50 Aan: Fischer, Katrin CC: koha-devel@lists.koha-community.org; Marcel de Rooy Onderwerp: Re: [Koha-devel] About Release process I'm mostly in favour of solution 1 (in that it's close to what I think we should to, and solution 2 is not good in my opinion). But I think there is a little more to it. There are several factors in our current workflow and environment that are causing these bugs and regressions: * Lack of robust testing suites: All testing is done at the discretion of the sign-offer and QA team, to the best of their ability at the time. But, any 1 person is bound to miss some aspects of even an only mildly-complex patch, either by not testing a fringe case, or not thinking of another workflow that's not common to their experience (a quote123 problem) * Enigmatic codebase: Koha has grown organically for the last 12 years. In that time, we've had not only numerous release managers and sponsoring libraries, but a significant change in the technologies underlying web services. We are now stumbling over the anachronisms and shortcuts that have gotten us this far * Limited personnel: There are only a few dozen people in the world who really, really know the Koha codebase. These folks are the best suited to do testing and catch bugs, but they also tend to be the ones doing the development, migrations and support for libraries throughout the world. It won't matter how clear and clean our procedures are if there aren't enough people to process them. * Pressure to "leave no patch behind": we don't want to drop patches, but sometimes when a patch (particularly a large one) has gone untested or un-QAed for a long while, it's swept forward anyway in an attempt to clean out the queue. Even if someone's code is structurally sound in and of itself, the way it interacts with the rest of the code can result in bugs. Or, more insidiously, it introduces yet another complexity to the code that makes maintenance that much harder down the road. * A "we can fix it later" mentality: several large feature get pushed through the door without being completely ready to go. The idea is that we just need to get them in, make them part of master, and then we can fix them later. And that's exactly what we're doing. Rather than adjusting our numbering and labeling practices, or adjusting the dates on feature freezes, I think we need to focus on resolving the above issues. Treat the problem, not the symptoms, as it were. It's not going to be as simple as the proposed solutions earlier in this thread, but it will lead us to a more solid, stable and extensible ILS long into the future. Cheers, -Ian On Thu, Jul 12, 2012 at 6:58 AM, Fischer, Katrin <Katrin.Fischer@bsz-bw.de<mailto:Katrin.Fischer@bsz-bw.de>> wrote: Hi all, I agree with Chris on option 1). We should always aim for the most stable release possible. Part of that is to add more automated tests and to get our human testing better organized. I think releasing a beta will mean that we will end up pushing risky things to that version, because there will still be time to fix it. So it will take even longer, until we reach a stable release. Also, if something is named "beta" or "RC", I think we are unlikely to get libraries to use and test it. So it looks to me like we would be losing time and losing 2 releases that could be great. Going into feature freeze earlier like suggested as option 1) is only about announcing too - it requires no change at all, not even a change in naming. And whenever we announce freezes - it will always be too early for some features and things we wanted to do for this next version. Katrin
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org<mailto:koha-devel-bounces@lists.koha-community.org> [mailto:koha-devel-<mailto:koha-devel-> bounces@lists.koha-community.org<mailto:bounces@lists.koha-community.org>] On Behalf Of Chris Cormack Sent: Thursday, July 12, 2012 12:22 PM To: Marcel de Rooy Cc: koha-devel@lists.koha-community.org<mailto:koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] About Release process
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl<mailto:M.de.Rooy@rijksmuseum.nl>) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal +what one could already suspect about a 3.X.0 release.. No real change +in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4.
But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide.
I think this is also a good time to start thinking about 3.12.x
http://wiki.koha-community.org/wiki/Roles_for_3.12
Chris
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238<tel:%2B64%204%20803%202238> PO Box 11-053, Manners St, Wellington 6142, New Zealand
Koha-devel mailing list Koha-devel@lists.koha-community.org<mailto:Koha-devel@lists.koha-community.org> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
+1 - I think Ian made a lot of good points in his mail And I also agree with Marcel, I only think we should stop it from happening in the first place J Katrin From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Marcel de Rooy Sent: Thursday, July 12, 2012 1:56 PM To: Ian Walls; koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] About Release process Nothing to say against that, of course :) Option 1 is/has been our goal, but what actually happened was option 2.. Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Ian Walls Verzonden: donderdag 12 juli 2012 13:50 Aan: Fischer, Katrin CC: koha-devel@lists.koha-community.org; Marcel de Rooy Onderwerp: Re: [Koha-devel] About Release process I'm mostly in favour of solution 1 (in that it's close to what I think we should to, and solution 2 is not good in my opinion). But I think there is a little more to it. There are several factors in our current workflow and environment that are causing these bugs and regressions: * Lack of robust testing suites: All testing is done at the discretion of the sign-offer and QA team, to the best of their ability at the time. But, any 1 person is bound to miss some aspects of even an only mildly-complex patch, either by not testing a fringe case, or not thinking of another workflow that's not common to their experience (a quote123 problem) * Enigmatic codebase: Koha has grown organically for the last 12 years. In that time, we've had not only numerous release managers and sponsoring libraries, but a significant change in the technologies underlying web services. We are now stumbling over the anachronisms and shortcuts that have gotten us this far * Limited personnel: There are only a few dozen people in the world who really, really know the Koha codebase. These folks are the best suited to do testing and catch bugs, but they also tend to be the ones doing the development, migrations and support for libraries throughout the world. It won't matter how clear and clean our procedures are if there aren't enough people to process them. * Pressure to "leave no patch behind": we don't want to drop patches, but sometimes when a patch (particularly a large one) has gone untested or un-QAed for a long while, it's swept forward anyway in an attempt to clean out the queue. Even if someone's code is structurally sound in and of itself, the way it interacts with the rest of the code can result in bugs. Or, more insidiously, it introduces yet another complexity to the code that makes maintenance that much harder down the road. * A "we can fix it later" mentality: several large feature get pushed through the door without being completely ready to go. The idea is that we just need to get them in, make them part of master, and then we can fix them later. And that's exactly what we're doing. Rather than adjusting our numbering and labeling practices, or adjusting the dates on feature freezes, I think we need to focus on resolving the above issues. Treat the problem, not the symptoms, as it were. It's not going to be as simple as the proposed solutions earlier in this thread, but it will lead us to a more solid, stable and extensible ILS long into the future. Cheers, -Ian On Thu, Jul 12, 2012 at 6:58 AM, Fischer, Katrin <Katrin.Fischer@bsz-bw.de> wrote: Hi all, I agree with Chris on option 1). We should always aim for the most stable release possible. Part of that is to add more automated tests and to get our human testing better organized. I think releasing a beta will mean that we will end up pushing risky things to that version, because there will still be time to fix it. So it will take even longer, until we reach a stable release. Also, if something is named "beta" or "RC", I think we are unlikely to get libraries to use and test it. So it looks to me like we would be losing time and losing 2 releases that could be great. Going into feature freeze earlier like suggested as option 1) is only about announcing too - it requires no change at all, not even a change in naming. And whenever we announce freezes - it will always be too early for some features and things we wanted to do for this next version. Katrin
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel- bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Thursday, July 12, 2012 12:22 PM To: Marcel de Rooy Cc: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] About Release process
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal
+what one could already suspect about a 3.X.0 release.. No real change +in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4.
But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide.
I think this is also a good time to start thinking about 3.12.x
http://wiki.koha-community.org/wiki/Roles_for_3.12
Chris
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 <tel:%2B64%204%20803%202238> PO Box 11-053, Manners St, Wellington 6142, New Zealand
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Salvete! Perhaps no one will notice that I'm commenting from the wrong side of the marae. >.> <.< I know I tell this story all the bloody time, and I realise that this is the Developers' list so imagine me ducking rotten veg. I agree with Ian's observations. Also, he spelt "favour" correctly :) An Unacceptable Wait: A Cautionary Tale A long time ago, when animals could talk, I heard this tale from an older Librarian. This Librarian had a daughter. About the time that daughter was born, she asked her proprietary vendor to please code a bookmobile plug in. This was a keen Librarian. I would say that specifications were probably not an issue here. We Librarians are patient. Our fur is not so shiny as otter's. We are not so fast as horse. But we are patient. That said, every now and then, the Librarian would go back to the proprietary vendor and ask after this bookmobile plug in. "Oh no, we aren't done yet, you will just have to wait." So the Librarian did. Every now and then, she'd politely (and after many, many moons perhaps not so politely) inquire with the great white proprietary father about her plug in. The answer was always the same. Perhaps she was not praying hard enough. After a very long wait indeed, she was finally told that it was ready. It of course didn't do what it was meant to actually do. But, it was ready. "But Brooke what of her daughter?" You might ask. When the plug in was released, her daughter was married to a handsome man from the next village over. So. THAT is an unacceptable wait in Libraryland. Surely there is summat between that and our release cycle that can be settled on. At first, six months was a matter of pride. We had taken a very long time on a given release. Everyone got frustrated. This was bad. We all agreed. It was great to get back to six months just to show the world that we could. I have been very nervous about the 6 month cycle, and I have said several times that it could easily be 9 months or even a year. I worry enormously about Developer burnout. I also will say that I worry a whole lot about Library funding to upgrade every bloody year if one skips a version. Not too terribly long ago, it was mentioned that perhaps we should consider alternating feature releases and bug fixing releases. Perhaps function should dictate the schedule. You all might feel like there's this giant pressure on you to do things and do them RIGHT NOW but as long as you tell us when you first sit down with us how long you think it will take, and why it might be taking longer if it is taking longer, perhaps this pressure wouldn't be there. It is the old good, fast, or cheap you can have any two equation. I would pick good and cheap. Anyway, I will shut up now and stop invading your listserv. Cheers, Brooke
I think a recruitment and training effort by the Koha community targeting potential programmers should be considered; I do understand the decentralized structure of the open-source community might make this difficult. Below I suggest one potential candidate pool of Koha developers. I have an educational background that includes programming, but my general work history has been such that I have tended more towards network administration and management, so that my coding skills are rusty. If I had the time I probably could go back and figure out the Koha structure and start doing some development, but I have so much other stuff to do that there's no way this will happen. However, at any given point in time I have 1-3 part-time tech assistants working for me, typically college computer science students that are passable programmers, and who actually do some programming here at work. I usually have enough hours available for them that I could have them working on Koha, maybe not designing new, novel features from scratch, but certainly coding simpler tasks, bug swatting or signoffs. The problem is the learning curve. These students work for me from 1-3 years until they graduate, and while they may be perfectly capable of writing a bubble up-and-down sort, a comprehensive understanding of a large application is outside their experience base. Me sitting down and working with them to do a dozen trial Koha installs and working with git and going over the documentation and then coding something is just too much of a time challenge and learning curve for all of us. If it were possible, however, to have a training class, perhaps 15-40 hours, covering beginning Koha development, that might be a truly great thing. They don't need perl training, they pick that up quite easily. Same with databases--my last two assistants have been very good with MySql, and one was a real whiz with phpmyadmin. <alert-political/religious statement follows>But they don't immediately, intuitively understand an ILS, a simple thing like a database and an interface to the database that librarians have taken and cluttered up with arcane terminology and convoluted "stuff" so that they could call it an ILS and demand that only MLS degree holders get to have jobs and the title of Librarian and get to touch the ILS. </alert> It is frequently to my great amusement that I recall one of my military assignments working in a large database center where the lowest ranking individuals, several hundred of them I suppose, were the ones poking the database (cataloging), and working on terminals connected to a Burroughs mainframe database with no modern GUI. Ideally it would be great to have this training I suggest in person, but that implies attendance at a conference where there are instructors, and that is a problem for the tech's part-time work hours, and Koha conferences are infrequent and geographically problematic. On site training would not be cost effective. Live video conference training might be possible. I'm much less enthusiastic about online training than the world at large, but reluctantly concede that may be the most feasible option. I'm well aware that vendors are sometimes stretched thin and that training materials need frequent updating, so this would be a challenge. On the client side money is always a consideration, but I probably have enough in my budget that I could pay for a <em>modestly priced</em> class for my techs. A potential side benefit for instructors/vendors is that promising future employees could be identified. Well, that's my input for the morning, now I need to go buy some plywood and glue it to the wall and attach my three T1 lines to it so they don't fall out of the ceiling and destroy the peace and harmony of the Internet before we get our ethernet/fiber connection when we stand-up our new Technology Center. See why managers don't have time for Koha development? We have to build stuff. Greg Lawson Network Administrator Rolling Hills Consolidated Library 1912 N. Belt Highway St. Joseph, MO 64506 ------------------------------- On 07/12/2012 06:50 AM, Ian Walls wrote:
-----------------snip----------------------
* *Limited personnel*: There are only a few dozen people in the world who really, really know the Koha codebase. These folks are the best suited to do testing and catch bugs, but they also tend to be the ones doing the development, migrations and support for libraries throughout the world. It won't matter how clear and clean our procedures are if there aren't enough people to process them
-------------------/snip--------------------------
* --
* glaws (glawson@rhcl.org) wrote:
I think a recruitment and training effort by the Koha community targeting potential programmers should be considered; I do understand the decentralized structure of the open-source community might make this difficult. Below I suggest one potential candidate pool of Koha developers.
<snip lots of interesting stuff> This is a great idea, at Catalyst as you know we run the Open Source Academy. And we also participate in the summer of tech, Chris Hall, who most of you will know, got into Koha development throught the summer of tech project. And we have Template::Toolkit support in Koha in a large part thanks to him. These kind of schemes work very well. A while ago after Kohacon11 Koustubha Kale floated the idea of a Koha Academy, and even got as far as setting up a moodle instance to start it. What an idea needs is someone with the time, or a team of people with the time to coordinate and manage it. And now we are back where we started, finding some people to do that... Chris -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
I think I too side with option 1. As much as I would enjoy googleizing our naming scheme and adding beta to the end of every initial release I don't think it is practical for our product. While enhancements are an important part of Koha I don't believe we should put them ahead of its stability. On Thu, Jul 12, 2012 at 5:58 AM, Fischer, Katrin <Katrin.Fischer@bsz-bw.de>wrote:
Hi all,
I agree with Chris on option 1).
We should always aim for the most stable release possible. Part of that is to add more automated tests and to get our human testing better organized.
I think releasing a beta will mean that we will end up pushing risky things to that version, because there will still be time to fix it. So it will take even longer, until we reach a stable release. Also, if something is named "beta" or "RC", I think we are unlikely to get libraries to use and test it. So it looks to me like we would be losing time and losing 2 releases that could be great.
Going into feature freeze earlier like suggested as option 1) is only about announcing too - it requires no change at all, not even a change in naming. And whenever we announce freezes - it will always be too early for some features and things we wanted to do for this next version.
Katrin
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel- bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Thursday, July 12, 2012 12:22 PM To: Marcel de Rooy Cc: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] About Release process
* Marcel de Rooy (M.de.Rooy@rijksmuseum.nl) wrote:
Hi Paul, all,
* Release the 3.X.0 saying it's a beta, could have some bugs not detected. The 3.X.1 being a RC, and the 3.X.2 being the 1st really stable version. +1 for this second option. You could say that it makes more formal +what one could already suspect about a 3.X.0 release.. No real change +in workflow.. (3.X.1 being stable might be nice too :-)
I think I prefer the first option, or at least that is what we should be aiming for. Aiming for a stable release at the .1 or .2 level will mean it wont be stable until .3 or .4.
But as the elected release manager for 3.10.0 it's your call, and I will work with whatever you decide.
I think this is also a good time to start thinking about 3.12.x
http://wiki.koha-community.org/wiki/Roles_for_3.12
Chris
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Elliott Davis Development Support Specialist Elliott@bywatersolutions.com 903.571.4257
On Thu, Jul 12, 2012 at 06:54:36AM -0500, Elliott Davis wrote:
I think I too side with option 1.
Releases are inherently evil things, (like all deadlines). There is a temptation always to pack stuff in there to make a release an event. I dont think having beta releases helps ( or skipping .1 releases in deployment) because a lot of that testing just does not occur until the release goes out into the world no matter how good our intentions are. How can we improve things, well any big changes should go into master sooner rather than later. The longer big changes are promised the more pressure the RM is to ship them. The key task for the RM is really deciding what's not yet been proved, and should not make this iteration. We do need lots more tests. We rely on far too much untested functionality (which may mean we're wrong in our assumptions of what it does) We have a specific problem that it is much easier to add bits of functionality to the system, bits that up the level of entropy in the code base, rather than make the strategic changes that build reliability into core. A specific problem is that we tend to test functionality of an enhancement not necessarily how that enhancement integrates with the eco-system of Koha and its these kind of conflicts that tend not to surface until its been deployed in the real world. Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com
Weighing in... I prefer option 1. A specific problem is that we tend to test functionality of an
enhancement not necessarily how that enhancement integrates with the eco-system of Koha and its these kind of conflicts that tend not to surface until its been deployed in the real world.
This is a huge issue, and while it is not entirely solvable, we can definitely do *better*. Part of the problem is conflicting expectations, which we can't automate out of existence, but part of the problem is regression, plain and simple. One of the lowest impact ways to reduce regression is automated regression testing, which we can do, for example, with WWW::Mechanize. The tests are dead simple to write, and fantastically useful. There's even a wiki page about interface testing using WWW::Mechanize: http://wiki.koha-community.org/wiki/Interface_testing_with_WWW::Mechanize Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
Jared:
Weighing in...
I prefer option 1.
Me too. As soon as x.x.0 comes out, libraries start asking for upgrades, we exist to provide services they want, so we'd like to give them that. It's already enough to explain the versioning scheme, without having numbered releases that aren't usable in the real world. I agree about the Mechanize and freeze comments: anyone got funding to make those happen? ;-) More later, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and library systems developer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire (including development) at http://www.software.coop/
Hello - We have recently hired another Developer (Elliot Davis - libsysguy ;) ), who is charged with creating a more integrated testing suite for ByWater and getting/challenging some of our partners to complete testing that isn't automated already or would be difficult to automate. As for the options... I totally agree with Colin that the releases are an Event and should be promoted as an exciting EVENT. That being said, I would not want to delay any major new features being put into a new release no matter the time (we can never really fight the timing of things). I think my thought and my response is that We (ByWater at least) need to do better testing (what does that mean - well we are still working on that). I think the thing that I struggle with is the definition of a "Freeze"... Please speak up and correct me if I am wrong here. This is what I think it means currently - if something has an enhancement bug listed (i.e. the idea is out there before the freeze date) - then it could still be included as long as the developer gets the code onto the bug report sometime around the freeze date. What I think should happen - The code should be pushed to master before the freeze date - nothing new after the freeze date.... (I hope that makes sense - with what I am trying to explain). But as others have said - this is really up to the RM. Here's a hint - whomever is going to be the next RM, please include a definition of "Freeze" for me ;) Thanks, Brendan On Thu, Jul 12, 2012 at 8:43 AM, Colin Campbell < colin.campbell@ptfs-europe.com> wrote:
On Thu, Jul 12, 2012 at 06:54:36AM -0500, Elliott Davis wrote:
I think I too side with option 1.
Releases are inherently evil things, (like all deadlines). There is a temptation always to pack stuff in there to make a release an event. I dont think having beta releases helps ( or skipping .1 releases in deployment) because a lot of that testing just does not occur until the release goes out into the world no matter how good our intentions are. How can we improve things, well any big changes should go into master sooner rather than later. The longer big changes are promised the more pressure the RM is to ship them. The key task for the RM is really deciding what's not yet been proved, and should not make this iteration. We do need lots more tests. We rely on far too much untested functionality (which may mean we're wrong in our assumptions of what it does) We have a specific problem that it is much easier to add bits of functionality to the system, bits that up the level of entropy in the code base, rather than make the strategic changes that build reliability into core. A specific problem is that we tend to test functionality of an enhancement not necessarily how that enhancement integrates with the eco-system of Koha and its these kind of conflicts that tend not to surface until its been deployed in the real world. Colin
-- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2
http://www.ptfs-europe.com _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- --------------------------------------------------------------------------------------------------------------- Brendan A. Gallagher ByWater Solutions CEO Headquarters: Santa Barbara, CA
Brendan, I think the thing that I struggle with is the definition of a "Freeze"...
Please speak up and correct me if I am wrong here. This is what I think it means currently - if something has an enhancement bug listed (i.e. the idea is out there before the freeze date) - then it could still be included as long as the developer gets the code onto the bug report sometime around the freeze date. What I think should happen - The code should be pushed to master before the freeze date - nothing new after the freeze date.... (I hope that makes sense - with what I am trying to explain). But as others have said - this is really up to the RM. Here's a hint - whomever is going to be the next RM, please include a definition of "Freeze" for me ;)
For what it's worth, I updated my proposal to make clear (I think) what sorts of patches will be considered when. http://wiki.koha-community.org/wiki/Jcamins_Proposal_for_RM-3.12 Exact time periods are subject to change, based on feedback. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
Le 12/07/2012 10:53, Paul Poulain a écrit :
Hello koha-devel, I have some points to rise about our release process.
OK, we spoke of this during our IRC meeting. I also spoke of this with Jared, that applied as RM for 3.12 (he's not elected yet, but I suspect he will be the only one fool enough to apply ;-) ) And a 3rd option has been risen, I think it's the one we will use. What I/we plan to do: * 2 months before the release (number N) = - declare small feature freeze: only bugfixes and small/non-core enhancement are pushed for the next release - branch the release on git. master goes his way for N+1, major ENH can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint * 1 month before the release = - declare string freeze: no more string changes to let translators do their work By "are pushed" I really mean the freeze is about *pushing* before the freeze. In previous releases, we've accepted some ENH provided they've been *submitted* before the freeze. That's an important change, be careful ! If we want stability, we need more time to test. There is another positive aspect with this refinement of the workflow = the documentation writers will have more time to write the documentation: 2 months before the release, they'll be sure there won't be major enhancements/change pushed. Last (but not least) point: * August is holiday time for Northern hemisphere, and oct22 minus 2 months = Aug 22, that's a problem. That's why I propose to switch to may/november for our releases. The timeline would be: * today => sept 22 => hack ! hack ! hack ! * sept 22 => feature freeze, no more large or core enhancements pushed (whatever the date of submission of the patch) * oct 22 => string freeze, no more string changes * nov 22 => Koha 3.10 (and I'm no more RM. Yippee !!! :D :D ) The only bad thing with this method is that it gives more work to the RM, that must push on 2 branches for 2 months. But for 2012-11 release, that's not a problem, as we will have 2 different RMs ;-) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
- branch the release on git. master goes his way for N+1, major ENH can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint
I'm not sure I understand this aspect of it. The branching is to allow enhancements to still be pushed somewhere while what-was-master is in feature freeze? I would think there is some benefit of not pushing any enhancements *anywhere* to encourage folks to be bug-fixing instead. Or do I misunderstand? -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org
Owen,
- branch the release on git. master goes his way for N+1, major ENH
can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint
I'm not sure I understand this aspect of it. The branching is to allow enhancements to still be pushed somewhere while what-was-master is in feature freeze? I would think there is some benefit of not pushing any enhancements *anywhere* to encourage folks to be bug-fixing instead.
That is correct. As RM presumptive (but perhaps someone else would like to be RM, and eliminate my presumptive status? There's still time :D ), my plan for this period would be to push any enhancements that might be submitted only after bugfixes have been dealt with. Ideally it will be possible to get all submitted enhancements pushed to Master during this period, but given the volume of patches that can precede a release, at my discretion I may delay large features until after the release of 3.10.0. Paul and I discussed this at length, actually. I originally suggested not pushing any large enhancements to master (i.e. 3.11) during the 3.10 feature freeze, but Paul (quite reasonably, I think) argued that not pushing enhancements that were sitting in the queue would not serve to get those developers working on bugfixes, but rather just result in a heavier burden on the RM for 3.12 once 3.10.0 was released. As a compromise, any enhancements that can be pushed will be, but anything that gets in the way of the more important bug fixing will not be pushed until 3.10.0 has left the nest. Hope this clarifies. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
Hi, I do have [some] doubts too on this double master-approach. If there is a feature-frozen old-master 3.9 branch and a new master 3.11 branch, where will most developers be spending their time? Will they be bug-fixing in 3.9 or rather go to 3.11 and send new patches? If new master has the most attraction, this move will not really work (who is testing?). If most people would stick to old master, why start new master in the first place? I just fear that many will go to new master.. Bugs fixed in 3.9 should go up to 3.11; this will be an additional effort. (Small side note: When patches are going up and down, workflow needs to be adjusted?) So, whatever we do, probably this old problem will always be there.. Much depends on the RM, and better testing :) Marcel Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Jared Camins-Esakov Verzonden: woensdag 18 juli 2012 18:51 Aan: Owen Leonard CC: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] About Release process Owen,
- branch the release on git. master goes his way for N+1, major ENH can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint
I'm not sure I understand this aspect of it. The branching is to allow enhancements to still be pushed somewhere while what-was-master is in feature freeze? I would think there is some benefit of not pushing any enhancements *anywhere* to encourage folks to be bug-fixing instead. That is correct. As RM presumptive (but perhaps someone else would like to be RM, and eliminate my presumptive status? There's still time :D ), my plan for this period would be to push any enhancements that might be submitted only after bugfixes have been dealt with. Ideally it will be possible to get all submitted enhancements pushed to Master during this period, but given the volume of patches that can precede a release, at my discretion I may delay large features until after the release of 3.10.0. Paul and I discussed this at length, actually. I originally suggested not pushing any large enhancements to master (i.e. 3.11) during the 3.10 feature freeze, but Paul (quite reasonably, I think) argued that not pushing enhancements that were sitting in the queue would not serve to get those developers working on bugfixes, but rather just result in a heavier burden on the RM for 3.12 once 3.10.0 was released. As a compromise, any enhancements that can be pushed will be, but anything that gets in the way of the more important bug fixing will not be pushed until 3.10.0 has left the nest. Hope this clarifies. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com<mailto:jcamins@cpbibliography.com> (web) http://www.cpbibliography.com/
Hi all, I am also worried about this. I agree with Owen and Marcel. Who will be testing and who will be fixing bugs? I think the problem will only get moved a bit in time, but it will still be there. Both branches will diverge quickly as soon as patches are only pushed to one of them. Then we will start to need different patches for bug fixes, patches will not apply, testers will have to test on each branch separately etc. And if we decide to only push bug fixes to both branches at once, where is the benefit of having 2 branches? We already have problems with testing - why make this even harder? I am also not persuaded moving the release date is a good idea, because we switched to time based releases for having reliable release dates and being able to plan. I also fear by moving it to November, we will move it right into the "end of year madness". Also we will maybe have the same problem again with a RM from a different country/continent. So if we decide to move the dates, please let us stick to those forever. Katrin -----Ursprüngliche Nachricht----- Von: koha-devel-bounces@lists.koha-community.org im Auftrag von Marcel de Rooy Gesendet: Do 19.07.2012 10:48 An: koha-devel@lists.koha-community.org Betreff: Re: [Koha-devel] About Release process Hi, I do have [some] doubts too on this double master-approach. If there is a feature-frozen old-master 3.9 branch and a new master 3.11 branch, where will most developers be spending their time? Will they be bug-fixing in 3.9 or rather go to 3.11 and send new patches? If new master has the most attraction, this move will not really work (who is testing?). If most people would stick to old master, why start new master in the first place? I just fear that many will go to new master.. Bugs fixed in 3.9 should go up to 3.11; this will be an additional effort. (Small side note: When patches are going up and down, workflow needs to be adjusted?) So, whatever we do, probably this old problem will always be there.. Much depends on the RM, and better testing :) Marcel Van: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] Namens Jared Camins-Esakov Verzonden: woensdag 18 juli 2012 18:51 Aan: Owen Leonard CC: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] About Release process Owen,
- branch the release on git. master goes his way for N+1, major ENH can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint
I'm not sure I understand this aspect of it. The branching is to allow enhancements to still be pushed somewhere while what-was-master is in feature freeze? I would think there is some benefit of not pushing any enhancements *anywhere* to encourage folks to be bug-fixing instead. That is correct. As RM presumptive (but perhaps someone else would like to be RM, and eliminate my presumptive status? There's still time :D ), my plan for this period would be to push any enhancements that might be submitted only after bugfixes have been dealt with. Ideally it will be possible to get all submitted enhancements pushed to Master during this period, but given the volume of patches that can precede a release, at my discretion I may delay large features until after the release of 3.10.0. Paul and I discussed this at length, actually. I originally suggested not pushing any large enhancements to master (i.e. 3.11) during the 3.10 feature freeze, but Paul (quite reasonably, I think) argued that not pushing enhancements that were sitting in the queue would not serve to get those developers working on bugfixes, but rather just result in a heavier burden on the RM for 3.12 once 3.10.0 was released. As a compromise, any enhancements that can be pushed will be, but anything that gets in the way of the more important bug fixing will not be pushed until 3.10.0 has left the nest. Hope this clarifies. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com<mailto:jcamins@cpbibliography.com> (web) http://www.cpbibliography.com/
Le 19/07/2012 11:12, Fischer, Katrin a écrit :
Hi all, Hi all,
Who will be testing and who will be fixing bugs? I think the problem will only get moved a bit in time, but it will still be there. Both branches will diverge quickly as soon as patches are only pushed to one of them. Then we will start to need different patches for bug fixes, patches will not apply, testers will have to test on each branch separately etc. I've been unclear. What we plan to do with Jared is pushing patches on master, and also pushing bugfixes/non-core enhancements to 3.10 branch.
That's the workflow we already have for the stable release. The idea is to branch 3.10 soon to ensure the best possible stability. We will encourage developers to test things against master, we (the RMs) will take care of deciding if something is to push against master only or against master and 3.10 I think we can say this workflow is tried for 3.10, we will decide after the release if it is adopted for the next releases. I'll announce the updated timing in my next RM newsletter (that i'll publish tomorrow, before leaving for vacation) * today => sept 22 => development process * sept 22 => feature freeze, no more large or core enhancements pushed, whatever the date of submission of the patch. 3.10 version is branched in git. * oct 22 => string freeze, no more string changes * nov 22 => Koha 3.10.0 is released
I am also not persuaded moving the release date is a good idea, because we switched to time based releases for having reliable release dates and being able to plan. I also fear by moving it to November, we will move it right into the "end of year madness". Also we will maybe have the same problem again with a RM from a different country/continent. So if we decide to move the dates, please let us stick to those forever. In French we have an expression that is "you must choose your poison my friend". It means there is no "good" option, you just must choose the less annoying one. Moving by 1 month is the best option I can see. Could the release of a 3.10.0RC1 -or 3.9.99 the numbering doesn't matter-, on Oct 22th please you ? That's an option I could deal with.
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On 2 August 2012 20:46, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 19/07/2012 11:12, Fischer, Katrin a écrit :
Hi all, Hi all,
Moving by 1 month is the best option I can see. Could the release of a 3.10.0RC1 -or 3.9.99 the numbering doesn't matter-, on Oct 22th please you ? That's an option I could deal with.
So the question is, do we now have a 7th month cycle? Or is it just this once? Is the next release a 5 month one, or are we back to 6? Chris
On Thu, Aug 2, 2012 at 4:55 AM, Chris Cormack <chris@bigballofwax.co.nz> wrote:
On 2 August 2012 20:46, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 19/07/2012 11:12, Fischer, Katrin a écrit :
Hi all, Hi all,
Moving by 1 month is the best option I can see. Could the release of a 3.10.0RC1 -or 3.9.99 the numbering doesn't matter-, on Oct 22th please you ? That's an option I could deal with.
So the question is, do we now have a 7th month cycle? Or is it just this once? Is the next release a 5 month one, or are we back to 6?
I have some experience in date-shifting from my time as Rmaint, FWIW. When I found I had to push a release date, I generally tried to do one of two things depending on the date: 1. If I was less than halfway to the next release date, i released and the released again on the next scheduled date. 2. If I was over halfway to the next release date, I skipped a release. This is probably *not* the exact fix for major releases, but.... In light of this, I would suggested that we slide this release date forward if needed and retain the next release date as scheduled. This track probably causes the least disruption to those who have planned their upgrade schedule around the major release schedule. Kind Regards, Chris
Le 02/08/2012 10:55, Chris Cormack a écrit : Hi Chris & al, >> Moving by 1 month is the best option I can see. >> Could the release of a 3.10.0RC1 -or 3.9.99 the numbering doesn't >> matter-, on Oct 22th please you ? That's an option I could deal with. >> > So the question is, do we now have a 7th month cycle? Or is it just this once? > Is the next release a 5 month one, or are we back to 6? We stick to a 6 months release: * oct 22th = RC1, nov 22th = .0 release * apr 22th = RC1, may 22th = .0 release -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Paul, et. al., I just wanted to express my support for this plan on the mailing list. What I/we plan to do:
* 2 months before the release (number N) = - declare small feature freeze: only bugfixes and small/non-core enhancement are pushed for the next release - branch the release on git. master goes his way for N+1, major ENH can be pushed here. N is still controlled by the RM, N-1 controlled by the RMaint * 1 month before the release = - declare string freeze: no more string changes to let translators do their work
+1 By "are pushed" I really mean the freeze is about *pushing* before the
freeze. In previous releases, we've accepted some ENH provided they've been *submitted* before the freeze. That's an important change, be careful ! If we want stability, we need more time to test.
+INFINITY!!!! This is a very good idea. Last (but not least) point:
* August is holiday time for Northern hemisphere, and oct22 minus 2 months = Aug 22, that's a problem. That's why I propose to switch to may/november for our releases. The timeline would be: * today => sept 22 => hack ! hack ! hack ! * sept 22 => feature freeze, no more large or core enhancements pushed (whatever the date of submission of the patch) * oct 22 => string freeze, no more string changes * nov 22 => Koha 3.10 (and I'm no more RM. Yippee !!! :D :D )
I think with this modified release process we can look forward to the release of 3.10.0 on November 22 as the most stable .0 release yet. Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
participants (15)
-
Brendan Gallagher -
BWS Johnson -
Chris Cormack -
Chris Cormack -
Chris Nighswonger -
Colin Campbell -
Elliott Davis -
Fischer, Katrin -
glaws -
Ian Walls -
Jared Camins-Esakov -
Marcel de Rooy -
MJ Ray -
Owen Leonard -
Paul Poulain