Commentary on Paul's Koha 3.8 Release Manager proposal
Everyone, Enclosed is my response to Paul Poulain's proposal for Koha 3.8 Release Manager. The details of his document can be found here: http://wiki.koha-community.org/wiki/Proposal_paul_p_RM38 I thoroughly agree that the time has come to start thinking beyond the next stable release, and on to Koha 4.0. Those of you who have seen either my presentation at KohaCon 2010's hackfest or KUDOScon 2011 know I'm very excited about the possibilities of where we can take Koha in it's next major iteration. However, I disagree with a couple of the details of Paul's proposal. The major disagreement is with the timeline. The proposal as it stands is to start working on Koha 3.8 and Koha 4.0 in parallel, with Koha 3.8 releasing April 2012, and Koha 4.0 releasing Oct. 2012. I feel that going from Koha 3.8 directly to 4.0 is unwarranted. To my mind, there are many possible releases between 3.8 and 4.0, like 3.10, 3.12, 3.14 and so forth. This is supported by the actual database version numbers in Koha: the current release is actually 3.04.04; the zeros are squashed out for convenience. I would counter that, while we should continue releasing new features every 6 months on the 3.X line, Koha 4.0 should be defined by it's features. Paul's proposal calls several features to be key, including Solr integration, database abstraction, updated online help, improved authentication stack and automated tests. I agree that all these things are important and should be worked towards, but I'm not convinced they should be the sum total of the changes that define Koha 4.0 I would like to see the following in Koha 4.0: - arbitrary metadata formats (beyond MARC) - arbitrary biblio relationships - full support for authority records, including uploading through the GUI, automatic linking, "explode" searching and more - improvements to the borrower data structure (fewer pre-defined data fields, more flexibility) - separation of "receiving" and "payment" in acquisitions - serials import and export (MFHD, prediction patterns, etc) - many, many more things I think that the community should meet and discuss what features are both desirable and reasonably likely to be completed in the next few years (is there interest by libraries to sponsor these developments, or from developers to code it "for fun"?). Once a feature set is agreed on to "define" Koha 4.0, the developers of these features would meet to hash out what underlying structural changes would be required to make them happen. Hopefully commonalities could be found, so that the underlying structure could be developed in a way that accommodates multiple developments at once. Identifying these prerequisite changes early on would reduce the difficulties of rebasing developments, since we'd all be working off a similar set of base assumptions. It would also provide a reasonable basis for excluding or delaying developments that don't fit into the overall plan. Once the features were chosen and a framework for their development agreed upon, it would be time to code. The development and submission process would be the same as it is now (basing patches off master and submitting to the patches listserv and Bugzilla). As new features become ready, they would be incorporated into the time-based 3.X releases by the release manager for that particular cycle. This way, what works is made available, and what needs more time continues to be tested until it's stable for production use. Once all the agreed upon features were ready, it would be time to release Koha 4.0. Hopefully, that would be with 1-2 years, but there would be flexibility here. One of my other issues with the proposal was the "lightening" of Quality Assurance standards for the first 3 months of the 3.8 release cycle. If operating under the 1 year time frame for Koha 4.0 as outlined in Paul's proposal, this would seem necessary in order to get everything included in time. Unfortunately, while the next 3 months would be dedicated to clean up of features rolled in, there is never any guarantee of bug fixes being generated in a timely fashion. You cannot force people to write or sign off on bug fixes; we're all free agents here. So code that has not been rigourously tested could get in and stay in for a long time, long after 4.0 would release, because no one knows how to test it or fix it. However, if we operate under my proposed model, all code could continue to operate under the same quality assurance standards, and nothing that is broken or otherwise incomplete would make it in until it was ready. Releases would still come out every 6 months, even if a major development was inoperable, until that development could be completed. This has been a long email, so let me sum up. I think: - we should continue to release Koha 3.8, 3.10, 3.12 etc on 6 month cycles using the current community practices for feature inclusion - we should, as a community, meet and decide what features we want to define Koha 4.0 - the developers of those features should get together and come up with a unified plan for accomplishing them all, in a way that requires minimal rebasing - as new features become stable and tested, they should be folded into the 3.X line until all the features defined as "Koha 4.0" are ready - Once all the agreed upon features are complete, Koha 4.0 is then released. I've already gotten some feedback on this in today's Koha IRC meeting, but I'd very much like to hear what the rest of the community thinks about this, particular in relation to Paul's proposal for 3.8 Release Manager. Our time is limited for discussion, as the election time draws near, but hopefully I've made my thoughts clear enough for a dialogue to get started. Please let me know if I've been at all unclear. Cheers, -Ian -- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Le 07/09/2011 23:58, Ian Walls a écrit :
Everyone, Hi Ian & everyone, The major disagreement is with the timeline. The proposal as it stands is to start working on Koha 3.8 and Koha 4.0 in parallel, with Koha 3.8 releasing April 2012, and Koha 4.0 releasing Oct. 2012. I feel that going from Koha 3.8 directly to 4.0 is unwarranted. To my mind, there are many possible releases between 3.8 and 4.0, like 3.10, 3.12, 3.14 and so forth. This is supported by the actual database version numbers in Koha: the current release is actually 3.04.04; the zeros are squashed out for convenience. I think our main (only ?) disagreement is based on some misunderstandings. I'll try to explain more clearly.
One of my other issues with the proposal was the "lightening" of Quality Assurance standards for the first 3 months of the 3.8 release cycle. You've forgotten to specify i've written "how exactly, To Be Discussed". This is only a main concern I have expressed before : smoothen things. The idea of sandboxes to have more ppl testing will be a step in this
Here is the timeline for Koha 3.8 : * starts 2011-11 * feature freeze 2012-03 * release 2012-04 In my proposal, Koha 4.0 has the following timeline : * defining the strategy & the roadmap start 2011-11 and end in 2012-01 (3 months) * hacking starts in 2012-02 * feature freeze start in 2012-09 * release 2012-10 It means there will/can have a real hacking overlap of something like 2 months (feb-march 2012) According to me, we could even start immediately to list everything we're dreaming of. The 2nd step being to evaluate how we can achieve each goal, who can endorse the needed work, which priority for what. Maybe another point of disagreement/misunderstanding is that you propose "a 4.0 with everything we want" while I propose "a 4.0 with everything that can fit in the 6 months timeline" We switched from Feature-based release to Time-based release, and I definitely don't want to change this. This has been a major good decision we shouldn't change ! My second rule is "if you change something important in the structure of Koha, 1st digit get a +1". [ With the change of templating system, we could have updated the version to 4.0 instead of 3.6, that would not have been a scandal (frenchism suspected...) ! ] It means that, for example, we decide that the priorities are (from biggest to lowest) 1- update indexing engine 2- enable database persistency 3- arbitrary metadata formats (beyond MARC) Step 1 and 2 have volunteers & code & ... => it will be in 4.0 Step 3 doesn't have a volunteer => it will be in 5.0 After 4.0 there will be a 4.2 or a 5.0 depending on step 3 being ready. I like the idea of a roadmap, with the following chapters : Will be in version XX : * feature A already merged * feature B already merged * ... Should be in version XX : * feature C, patch submitted, being examinated Could be in version XX : * feature D, announced by mail@people.com Have been submitted but rejected for instance : * feature E, patch does not apply * feature F, failed QA * ... (doesn't eclipse have such a roadmap ? not sure) Should I conclude that the version number should be decided/defined when feature freeze reached ? Maybe, i'm not sure. About your idea of 3.10, 3.12,... I fear/feel that we can't afford managing the conflicts that will result : * "4.0" has database persistency merged, we still lack "indexing engine changes", so 4.0 has not been released * a patch to enable "arbitrary biblio relationships" is submitted for 3.x It will be totally incompatible with "4.0 still unreleased" and we have a problem... direction. I'll also send regular mails to try to motivate ppl to test/sign-off. Maybe that will be enough. But maybe we could also accept a derivative workflow like "there are some experienced ppl that can decide to do sign-off and passed QA at the same time, they're experienced&wise enough to decide if a patch is small/trivial enough to be applied without a 2nd eye checking". Or ther(s) rule(s) we could discuss (I have a lot of suggestions I could/will do). My idea is not to lower the quality. Ian, thanks to continue this thread. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Paul, Yes, the main difference in our proposals is primarily one of timing. You're proposing that we have Koha 4.0 ready in about 1 year, and will include Solr integration and updated Perl coding practices. Those are certainly good things, yes, and they may well be possible in 1 year. Or they may not. I agree that we should continue to do time-based releases; every 6 months seems to be working well for us. The difficulty here, though, is that with time-based releases, you cannot guarantee that any given feature will be ready in time. Sure, any one development team can promise they'll have it coded up to their clients standards and rebased for inclusion, but how will that interact with other code written by other teams? Who, outside the original development team, can be committed to test and sign-off on the work? So, when working on time-based releases, you cannot promise features. This is why I propose we continue with time-based releases on the 3.X line until such time as all the agreed-upon features for Koha 4.0 are ready. If we're quick, sure, we can jump straight from 3.8 to 4.0. I think it's much more likely we'll have a 3.10 in there, but maybe that's just me. By no longer attempting to set a release date for a feature-based release, we open ourselves up to more feature possibilities for Koha 4.0. Solr and updated Perl pratices are both very developer-centric improvements. What are the positive end results for the libraries? I know with Solr, you can more easily customize your facets. Updated Perl coding could result in faster performance. I think it's more important to define our goals in those terms, rather than in how we're going to accomplish them. HDL pointed out early in the thread that some of the features I was floating as possible for Koha 4.0 were librarian-centric, and some were developer-centric. He's right, I wasn't really clearly differentiating in that list. It was mostly meant as an example of what else would could reasonably accomplish for the next major release of the software. I don't think there will be nearly as many merge conflicts and rebase issues as you fear, Paul. Once we decide on our feature set, any interested developers would meet to figure out the coding requirements. This may take several meetings, but in the end, we'll have a roadmap of what features depend on what. A couple points of practice will make the development go more smoothly: - Those that can be done first will be, which will then open the door for other features, and so on. - Developments for each feature are done on frequently-rebased topic branches. Since we'll have identified them as a community, we can put them on the main Koha git repo. We can give keys on a per-branch basis to the developers working most heavily on those branches. - Small, atomic, easily-tested patches are the key here. If two features can both be developed concurrently, and they happen to tread on each others toes, this will keep any merge conflicts small and easily reconciled. Whatever features are ready by the feature freeze for 3.8 would be what go into 3.8; this is a matter of distribution and labeling, and won't affect the developers who are all working on master (or it's closely-based feature branches). Release Maintainers will continue their job backporting fixes to those stable branches we still support. I really like the idea of having multiple sandboxes set up so that folks can test the different features more easily. We should definitely do this. Having more people doing testing and Quality Assurance is always a good thing. Providing test plans, test results and reasons for passing/failing QA on the bug report will make it much easier to keep track of the history of any particular fix or feature. Our end goal is the same: make Koha the best it can be. We're mostly disagreeing over small details. Whatever the outcome of this discussion, I know we will all continue to work hard together to improve Koha. Cheers, -Ian On Mon, Sep 12, 2011 at 5:33 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Le 07/09/2011 23:58, Ian Walls a écrit :
Everyone, Hi Ian & everyone, The major disagreement is with the timeline. The proposal as it stands is to start working on Koha 3.8 and Koha 4.0 in parallel, with Koha 3.8 releasing April 2012, and Koha 4.0 releasing Oct. 2012. I feel that going from Koha 3.8 directly to 4.0 is unwarranted. To my mind, there are many possible releases between 3.8 and 4.0, like 3.10, 3.12, 3.14 and so forth. This is supported by the actual database version numbers in Koha: the current release is actually 3.04.04; the zeros are squashed out for convenience. I think our main (only ?) disagreement is based on some misunderstandings. I'll try to explain more clearly.
Here is the timeline for Koha 3.8 : * starts 2011-11 * feature freeze 2012-03 * release 2012-04
In my proposal, Koha 4.0 has the following timeline : * defining the strategy & the roadmap start 2011-11 and end in 2012-01 (3 months) * hacking starts in 2012-02 * feature freeze start in 2012-09 * release 2012-10
It means there will/can have a real hacking overlap of something like 2 months (feb-march 2012)
According to me, we could even start immediately to list everything we're dreaming of. The 2nd step being to evaluate how we can achieve each goal, who can endorse the needed work, which priority for what.
Maybe another point of disagreement/misunderstanding is that you propose "a 4.0 with everything we want" while I propose "a 4.0 with everything that can fit in the 6 months timeline" We switched from Feature-based release to Time-based release, and I definitely don't want to change this. This has been a major good decision we shouldn't change !
My second rule is "if you change something important in the structure of Koha, 1st digit get a +1". [ With the change of templating system, we could have updated the version to 4.0 instead of 3.6, that would not have been a scandal (frenchism suspected...) ! ]
It means that, for example, we decide that the priorities are (from biggest to lowest) 1- update indexing engine 2- enable database persistency 3- arbitrary metadata formats (beyond MARC) Step 1 and 2 have volunteers & code & ... => it will be in 4.0 Step 3 doesn't have a volunteer => it will be in 5.0 After 4.0 there will be a 4.2 or a 5.0 depending on step 3 being ready.
I like the idea of a roadmap, with the following chapters : Will be in version XX : * feature A already merged * feature B already merged * ... Should be in version XX : * feature C, patch submitted, being examinated
Could be in version XX : * feature D, announced by mail@people.com
Have been submitted but rejected for instance : * feature E, patch does not apply * feature F, failed QA * ...
(doesn't eclipse have such a roadmap ? not sure)
Should I conclude that the version number should be decided/defined when feature freeze reached ? Maybe, i'm not sure.
One of my other issues with the proposal was the "lightening" of Quality Assurance standards for the first 3 months of the 3.8 release cycle. You've forgotten to specify i've written "how exactly, To Be Discussed". This is only a main concern I have expressed before : smoothen things. The idea of sandboxes to have more ppl testing will be a step in this
About your idea of 3.10, 3.12,... I fear/feel that we can't afford managing the conflicts that will result : * "4.0" has database persistency merged, we still lack "indexing engine changes", so 4.0 has not been released * a patch to enable "arbitrary biblio relationships" is submitted for 3.x It will be totally incompatible with "4.0 still unreleased" and we have a problem... direction. I'll also send regular mails to try to motivate ppl to test/sign-off. Maybe that will be enough. But maybe we could also accept a derivative workflow like "there are some experienced ppl that can decide to do sign-off and passed QA at the same time, they're experienced&wise enough to decide if a patch is small/trivial enough to be applied without a 2nd eye checking". Or ther(s) rule(s) we could discuss (I have a lot of suggestions I could/will do). My idea is not to lower the quality.
Ian, thanks to continue this thread.
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Yes, the main difference in our proposals is primarily one of timing. You're proposing that we have Koha 4.0 ready in about 1 year, and will include Solr integration and updated Perl coding practices. Those are certainly good things, yes, and they may well be possible in 1 year. Or they may not. well, considering Solr is in production at 3 of our libraries, that some
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 12/09/2011 15:37, Ian Walls a écrit : ppl are already involved in reintroducing zebra over the updated API we've developped for Solr, I think it's possible in 1 year. And BibLibre is willing to dedicate the needed ressources to achieve this goal !
I agree that we should continue to do time-based releases; every 6 months seems to be working well for us. The difficulty here, though, is that with time-based releases, you cannot guarantee that any given feature will be ready in time. Sure, any one development team can promise they'll have it coded up to their clients standards and rebased for inclusion, but how will that interact with other code written by other teams? Who, outside the original development team, can be committed to test and sign-off on the work? I don't say it will be easy. But not harder than what had been done before. (Note for newbies : i'm an always optimistic man ;-) ) I add that more coordination will result in less conflicts. And my application is coordination-centric.
So, when working on time-based releases, you cannot promise features. agreed This is why I propose we continue with time-based releases on the 3.X line until such time as all the agreed-upon features for Koha 4.0 are ready. If we're quick, sure, we can jump straight from 3.8 to 4.0. I think it's much more likely we'll have a 3.10 in there, but maybe that's just me. isn't it just a numbering difference ? I feel it is. I've nothing about updating the 1st digit on each major structural change. So, i've nothing against saying something like : "the final goal : solr/persistency/DB abstraction/supporting more than MARC oct 12, Solr & DB abstraction ready = Koha 4.0 april 13, persistency ready = Koha 5.0 oct 13, supporting more than MARC = Koha 6.0
It's just a numbering question, very minor difference ;-)
I think it's more important to define our goals in those terms, rather than in how we're going to accomplish them. Agreed
HDL pointed out early in the thread that some of the features I was floating as possible for Koha 4.0 were librarian-centric, and some were developer-centric. He's right, I wasn't really clearly differentiating in that list. It was mostly meant as an example of what else would could reasonably accomplish for the next major release of the software. Agreed, and coordination (Hackfest in 4 weeks++ on this matter !)
I don't think there will be nearly as many merge conflicts and rebase issues as you fear, Paul. Once we decide on our feature set, any interested developers would meet to figure out the coding requirements. This may take several meetings, but in the end, we'll have a roadmap of what features depend on what. Agreed. I really like the idea of having multiple sandboxes set up so that folks can test the different features more easily. We should definitely do this. Having more people doing testing and Quality Assurance is always a good thing. Providing test plans, test results and reasons for passing/failing QA on the bug report will make it much easier to keep track of the history of any particular fix or feature. BibLibre has now a "private cloud", with 4 servers, virtualization, template VMs. We can easily add physical servers, have a centralised authentification,... Still to be discussed with all of us, but we could add a (physical) server (16GB, quadri code) dedicated to sandboxes (or add VMs the physical server we already use for testing). We will discuss details later.
Our end goal is the same: make Koha the best it can be. We're mostly disagreeing over small details. Whatever the outcome of this discussion, I know we will all continue to work hard together to improve Koha. Agreed ;-)
Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOjBA3AAoJEK81SonuhyGofA4IALE+fszcuU4fqkJWh6pFoBIi zowZwPnElz68LXyAfMLzS2USkvD2sYJ4CYSw3QjgLE0oLMMyvVAobi7wvPLo/u7y S8IL8E/c7ZYeSzoQDI600MzNWt+FrpCQlcuwryxNjCPy8MyNuV/Q4aeei+/Qyj7S TVusfwKf8zaaNeoDMZC8xFj00Ij240pCCoIk0dadVtmINTtGOkmuTYpHOd+Fz4Pb GFDGfss8M70gZwEr8f/bo7kj4b0r+LHxv6xvtz+LR1iPd+f7OL7iikhnnRgSerPI vniy6oImc3sMR6oQGBcOaOhkWVochJMUs4iXRWkwCM3tSyjIWxvSYGFOoxquLiM= =9GlW -----END PGP SIGNATURE-----
Ian Walls <ian.walls@bywatersolutions.com> [...]
The major disagreement is with the timeline. The proposal as it stands is to start working on Koha 3.8 and Koha 4.0 in parallel, with Koha 3.8 releasing April 2012, and Koha 4.0 releasing Oct. 2012. I feel that going from Koha 3.8 directly to 4.0 is unwarranted. To my mind, there are many possible releases between 3.8 and 4.0, like 3.10, 3.12, 3.14 and so forth. This is supported by the actual database version numbers in Koha: the current release is actually 3.04.04; the zeros are squashed out for convenience. [...]
A minor disagreement is with the platform. Paul's proposal seems to be for 3.8 and 4.0 RM, yet there's very little detail on 4.0 because the community discussion hasn't really started: lots of great features suggested, some scary changes, but not much agreed. Let's agree the RM for 3.8 and then when the next release rolls around, then let's see what we want to do then? In general, I'm fine with folding into 3.x progressively. Seems like a good idea. (Now find me more libraries so I can spend more time on it... ;-) ) Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 15/09/2011 00:51, MJ Ray a écrit :
A minor disagreement is with the platform. Paul's proposal seems to be for 3.8 and 4.0 RM, yet there's very little detail on 4.0 because the community discussion hasn't really started: lots of great features suggested, some scary changes, but not much agreed. Let's agree the RM for 3.8 and then when the next release rolls around, then let's see what we want to do then?
In general, I'm fine with folding into 3.x progressively. Seems like a good idea. (Now find me more libraries so I can spend more time on it... ;-) )
Answering your mail just before the IRC meeting, sorry for not taking time before. I've already expressed that changing RM every 6 months is too much. In my idea, the RM should promise to stay RM for 2 releases. That's with this idea that I?ve submitted my application. I agree that there are little details for 4.0, but the most important thing is that I want to discuss how to do things early (starting "immediately", organize, ... HTE (Hope This Explain ;-) ) - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOjA2MAAoJEK81SonuhyGo21cH/1wYFS/lqRZMe9Zi69DTVInk f7deqSvyimsGLfpXKxJ8DPq2hLsVqTA8o5aCg8sENFMboNYgxqoYaaAjLaKCc5tY PPhMLN2mXhHVQ1aOUpucqkpFX6NIA+gsVOjGJW+EYZ3ika+czVnzlMR/TfzDP/vH mpxZnZEVylgDcprHdG0Ty15Rgnx3u8ZnTRoYJe2jLZORs1oijUJYZGpUnSKFSjKO pus/JFrBiTr1OUwsXRRQHlw/S2YefV2B+NreIiZlY51TP+TstNtbfx36qM4O2JMw ZHEQLDrgKzKj7Zg6shhT+69+lDmF1QP2eD/v6Fjmfq+ejqiwq7c7BdHSk/iJNgE= =5gaP -----END PGP SIGNATURE-----
Paul Poulain <paul.poulain@biblibre.com>
I've already expressed that changing RM every 6 months is too much. In my idea, the RM should promise to stay RM for 2 releases. That's with this idea that I?ve submitted my application.
Offer if you want, but that's not how it was worded. Anyway, it seems this is moot for now. Meeting minutes to follow. Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and LMS developer, statistician. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for Koha work http://www.software.coop/products/koha
participants (3)
-
Ian Walls -
MJ Ray -
Paul Poulain