A Discussion on A Policy Setting Forth Standards of Code Submission, etc. [WAS: RFCs for 3.4 from BibLibre (serials & acquisitions)]
Starting a new thread because this is indirectly related to Paul's request. 2010/11/2 Paul Poulain <paul.poulain@biblibre.com>
Hello,
I have filed RFCs for all enhancements you can expect from BibLibre in the next few months. All of them are sponsored (by Saint-Etienne University), and will have to be delivered to the library for a "go live" in May-2011
* You can find them on the wiki (search biblibre rfc for3.4) : http://wiki.koha-community.org/w/index.php?title=Special:Search&ns0=1&redirs=1&search=biblibre+rfc+for3.4&limit=50&offset=0 * You can also find them on bugzilla : i've added a saved search, shared with everybody, called "Koha 3.4 enhancements".
They are only related to serials and acquisitions.
What's the next step ? * read the RFCs * comment them (on the wiki or on bugzilla) * we will probably propose/organise an IRC meeting to discuss & get any feedback for all those incoming developments. None of them (except the solR one) has started, so if you want to give us an advice, add something, ... it's now or it may be too late ;-)
As a vendor-neutral voice, I would like to encourage everyone who has an vested interest in these areas and the best interests of the Koha project at heart to actively participate and respond to these RFC's. It seems that often there is little dicussion, etc. on RFCs in the community. And even when there is discussion, etc. it is often unclear if a consensus is reached (at least publicly). Furthermore, I would encourage vendors and others who post RFC's to do so with a willingness to adapt, adjust, bend, compromise, and/or <your-favorite-term-goes-here> to positions on those RFC's which may be different but are clearly the consensus of the community at large. Vendors may and often do have the resources to implement "what they want," however, this is not in the spirit of cooperation which this project so greatly depends upon for its success. Clients of vendors should be educated during the RFQ process as to this aspect of open source, and their expectations managed accordingly, imho. I would also suggest that we implement a policy that states in some agreeable way that code/features will not be pushed to master which have not passed through a review and consensus process by the community and the RM (as the elected head of development by the community). No one excepting possibly the RM should presuppose that their code is guaranteed inclusion by default. Secondly, I would suggest that we implement a strong recommendation that larger shops submit timely RFC's *prior to* beginning work on code and then promote discussion on those RFC's. This recommendation should with some lesser strength suggest that everyone submit timely RFC's to maximize productivity and usefulness of the resources of all concerned. Thirdly, I would suggest a stated policy (and such a policy is presently in place practically) which requires all submissions to pass through a QA branch and receive at a minimum one sign-off prior to being pushed into master. This policy should also assign a certain amount of responsibility to the one signing off to avoid "frivolous" sign-offs. It should also, perhaps, include a restriction that the required sign-off for pushing to master be a disinterested developer perhaps from another vendor or the community at large. This is a discussion we need to have. I would encourage everyone to invest time (the operative term here is 'invest') in this discussion. Kind Regards, Chris Nighswonger Koha 3.2.x Release Maintainer
Chris, I disagree that the first sign-off on a major vendor's patches should be external. The first sign-off from a major vendor should be *internal* to their quality control process. This was at one time the standing policy amongst LibLime and BibLibre for major changes. I think encouraging abstraction and RFC-aware flexibility is fine, but I think it is unwise to suggest that we should block *working code* from getting in just because a bigger, different or more-deluxe RFC exists. RFCs and the widespread desire for a feature flavor X are really quite removed from a working implementation ready for action, testing and revision now. Also, I think if you develop your features "in the open" (e.g., posted w/ gitweb, or on github), the burden of synthesizing multiple RFCs and general "feature consensus" sentiment isn't on you in quite the same way as when changes are delivered en bloc. A vendor has what their clients are paying for, and if other devs have an unfunded desire for extra feature X, they can follow the branch right along add the last bit themselves, all while still in development. Whether X is pulled in by the vendor, or separately submitted by the dev to the RM doesn't really matter. --joe 2010/11/2 Chris Nighswonger <cnighswonger@foundations.edu>
Starting a new thread because this is indirectly related to Paul's request.
2010/11/2 Paul Poulain <paul.poulain@biblibre.com>
Hello,
I have filed RFCs for all enhancements you can expect from BibLibre in the next few months. All of them are sponsored (by Saint-Etienne University), and will have to be delivered to the library for a "go live" in May-2011
* You can find them on the wiki (search biblibre rfc for3.4) : http://wiki.koha-community.org/w/index.php?title=Special:Search&ns0=1&redirs=1&search=biblibre+rfc+for3.4&limit=50&offset=0 * You can also find them on bugzilla : i've added a saved search, shared with everybody, called "Koha 3.4 enhancements".
They are only related to serials and acquisitions.
What's the next step ? * read the RFCs * comment them (on the wiki or on bugzilla) * we will probably propose/organise an IRC meeting to discuss & get any feedback for all those incoming developments. None of them (except the solR one) has started, so if you want to give us an advice, add something, ... it's now or it may be too late ;-)
As a vendor-neutral voice, I would like to encourage everyone who has an vested interest in these areas and the best interests of the Koha project at heart to actively participate and respond to these RFC's. It seems that often there is little dicussion, etc. on RFCs in the community. And even when there is discussion, etc. it is often unclear if a consensus is reached (at least publicly).
Furthermore, I would encourage vendors and others who post RFC's to do so with a willingness to adapt, adjust, bend, compromise, and/or <your-favorite-term-goes-here> to positions on those RFC's which may be different but are clearly the consensus of the community at large. Vendors may and often do have the resources to implement "what they want," however, this is not in the spirit of cooperation which this project so greatly depends upon for its success. Clients of vendors should be educated during the RFQ process as to this aspect of open source, and their expectations managed accordingly, imho.
I would also suggest that we implement a policy that states in some agreeable way that code/features will not be pushed to master which have not passed through a review and consensus process by the community and the RM (as the elected head of development by the community). No one excepting possibly the RM should presuppose that their code is guaranteed inclusion by default.
Secondly, I would suggest that we implement a strong recommendation that larger shops submit timely RFC's *prior to* beginning work on code and then promote discussion on those RFC's. This recommendation should with some lesser strength suggest that everyone submit timely RFC's to maximize productivity and usefulness of the resources of all concerned.
Thirdly, I would suggest a stated policy (and such a policy is presently in place practically) which requires all submissions to pass through a QA branch and receive at a minimum one sign-off prior to being pushed into master. This policy should also assign a certain amount of responsibility to the one signing off to avoid "frivolous" sign-offs. It should also, perhaps, include a restriction that the required sign-off for pushing to master be a disinterested developer perhaps from another vendor or the community at large.
This is a discussion we need to have. I would encourage everyone to invest time (the operative term here is 'invest') in this discussion.
Kind Regards, Chris Nighswonger Koha 3.2.x Release Maintainer
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi Joe, On Tue, Nov 2, 2010 at 2:10 PM, Joe Atzberger <ohiocore@gmail.com> wrote:
Chris, I disagree that the first sign-off on a major vendor's patches should be external. The first sign-off from a major vendor should be *internal* to their quality control process. This was at one time the standing policy amongst LibLime and BibLibre for major changes.
My recommendation is that we require a *minimum* of one sign-off by a disinterested party. This in no way excludes a company having their own QA internally and signing off. Furthermore, it really provides no impedance to the entire process when one considers how simple it should be to obtain a sign-off on good working code.
I think encouraging abstraction and RFC-aware flexibility is fine, but I think it is unwise to suggest that we should block *working code* from getting in just because a bigger, different or more-deluxe RFC exists. RFCs and the widespread desire for a feature flavor X are really quite removed from a working implementation ready for action, testing and revision now.
Ahh... nothing in my recommendation suggests that "should block *working code* from getting in just because a bigger, different or more-deluxe RFC exists." It simply suggests we have a policy in place which will actively promote some sort of community collaboration particularly on the part of large shops who "should" know better than to clam up especially on large feature development. That is patently bad behavior in the light of community participation which is the foundation of this project. It is certainly within the rights of anyone to take the source and run with it in whatever way they like. They may even take it and never contribute back. However, it is not within their right to do large, unilateral development and then expect it to be pushed into the main codebase.
Also, I think if you develop your features "in the open" (e.g., posted w/ gitweb, or on github), the burden of synthesizing multiple RFCs and general "feature consensus" sentiment isn't on you in quite the same way as when changes are delivered en bloc. A vendor has what their clients are paying for, and if other devs have an unfunded desire for extra feature X, they can follow the branch right along add the last bit themselves, all while still in development. Whether X is pulled in by the vendor, or separately submitted by the dev to the RM doesn't really matter.
A couple of points addressing this scenario as a whole (none of these is against the principle of open development): 1. That scenario may work for simpler features/code. But consider the current press to switch from zebra to 'foo' (any one of several suggestions recently). If a vendor develops an entire replacement for C4::Search and friends which centers around 'foo' in a unilateral fashion, the de facto assertion is that the community must either take what they have done or leave it. That assertion is within their prerogative as a vendor, however, it is equally possible that the community consensus may be to use 'foobar' rather than 'foo' and so away we go on a different path leaving said vendor to sink or swim on their own fork. 2. Given the problems we already have with a lack of development cooperation, this scenario at best does nothing to address those problems. 3. This scenario appears a bit "vendor-centric." I am of the opinion that Koha should be "community-centric" with individual's first and vendor's second in order of relationship. This may not be the view of all involved. However, if it were not for Koha, Koha support vendors would be out of some amount of business. Yes, I realize there are other FOSS ILS's available. However, the point is that many Koha support vendors (especially the names you mention earlier) came into existence because of Koha. I think it is in their best interest to help assure the survival of the community by putting in as they take out, and it is the community as a whole that decides the "rules" (if you will) for putting in. 4. Regarding the statement "A vendor has what their clients are paying for..." True, vendors are client driven. However, as I said in my initial post, vendors must educate their clients as to the nature of open source. Client expectations should be set based on known, published community procedures. If this were properly done, many problems would be resolved. As it is, I think vendors have a very hard time managing their own growth once it reaches the ballooning point. Unmanageable growth will kill you... as we have seen. In the final analysis, if each vendor pursues their own direction, we will end up with a Baskin-Robins of Koha. Make the job of RM hard enough and no one will want it. At some point the project dies due to leanness and overextension. The strength of the project lies in the *two-way* cooperation of its members. The "we have developed it: take it or leave it" approach is a one-way, dead-end street for the community. Kind Regards, Chris
On Tue, Nov 2, 2010 at 4:12 PM, Chris Nighswonger < cnighswonger@foundations.edu> wrote:
On Tue, Nov 2, 2010 at 2:10 PM, Joe Atzberger <ohiocore@gmail.com> wrote:
Chris, I disagree that the first sign-off on a major vendor's patches should be external. The first sign-off from a major vendor should be *internal* to their quality control process. This was at one time the standing policy amongst LibLime and BibLibre for major changes.
My recommendation is that we require a *minimum* of one sign-off by a disinterested party. This in no way excludes a company having their own QA internally and signing off. Furthermore, it really provides no impedance to the entire process when one considers how simple it should be to obtain a sign-off on good working code.
Ok, I read that as amending "that *the *required sign-off for pushing to master be a disinterested developer" to "that *a *...". Regarding the simplicity of signing off, I take some issue. It is *severely* non-trivial to test major integration features. Consider SIP2 and LDAP, or something like EDI. It can depend not just on accurate test data, but entire servers, network environments, remote accounts granted by a supplier, foreign domain/language knowledge, etc. Sure, I'd love it for everybody to have a dozen signoffs. I just think blocking code while waiting on a 3rd party (who by design is disinterested) to come around and dedicate some resources is a questionable policy. Right now we can't even get comment on RFCs, let alone dedicated VMs and manpower.
Ahh... nothing in my recommendation suggests that "should block *working code* from getting in just because a bigger, different or more-deluxe RFC exists." It simply suggests we have a policy in place which will actively promote some sort of community collaboration particularly on the part of large shops who "should" know better than to clam up especially on large feature development. That is patently bad behavior in the light of community participation which is the foundation of this project. It is certainly within the rights of anyone to take the source and run with it in whatever way they like. They may even take it and never contribute back. However, it is not within their right to do large, unilateral development and then expect it to be pushed into the main codebase.
Forgive me if I'm off the pulse a bit, but do these expectations exist today? The release process establishes when new features are accepted or not, and it has been pretty explicit and clear. The problem used to be big unilateral changes that weren't getting submitted (including some code that I wrote). Now the problem is they're getting submitted?
Also, I think if you develop your features "in the open" (e.g., posted w/
gitweb, or on github), the burden of synthesizing multiple RFCs and general "feature consensus" sentiment isn't on you in quite the same way as when changes are delivered en bloc. A vendor has what their clients are paying for, and if other devs have an unfunded desire for extra feature X, they can follow the branch right along add the last bit themselves, all while still in development. Whether X is pulled in by the vendor, or separately submitted by the dev to the RM doesn't really matter.
A couple of points addressing this scenario as a whole (none of these is against the principle of open development):
1. That scenario may work for simpler features/code. But consider the current press to switch from zebra to 'foo' (any one of several suggestions recently). If a vendor develops an entire replacement for C4::Search and friends which centers around 'foo' in a unilateral fashion, the de facto assertion is that the community must either take what they have done or leave it.
Or patch it, or extend it, or put in on a feature branch, or break it into discrete elements, or defer acceptance until sufficiently abstracted, etc. If the vendor cares about it hitting mainline, then they'll follow up to do what it takes, within reason. If they don't, then I don't think new policy requirements will affect them in the least.
2. Given the problems we already have with a lack of development cooperation, this scenario at best does nothing to address those problems.
This comes back to the question of whether you can force cooperation. I don't think we can, effectively. I support codifying expectations and best practices, but "requiring" disinterested, competing or downright hostile parties to cooperate or pretend to cooperate seems destined to fail.
3. This scenario appears a bit "vendor-centric."
That's because, as has been pointed out previously, Koha's authorship is de facto vendor-centric. The vast majority of Koha code historically originated from vendors and the majority of new code still comes from vendors. Every RM worked for a vendor (or two). I'm happy to see Koha institutions and cooperatives still able to contribute code back to the project on their own (some for longer than I've been associated with Koha), and I'd love to see more of it. Right now though, the code being affected by proposed policies is still *mostly* in vendors' houses.
I am of the opinion that Koha should be "community-centric" with individual's first and vendor's second in order of relationship.
OK, I know what a vendor is. But in this context I don't know what you mean by "community-centric" or "individual" or "order of relationship". If you just mean that community comes first, then yay, community. I suspect we can escape talking about whether vendors are first, second or other class citizens in the community because it doesn't really matter for the proposed policy.
This may not be the view of all involved. However, if it were not for Koha, Koha support vendors would be out of some amount of business.
And without vendors *no* version of Koha would ever have been written or released. It's sorta funny that I'm the one saying this stuff, as I am not currently affiliated with any vendor. 4. Regarding the statement "A vendor has what their clients are paying
for..." True, vendors are client driven. However, as I said in my initial post, vendors must educate their clients as to the nature of open source. Client expectations should be set based on known, published community procedures. If this were properly done, many problems would be resolved.
This is a good point, and I was interested to read Paul's and Ian's replies to it. Building community submission and acceptance management into the quotes and development timelines is an important measure to take, and doing some education up front really will help. Open source is different than other software development institutions may have contracted, so experienced parties may need more education than inexperienced ones.
As it is, I think vendors have a very hard time managing their own growth once it reaches the ballooning point. Unmanageable growth will kill you... as we have seen.
Yes, I've died a couple times already!
In the final analysis, if each vendor pursues their own direction, we will end up with a Baskin-Robins of Koha. Make the job of RM hard enough and no one will want it. At some point the project dies due to leanness and overextension. The strength of the project lies in the *two-way* cooperation of its members. The "we have developed it: take it or leave it" approach is a one-way, dead-end street for the community.
Thankfully, it doesn't ever actually work like that. No piece of code is accepted under the condition that it never be modified. We don't have a license like that. Say a submitter puts up a good bunch of code representing a considerable amount of work, but we are going to require that some hardcoded configuration X is abstracted into a syspref. If you asked them, "well, do you want to do it and resubmit, or do you want to wait an indefinite period until some random other person gets around to it, at which point you might have rebase issues?" I think 9 times of 10, they'll do it themselves. (Again, within reason. It's not going to happen if the hangup is something huge like "Make it work with UNIMARC in Turkish".) In short, extra signoff(s) are a fine recommendation and best practice, but I wouldn't support them as a strict requirement. Also I think concrete technical details about how the "right kind" of development can be done will do more good than just the policy preferences by themselves. --Joe
On Tue, Nov 2, 2010 at 11:08 PM, Joe Atzberger <ohiocore@gmail.com> wrote:
On Tue, Nov 2, 2010 at 4:12 PM, Chris Nighswonger <cnighswonger@foundations.edu> wrote:
On Tue, Nov 2, 2010 at 2:10 PM, Joe Atzberger <ohiocore@gmail.com> wrote:
Chris, I disagree that the first sign-off on a major vendor's patches should be external. The first sign-off from a major vendor should be *internal* to their quality control process. This was at one time the standing policy amongst LibLime and BibLibre for major changes.
My recommendation is that we require a *minimum* of one sign-off by a disinterested party. This in no way excludes a company having their own QA internally and signing off. Furthermore, it really provides no impedance to the entire process when one considers how simple it should be to obtain a sign-off on good working code.
Ok, I read that as amending "that the required sign-off for pushing to master be a disinterested developer" to "that a ...".
No, I think you may be reading it with an emphasis which I did not intend.
Regarding the simplicity of signing off, I take some issue. It is *severely* non-trivial to test major integration features. Consider SIP2 and LDAP, or something like EDI. It can depend not just on accurate test data, but entire servers, network environments, remote accounts granted by a supplier, foreign domain/language knowledge, etc. Sure, I'd love it for everybody to have a dozen signoffs. I just think blocking code while waiting on a 3rd party (who by design is disinterested) to come around and dedicate some resources is a questionable policy.
I'm sure there are any number of features which could be tested in very complex environments, and perhaps even more complex than those used or anticipated by the original developers themselves. I wonder if the original supplier of the SIP2 and LDAP features actually went to the level of testing you describe prior to committing those features. However, I do agree that testing should be defined in a way to ensure that it is both effective and yet not a de facto "block" on the acceptance of code.
Right now we can't even get comment on RFCs, let alone dedicated VMs and manpower.
The scant participation in this thread by others is very much proof of the problem pointed out here! I find it hard to believe how few have chimed in for as much "noise" as there is about these issues.
Ahh... nothing in my recommendation suggests that "should block *working code* from getting in just because a bigger, different or more-deluxe RFC exists." It simply suggests we have a policy in place which will actively promote some sort of community collaboration particularly on the part of large shops who "should" know better than to clam up especially on large feature development. That is patently bad behavior in the light of community participation which is the foundation of this project. It is certainly within the rights of anyone to take the source and run with it in whatever way they like. They may even take it and never contribute back. However, it is not within their right to do large, unilateral development and then expect it to be pushed into the main codebase.
Forgive me if I'm off the pulse a bit, but do these expectations exist today? The release process establishes when new features are accepted or not, and it has been pretty explicit and clear. The problem used to be big unilateral changes that weren't getting submitted (including some code that I wrote). Now the problem is they're getting submitted?
Perhaps this point bears greater clarification. By "expect it to be pushed" I meant particularly without any prior discussion/RFC/community participation. What was/is going on with LibLime/PTFS LEK is a classic example of the sort of thing which needs to be discouraged. I refer particularly to the process. PTFS has stated in a number of forums that once their "contract" obligations are met, they will submit this code to the community. However, the job is done at that point and clients will have implemented the product. There will be a certain level of "expectation" that "new" features "should" be pushed to the main code base. This sort of thing will probably be more prevalent as vendors become larger. This behavior may also be driven somewhat by the lack of a clear policy of inclusion. Simply having a policy of "whatever you dump on us by the (largely undefined) date that version X.Y.Z is to be released will be included" is a recipe for disaster in the long run.
Also, I think if you develop your features "in the open" (e.g., posted w/ gitweb, or on github), the burden of synthesizing multiple RFCs and general "feature consensus" sentiment isn't on you in quite the same way as when changes are delivered en bloc. A vendor has what their clients are paying for, and if other devs have an unfunded desire for extra feature X, they can follow the branch right along add the last bit themselves, all while still in development. Whether X is pulled in by the vendor, or separately submitted by the dev to the RM doesn't really matter.
A couple of points addressing this scenario as a whole (none of these is against the principle of open development):
1. That scenario may work for simpler features/code. But consider the current press to switch from zebra to 'foo' (any one of several suggestions recently). If a vendor develops an entire replacement for C4::Search and friends which centers around 'foo' in a unilateral fashion, the de facto assertion is that the community must either take what they have done or leave it.
Or patch it, or extend it, or put in on a feature branch, or break it into discrete elements, or defer acceptance until sufficiently abstracted, etc. If the vendor cares about it hitting mainline, then they'll follow up to do what it takes, within reason. If they don't, then I don't think new policy requirements will affect them in the least.
I agree that ultimately a vendor's desire to participate will be the deciding factor. But the fact that a policy won't affect a vendor is not necessarily a reason to not have a policy.
2. Given the problems we already have with a lack of development cooperation, this scenario at best does nothing to address those problems.
This comes back to the question of whether you can force cooperation. I don't think we can, effectively. I support codifying expectations and best practices, but "requiring" disinterested, competing or downright hostile parties to cooperate or pretend to cooperate seems destined to fail.
By the very nature of things, a project such as this entails a practical requirement of cooperation among competing parties. Just look at all of the vendors participating. While the vast majority are kind and well mannered toward each other, they are, in fact, in competition with each other in some sense of the word. And I'll not revisit my discussion of the fact that vendors and customers are in the business of "requiring" and "forcing cooperation" with each other all of the time in those little pieces of paper we call contracts. Now this is not to suggest that we begin any formal contractual relationships or that we attempt to "force" cooperation. But the recent work by ByWater and Software.coop on the EDI code and Catalyst and Biblibre on Biblibre's work illustrate that it is not beyond the realm of reason to expect and even strongly encourage competing parties to cooperate for the benefit of the thing that earns their bread and butter. And as for downright hostile parties, they should go elsewhere until they can leave off some of their hostility, imho.
3. This scenario appears a bit "vendor-centric."
That's because, as has been pointed out previously, Koha's authorship is de facto vendor-centric. The vast majority of Koha code historically originated from vendors and the majority of new code still comes from vendors. Every RM worked for a vendor (or two). I'm happy to see Koha institutions and cooperatives still able to contribute code back to the project on their own (some for longer than I've been associated with Koha), and I'd love to see more of it. Right now though, the code being affected by proposed policies is still *mostly* in vendors' houses.
In spite of this, I hope we can avoid becoming vendor-centric. "Vendor-centricity" leads to vendor dominance and control ultimately. I am not personally aware of a vendor controlled FOSS project that does not lean heavily in that vendor's favor. I furthermore hope that in the future we will see more individuals in my own position of vendor neutrality being elected to lead developer roles in the community. There are several who I can think of that would be very capable of filling these positions and help balance the trend you point out here.
I am of the opinion that Koha should be "community-centric" with individual's first and vendor's second in order of relationship.
OK, I know what a vendor is. But in this context I don't know what you mean by "community-centric" or "individual" or "order of relationship". If you just mean that community comes first, then yay, community. I suspect we can escape talking about whether vendors are first, second or other class citizens in the community because it doesn't really matter for the proposed policy.
The problem is not one of where to "relegate" vendors in the "social" structure of the community. Rather it is all about keeping vendors who have relatively limitless resources from holding the controlling interest in the community in whatever form that may take and at the same time, clearly defining the obligations of those individuals who *elect* to participate in the community in such a way that vendors can transact business with some amount of certainty.
This may not be the view of all involved. However, if it were not for Koha, Koha support vendors would be out of some amount of business.
And without vendors *no* version of Koha would ever have been written or released. It's sorta funny that I'm the one saying this stuff, as I am not currently affiliated with any vendor.
I think that 1.0 was written by coders for hire (aka Katipo) and was released by, not a vendor, but HLT, a library. So, in fact, no vendor in the technical understanding was involved. Chris C. can correct me if I'm wrong here.
4. Regarding the statement "A vendor has what their clients are paying for..." True, vendors are client driven. However, as I said in my initial post, vendors must educate their clients as to the nature of open source. Client expectations should be set based on known, published community procedures. If this were properly done, many problems would be resolved.
This is a good point, and I was interested to read Paul's and Ian's replies to it. Building community submission and acceptance management into the quotes and development timelines is an important measure to take, and doing some education up front really will help. Open source is different than other software development institutions may have contracted, so experienced parties may need more education than inexperienced ones.
I wish other vendor types would jump in. There sure has been enough.... complaining (/me ducks) that I would have thought this thread would be quite hot by now. As it is, I think you and I are just getting a workout in debate... :-)
As it is, I think vendors have a very hard time managing their own growth once it reaches the ballooning point. Unmanageable growth will kill you... as we have seen.
Yes, I've died a couple times already!
I hope you keep coming back!
In the final analysis, if each vendor pursues their own direction, we will end up with a Baskin-Robins of Koha. Make the job of RM hard enough and no one will want it. At some point the project dies due to leanness and overextension. The strength of the project lies in the *two-way* cooperation of its members. The "we have developed it: take it or leave it" approach is a one-way, dead-end street for the community.
Thankfully, it doesn't ever actually work like that. No piece of code is accepted under the condition that it never be modified. We don't have a license like that.
I understand the license and the ability to modify, but if you have a zillion lines of code dumped on you which you will have to somehow pick through in piece-meal fashion, extracting what you want and then integrating it with what you have, it is highly probable (as you mention, based on the facts of little to no responses to RFCs and small response to this thread) that that code will drop by the wayside and be left behind. That is not "two-way cooperation" in any definition I'm aware of. It's death by overdose.
Say a submitter puts up a good bunch of code representing a considerable amount of work, but we are going to require that some hardcoded configuration X is abstracted into a syspref. If you asked them, "well, do you want to do it and resubmit, or do you want to wait an indefinite period until some random other person gets around to it, at which point you might have rebase issues?" I think 9 times of 10, they'll do it themselves.
On this small scale, I agree.
(Again, within reason. It's not going to happen if the hangup is something huge like "Make it work with UNIMARC in Turkish".)
See the above paragraph.
In short, extra signoff(s) are a fine recommendation and best practice, but I wouldn't support them as a strict requirement. Also I think concrete technical details about how the "right kind" of development can be done will do more good than just the policy preferences by themselves.
I agree with you that we need concrete technical details about how the right kind of development can be done. That is the presupposition of this thread. However, I think they need to be formalized in written form (call that by whatever name you like the best). Kind Regards, Chris
On Mon, Nov 8, 2010 at 9:45 PM, Chris Nighswonger <cnighswonger@foundations.edu> wrote:
2. Given the problems we already have with a lack of development cooperation, this scenario at best does nothing to address those problems.
This comes back to the question of whether you can force cooperation. I don't think we can, effectively. I support codifying expectations and best practices, but "requiring" disinterested, competing or downright hostile parties to cooperate or pretend to cooperate seems destined to fail.
By the very nature of things, a project such as this entails a practical requirement of cooperation among competing parties. Just look at all of the vendors participating. While the vast majority are kind and well mannered toward each other, they are, in fact, in competition with each other in some sense of the word. And I'll not revisit my discussion of the fact that vendors and customers are in the business of "requiring" and "forcing cooperation" with each other all of the time in those little pieces of paper we call contracts. Now this is not to suggest that we begin any formal contractual relationships or that we attempt to "force" cooperation. But the recent work by ByWater and Software.coop on the EDI code and Catalyst and Biblibre on Biblibre's work illustrate that it is not beyond the realm of reason to expect and even strongly encourage competing parties to cooperate for the benefit of the thing that earns their bread and butter. And as for downright hostile parties, they should go elsewhere until they can leave off some of their hostility, imho.
This reply fits with a lot of what has been discussed in this thread and it's not even my own words. During a moment during the hackfest we talked about vendors and their customers' expectations. Someone (I can't remember who) mentioned that we have to educate our customers about the world they have just entered. Basically they can request features from us, but if those features are not guaranteed to be put into the final product the way they spec them out. Instead educating our customers that they have entered a new way of working and collaboration might make it a bit less likely that vendors run off on their own and work in a silo. Also Paul mentioned (and I love this) that he doesn't think he has any customers who would fight this process - that if he told them he can give them the feature they want but it needs to be changed a bit in this way or that they would probably agree to his suggestions because his customers are in it for the open source aspects and I hope everyone else's are. So, it doesn't really 'force' cooperation but it does make it easier for us to cooperate if our customers understand the process of getting their development ideas into the final releases of Koha. Nicole C. Engard ByWater Solutions
Chris Nighswonger wrote:
On Tue, Nov 2, 2010 at 11:08 PM, Joe Atzberger <ohiocore@gmail.com> wrote:
Right now we can't even get comment on RFCs, let alone dedicated VMs and manpower.
The scant participation in this thread by others is very much proof of the problem pointed out here! I find it hard to believe how few have chimed in for as much "noise" as there is about these issues.
!!! You hide this discussion in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying 'Beware of the Leopard' in an unlit cellar with broken stairs (or at least a subject line which shows only "A Discussion on A Policy Setting..." in my mailbox overview, which isn't really the sort of fun and attractive thread I rush to open, especially when others have already replied) and then grumble about low participation? Please excuse me while I pick my jaw off the floor. ;-) As you know, I only saw this thread now after you asked me to look for it... and even then, only at the second search attempt! I think this shares something with the low-comment RFCs. We authors should acknowledge how we're contributing to failure too: vague titles, fragmented discussion and information overload, to name three. Overload is a hard one to deal with, especially as people love Koha which maybe makes us too verbose, and summarising discussion is something probably Nicole can explain far better than me (judging by the great kohacon10 blog posts), so I'll take the vague titles one. Here's my tip on subject lines: http://mjr.towers.org.uk/email.html#subject "Good email has a good subject line. The subject line is your way to promote your message as one I should read first. Make it a short (max 10 words?) summary of what the email is about. Sometimes I look at "(no subject)" but not often if I don't know the sender's name. Stuff that looks like spam or viruses also gets mostly left unopened. Stuff with words like "URGENT" on the subject line often gets left until last, just to spite them [...] I know your email is important, but if I can't tell that it's important from the subject, I may delete it by mistake. So, if you don't hear back after a while, try resending with "RESEND" in the subject." Looking at this subject line with those glasses on: "A Discussion on A Policy Setting Forth Standards of Code Submission, etc. [WAS: RFCs for 3.4 from BibLibre (serials & acquisitions)]" deters in four ways: "Discussion" is redundant (this is a mailing list), "Policy" and "Standards" are unfun words, "etc." suggests a lack of focus, and the "WAS:..." suggests there's context that I probably don't remember. I feel a better title might be "RFC Code Submission Standards" and to link to the previous thread in the first paragraph, explaining if/how it's relevant. Some other advice may be in guides on how to write newspaper headlines or press release titles. Other than that, I simply re-emphasise my previous points that some recommendations seem unrealistic and I feel others are within the RM's power to decide and should be left there. Hope that helps, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Past Koha Release Manager (2.0), LMS programmer, statistician, webmaster. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for Koha work http://www.software.coop/products/koha
I apologize for the subject, line, but I was uncertain how to continue the thread when what I really want to respond to is the comment about " scant participation" issue. As a non-vendor, I have been unclear as to: 1. what is the "correct" way to respond to an RFC - is there a protocol? 2. where is the "correct" place to respond - on the list or on the wiki or where? 3. to whom do I respond? The community at large or the the vendor responsible for the RFC? 4. Ok, this is a weird one, I'll admit, but I've wondered at times whether my response would be welcomed or is this a "vendors-only" issue? -- Linda Culberson lculber@mdah.state.ms.us Archives and Records Services Division Ms. Dept. of Archives& History P. O. Box 571 Jackson, MS 39205-0571 Telephone: 601/576-6873 Facsimile: 601/576-6824
Linda, As a non-vendor, I have been unclear as to:
1. what is the "correct" way to respond to an RFC - is there a protocol?
I have been wondering this myself for a while now. I generally just comment via IRC, since I never know what else to do. That's definitely *not* the best way to do things, though.
2. where is the "correct" place to respond - on the list or on the wiki or where?
See above.
3. to whom do I respond? The community at large or the the vendor responsible for the RFC?
Errr... see above again.
4. Ok, this is a weird one, I'll admit, but I've wondered at times whether my response would be welcomed or is this a "vendors-only" issue?
Speaking for myself as a semi-vendor, I would welcome any and all comments, *particularly* those from non-vendor-type people, on my RFCs. Regards, Jared -- Jared Camins-Esakov Freelance bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
2010/11/9 Jared Camins-Esakov <jcamins@cpbibliography.com>:
1. what is the "correct" way to respond to an RFC - is there a protocol?
I have been wondering this myself for a while now. I generally just comment via IRC, since I never know what else to do. That's definitely *not* the best way to do things, though.
I don't know if there is an official policy but I've seen it done in two ways. The first is to reply to the email sent to the developers list. The other is to use the comments function on the wiki where you can discuss a specific page (aka RFC). While IRC works, it's not logged in the same place as the RFC so if others have comments and aren't on IRC at the same time as you things can get lost - so I'd stick to one of the above and if someone has more official knowledge they can let us know. Nicole C. Engard
Regarding the simplicity of signing off, I take some issue. It is *severely* non-trivial to test major integration features. Consider SIP2 and LDAP, or something like EDI. It can depend not just on accurate test data, but entire servers, network environments, remote accounts granted by a supplier, foreign domain/language knowledge, etc. Sure, I'd love it for everybody to have a dozen signoffs. I just think blocking code while waiting on a 3rd party (who by design is disinterested) to come around and dedicate some resources is a questionable policy.
I'm sure there are any number of features which could be tested in very complex environments, and perhaps even more complex than those used or anticipated by the original developers themselves. I wonder if the original supplier of the SIP2 and LDAP features actually went to the level of testing you describe prior to committing those features.
Since that was me in both cases, I can tell you: yes for SIP2, not so much for LDAP. Initially LDAP was written and tested against openldap, without any access to an Active Directory server, and it was a bitch. (Openldap has this horrible behavior of crashing the daemon completely if you give it a malformed command-line query or attempt to insert/edit a record unsuccessfully. It also simultaneously corrupts the data that is stored in compiled B-trees. Apparently performance was key, not reliability.) Only later did I test between VMs using Sun's OpenDirectory and remotely to ActiveDirectory. SIP2 testing was fairly robust, but required an extreme amount of data tuning to make it possible. I.E., a test requires that a patron have a $22 fine and an overdue item, so you have to make a user have a $22 fine and an overdue item. And then you cannot run overdue fines again. Ever. So basically that requires a dedicated instance.
Forgive me if I'm off the pulse a bit, but do these expectations exist today? The release process establishes when new features are accepted or not, and it has been pretty explicit and clear. The problem used to be big unilateral changes that weren't getting submitted (including some code that I wrote). Now the problem is they're getting submitted?
Perhaps this point bears greater clarification. By "expect it to be pushed" I meant particularly without any prior discussion/RFC/community participation. What was/is going on with LibLime/PTFS LEK is a classic example of the sort of thing which needs to be discouraged. I refer particularly to the process. PTFS has stated in a number of forums that once their "contract" obligations are met, they will submit this code to the community. However, the job is done at that point and clients will have implemented the product. There will be a certain level of "expectation" that "new" features "should" be pushed to the main code base.
On the plus side, this expectation is what should drive the vendor to submit the code in its most-likely-to-be-accepted form. In reality, the code that gets kept in the closet for a year or more *will* lose out. A vendor or their client may feel like they get a competitive advantage by deferring submission, but this is illusory. It just backloads and complicates the otherwise critical work of getting patches into mainline. While the patches are sitting around getting stale, if the feature is at all desirable, other people are working on similar or competing versions, and then extending the published version, bugfixing and documenting it. The work you do to extend or revise the withheld feature is quite possibly wasted effort. Resolving the competing implementations is often more work than it was to write either one of them, and could have been avoided entirely with earlier publication. Getting your patches in master is a *defensive* position that establishes your data model, API and presentation as accepted. But we can preach all day on this. Let's try not to.
In spite of this, I hope we can avoid becoming vendor-centric. "Vendor-centricity" leads to vendor dominance and control ultimately.
I am not personally aware of a vendor controlled FOSS project that
does not lean heavily in that vendor's favor.
I think you conflate the fact that vendors are primary players with the outcome of a single-vendor control. A healthy FOSS project has many players. Whether they come from different commercial interests, or different user bases, it doesn't really matter.
The problem is not one of where to "relegate" vendors in the "social" structure of the community. Rather it is all about keeping vendors who have relatively limitless resources from holding the controlling interest in the community ...
Your impression of a vendor's available resources is a bit fantastical. Having worked at both LibLime and Equinox, I can tell you neither company had more than 40 employees, with only a minority in development. If LibLime had such boundless resources, they would not have had to sell to PTFS. And PTFS, in terms of resources relevant to Koha, is not applying a team even half that size. Compared to clients like King County (WA), WALDO or GPLS, the operational capacity and physical resources of vendors (even PTFS) is scant.
This may not be the view of all involved. However, if it were not for Koha, Koha support vendors would be out of some amount of business.
And without vendors *no* version of Koha would ever have been written or released. It's sorta funny that I'm the one saying this stuff, as I am not currently affiliated with any vendor.
I think that 1.0 was written by coders for hire (aka Katipo) and was released by, not a vendor, but HLT, a library. So, in fact, no vendor in the technical understanding was involved. Chris C. can correct me if I'm wrong here.
Katipo was and is a vendor. They were not users of the software themselves. Instead they sold software development and services to HLT. This is the technical and practical definition of a vendor, and it is exactly the same as what other Koha development shops do, except Katipo was starting from scratch instead of an existing version. I'm not sure what additional indicators of vendorness you might be looking for. Perhaps an evil moustache? : }) --Joe
As a vendor-neutral voice, I would like to encourage everyone who has an vested interest in these areas and the best interests of the Koha project at heart to actively participate and respond to these RFC's. It seems that often there is little dicussion, etc. on RFCs in the community. And even when there is discussion, etc. it is often unclear if a consensus is reached (at least publicly). ++, ++, and more ++ ! That has been one of the main topic we spoke during the last morning of
Furthermore, I would encourage vendors and others who post RFC's to do so with a willingness to adapt, adjust, bend, compromise, and/or <your-favorite-term-goes-here> to positions on those RFC's which may be different but are clearly the consensus of the community at large. Vendors may and often do have the resources to implement "what they want," however, this is not in the spirit of cooperation which this project so greatly depends upon for its success. Clients of vendors should be educated during the RFQ process as to this aspect of open source, and their expectations managed accordingly, imho. I must add that (at least for BibLibre), our customers want to be a part of an OpenSource community. So if one of our customer ask for something
I would also suggest that we implement a policy that states in some agreable way that code/features will not be pushed to master which have not passed through a review and consensus process by the community and the RM (as the elected head of development by the community). No one excepting possibly the RM should presuppose that their code is guaranteed inclusion by default. I think it's better to have a review even for the RM (except for very small patches/obvious mistakes) Secondly, I would suggest that we implement a strong recommendation that larger shops submit timely RFC's *prior to* beginning work on code and then promote discussion on those RFC's. This recommendation should with some lesser strength suggest that everyone submit timely RFC's to maximize productivity and usefulness of the resources of all concerned. ++ We haven't started working on any of those RFCs (except solR, to have a
Thirdly, I would suggest a stated policy (and such a policy is presently in place practically) which requires all submissions to pass through a QA branch and receive at a minimum one sign-off prior to being pushed into master. This policy should also assign a certain amount of responsibility to the one signing off to avoid "frivolous" sign-offs. It should also, perhaps, include a restriction that the required sign-off for pushing to master be a disinterested developer perhaps from another vendor or the community at large. OK, except for obvious bugfixes/patches Another question : some librarians like liz started to test our branches, mainly the biggest one, and she find the features "awesome". How could we have librarian being more involved in QA from a functionnal
Le 02/11/2010 16:24, Chris Nighswonger a écrit : the hackfest (the other one being long term management of the project) Irma & Bob have taken notes, and should sent the minutes here soon. This meeting was, I hope, the start of a better management of our project. that is amended (or even rejected), we (BibLibre) are completly OK to reach him and explain : "well, your idea seems a wrong one, because of this, and that. So, either you confirm your request, and you already know the feature won't be a part of "Official Koha", you'll have your fork, or you update your request to have something useful for everybody". And i'm 95% sure that our customer(s) will answer : "well, ok, let's stay community oriented". The problems occurs for everybody if the "no-go" appears AFTER the dev has been done ! pain for the dev, pain for the library that has sponsored something that is not accepted in Koha (& pain for the community, because maybe, an amended RFC would have been OK). Sometimes ppl think/argue it's dangerous to trust "a community". But I always remember the "FLOSS moto" : given enough eyes, all bugs will be detected. Here it's not a matter of bug, but what may seem a good idea to a library may in fact not be one, and the community has enough eyes to see & argue it's a bad idea (& thus convince the original library to reconsider his request). Hint: in the RFCs I posted yesterday, I see at least one thing that could/should be amended. Very easy to amend & will definetly be a better idea. Let's see if someone find it ;-) proof of concept). What has really be a problem for us is that we published RFCs for Lyon3 university a long time ago (mail from Nicolas on koha-devel oct, 12, 2009), there has been strictly no reaction/feedback to those RFCs. Now they are done, and we have rebased them vs head (huge work, and huge QA to do, and probably a lot of time lost) Could they be rejected by the community ? hopefully I hope no, but I frankly don't know what we (BibLibre) could do if it were :-((( (because the customers are live now !) I think we (all) failed because Koha 3.2 was 9 months late. Well, in fact, I think the mistake was not to branch 3.4 immediatly on feature freeze. That would have been much less pain for us (that are customer-planning driven) (suggestion below). point of view ? (suggestions below)
This is a discussion we need to have. I would encourage everyone to invest time (the operative term here is 'invest') in this discussion.
SUGGESTIONS TO DISCUSS: * branch next version when the RM declare feature freeze for a given version * have a website rebuilded every night (week ?) (from which branch ? a waiting_librarian_feedback one ?), with all marc21 default values fitted in (with maybe a few biblios added), the librarians being requested to test from a functionnal point of view after the techies QA validation -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Le 02/11/2010 22:27, Paul Poulain a écrit :
SUGGESTIONS TO DISCUSS: * branch next version when the RM declare feature freeze for a given version * have a website rebuilded every night (week ?) (from which branch ? a waiting_librarian_feedback one ?), with all marc21 default values fitted in (with maybe a few biblios added), the librarians being requested to test from a functionnal point of view after the techies QA validation
I've added those topics, as well as a few others (like jqueryui) to the agenda of the next meeting (nov, 10th) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On Tue, Nov 2, 2010 at 5:57 PM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Le 02/11/2010 22:27, Paul Poulain a écrit :
SUGGESTIONS TO DISCUSS: * branch next version when the RM declare feature freeze for a given version
I'll let this one alone for now.
* have a website rebuilded every night (week ?) (from which branch ? a waiting_librarian_feedback one ?), with all marc21 default values fitted in (with maybe a few biblios added), the librarians being requested to test from a functionnal point of view after the techies QA validation
I think this is a great idea. If Hudson can do auto builds, surely we can setup a server with auto-built test installs for various branches. This would have to be limited to some reasonable number, though. Kind Regards, Chris
Just to throw in on this thread for ByWater Solutions: It is company policy to obtain at least one signoff from another staff member before submitting a patch to the community. We do not anticipate that this will be sufficient for inclusion into master (except perhaps on very simple patches), and hope other members of the community will sign off on our patches. We will do the same for others in the community as often as we can. The end goal of all our development is inclusion into master. We are willing to do whatever the Quality Assurance manager and Release Manager need to make our developments worthy of inclusion (provided we can do so and still meet our clients' needs). If this means recoding in a more generic way, so be it. All our work is done on separate branches, based off the current HEAD of master, and rebased/merged as needed to keep it applicable. You can see it at http://git.bywatersolutions.com. It would be helpful to have some kind of general guidelines for inclusion (i.e. follow the coding style rules, have working unit tests, include relevant documentation, etc.), even if in the end, it's up to the Release Manager's expert opinion. Perhaps this is something we can discuss in an IRC meeting? We've started on this (and must continue) to post all our feature ideas on the wiki. I'd like those to be public, working "open spec" that anyone is free to edit and enhance until someone can sponsor all or part of the feature(s). Even if no library has the money to pay for the creation of these features right now, we can all figure out what they'd need to be in order to meet our needs, worldwide. Perhaps the unsponsored RFCs can serve as a rough roadmap for the longterm of Koha (beyond the next release). I think running a demo site with code that's partially through the Quality Assurance process, and soliciting librarian feedback would be a good idea, provided we could actually get librarians to take time to test a system that's not theirs. That's all for now. It's been great meeting so many of you at KohaCon and the following hackfest. I look forward to working together with you all to make 3.4 something great. Now, off to explore some more of New Zealand before I have to head home. Cheers, -Ian 2010/11/2 Chris Nighswonger <cnighswonger@foundations.edu>
On Tue, Nov 2, 2010 at 5:57 PM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Le 02/11/2010 22:27, Paul Poulain a écrit :
SUGGESTIONS TO DISCUSS: * branch next version when the RM declare feature freeze for a given version
I'll let this one alone for now.
* have a website rebuilded every night (week ?) (from which branch ? a waiting_librarian_feedback one ?), with all marc21 default values fitted in (with maybe a few biblios added), the librarians being requested to test from a functionnal point of view after the techies QA validation
I think this is a great idea. If Hudson can do auto builds, surely we can setup a server with auto-built test installs for various branches. This would have to be limited to some reasonable number, though.
Kind Regards, Chris
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
On 3 November 2010 10:27, Paul Poulain <paul.poulain@biblibre.com> wrote:
We haven't started working on any of those RFCs (except solR, to have a proof of concept). What has really be a problem for us is that we published RFCs for Lyon3 university a long time ago (mail from Nicolas on koha-devel oct, 12, 2009), there has been strictly no reaction/feedback to those RFCs. Now they are done, and we have rebased them vs head (huge work, and huge QA to do, and probably a lot of time lost) Could they be rejected by the community ? hopefully I hope no, but I frankly don't know what we (BibLibre) could do if it were :-((( (because the customers are live now !) I think we (all) failed because Koha 3.2 was 9 months late. Well, in fact, I think the mistake was not to branch 3.4 immediatly on feature freeze. That would have been much less pain for us (that are customer-planning driven) (suggestion below).
What would have caused much much much less pain for you, was to develop your features in small branches, rather than one monolithic branch which makes rebasing much harder than it needs to be. This is a lesson that cannot be overstated, topic/bug/feature branches make everyones lives much easier. And they mean that if one feature is rejected ... then the whole stack doesn't need to be. I don't think branching sooner or an earlier release would have helped anywhere near as much as developing in smaller branches, not one huge one. Chris
Le 02/11/2010 23:05, Chris Cormack a écrit :
I think we (all) failed because Koha 3.2 was 9 months late. Well, in fact, I think the mistake was not to branch 3.4 immediatly on feature freeze. That would have been much less pain for us (that are customer-planning driven) (suggestion below).
What would have caused much much much less pain for you, was to develop your features in small branches, rather than one monolithic branch which makes rebasing much harder than it needs to be.
This is a lesson that cannot be overstated, topic/bug/feature branches make everyones lives much easier. And they mean that if one feature is rejected ... then the whole stack doesn't need to be.
agreed: we made a mistake here. (and don't plan to do it again !)
I don't think branching sooner or an earlier release would have helped anywhere near as much as developing in smaller branches, not one huge one.
Partially agreeing, I'd like to discuss of this topic on the next IRC meeting, i'm not fully sure I see clearly the best path (branching too early means that bugfix patches would have to be done on both branches) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
On 3 November 2010 11:10, Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 02/11/2010 23:05, Chris Cormack a écrit :
I think we (all) failed because Koha 3.2 was 9 months late. Well, in fact, I think the mistake was not to branch 3.4 immediatly on feature freeze. That would have been much less pain for us (that are customer-planning driven) (suggestion below).
What would have caused much much much less pain for you, was to develop your features in small branches, rather than one monolithic branch which makes rebasing much harder than it needs to be.
This is a lesson that cannot be overstated, topic/bug/feature branches make everyones lives much easier. And they mean that if one feature is rejected ... then the whole stack doesn't need to be.
agreed: we made a mistake here. (and don't plan to do it again !)
I don't think branching sooner or an earlier release would have helped anywhere near as much as developing in smaller branches, not one huge one.
Partially agreeing, I'd like to discuss of this topic on the next IRC meeting, i'm not fully sure I see clearly the best path (branching too early means that bugfix patches would have to be done on both branches)
Branching 3.2 earlier, in no way takes away the need to rebase, and so having smaller branches to rebase is still a much bigger win than a big branch to rebase. Branching earlier would not have meant the patches were anymore likely to go into master than they are now, they still have to go through QA etc, and having them in small feature set branches makes the chances of them passing much more likely. We can talk about this more at the meeting, but I am of the firm opinion branching earlier would have been of very little help in your situation. That's not to say its a bad idea, I just don't think it would have solved your problem. Chris
Releasing earlier is absolutely a factor in this, and I'm heartened that you've made it a priority for 3.4. Because Koha's API and schema lack consistency, abstraction, and isolation of concerns, adding nearly anything substantial demands that those elements change in ways that affect other areas radically. The amount of resources required to rebase dozens of individual feature branches when half of them require meddling with the key internals in way that will affect others increases in a non-linear fashion with the passage of time. Until Koha has a stable and more sophisticated API, short release cycles of working code is a necessity for developers who are creating lots of features stand a chance at being able to cooperate with each other in the long term. As is, it's very expensive for a developer to maintain a slew of feature branches compared to keeping a unified development trunk of their own. Clay On Tue, Nov 2, 2010 at 3:05 PM, Chris Cormack <chris@bigballofwax.co.nz>wrote:
On 3 November 2010 10:27, Paul Poulain <paul.poulain@biblibre.com> wrote:
We haven't started working on any of those RFCs (except solR, to have a proof of concept). What has really be a problem for us is that we published RFCs for Lyon3 university a long time ago (mail from Nicolas on koha-devel oct, 12, 2009), there has been strictly no reaction/feedback to those RFCs. Now they are done, and we have rebased them vs head (huge work, and huge QA to do, and probably a lot of time lost) Could they be rejected by the community ? hopefully I hope no, but I frankly don't know what we (BibLibre) could do if it were :-((( (because the customers are live now !) I think we (all) failed because Koha 3.2 was 9 months late. Well, in fact, I think the mistake was not to branch 3.4 immediatly on feature freeze. That would have been much less pain for us (that are customer-planning driven) (suggestion below).
What would have caused much much much less pain for you, was to develop your features in small branches, rather than one monolithic branch which makes rebasing much harder than it needs to be.
This is a lesson that cannot be overstated, topic/bug/feature branches make everyones lives much easier. And they mean that if one feature is rejected ... then the whole stack doesn't need to be.
I don't think branching sooner or an earlier release would have helped anywhere near as much as developing in smaller branches, not one huge one.
Chris _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Because Koha's API and schema lack consistency, abstraction, and isolation of concerns, adding nearly anything substantial demands that those elements change in ways that affect other areas radically. The amount of resources required to rebase dozens of individual feature branches when half of them require meddling with the key internals in way that will affect others increases in a non-linear fashion with the passage of time.
I agree. Rather than forming comitees, Koha community has to deal with software engenering challenges. A dump (and informal) rule should impose to any entity adding to Koha a large new feature to do also substantial code rationalization and cleanup. (I don't say it's easy...)
Because Koha's API and schema lack consistency, abstraction, and isolation of concerns, adding nearly anything substantial demands that those elements change in ways that affect other areas radically. The amount of resources required to rebase dozens of individual feature branches when half of them require meddling with the key internals in way that will affect others increases in a non-linear fashion with the passage of time.
I agree. Rather than forming comitees, Koha community has to deal with software engenering challenges. A dump (and informal) rule should impose to any entity adding to Koha a large new feature to do also substantial code rationalization and cleanup. (I don't say it's easy...) Doing so without sharing with community members or even working in pairs or groups with some of the parties interested on the way chosen is a waste of time for everyone. API changes would have to be done by concensus and could be eventually rejected and then the work done would be nullified. Comittee was maybe a misleading term or might not be what people think it is. What was finally proposed in the informal meeting on the last day was organising meetings on some technical issues in order to share work. Working on refactoring code or Data persistance, Plack usage or work on circular dependencies or DBIx::Class is a shared view and should be done in the open, could have consequences on the data structure and tasks could be quite easily be assigned to more than one company. Then charge could be also be shared communautary and become sustainable for everyone. a) multiple signoffs would become more evident. So integration into the code would be also more easy. and therefore, sustainability would be more achievable. b) the development cost would be shared by companies. drawbacks, it would require more time than working alone. Any library hiring a developer to work on Koha because they believe in
Le 10/11/2010 07:11, Frédéric Demians a écrit : the Free software,or any company (BibLibre but I think Tamil or Xercode, or even Catalyst) want their investment not to be lost. Even HLT have long "cried" on lost features from 1.0. Should we consider this is not an issue ? Friendly. -- Henri-Damien LAURENT
"Committee" in the sense that I'm reading you has the connotation of setting aside a space and time where a group of people can focus on a specific issue to discuss it and work toward building consensus. It seems some people have a knee-jerk response to the word as if it's an effort to establish a hegemonic regime bent on undercutting the release manager's role and authority. Frankly, Koha's long-term viability depends on creating an architectural vision more expansive than "the next release," and a working group focussed specifically on giving that vision some formal attention is a perfectly good way to approach that problem. Without an obvious candidate of eminence to establish a Linus-style authority, a working group/committee is probably the only viable approach to it. Yes, it will generally take longer to iron out differences and to code with an eye toward established best practices, but the extra short-term effort is necessary for long-term strategy. Clay On Wed, Nov 10, 2010 at 12:41 AM, LAURENT Henri-Damien < henridamien.laurent@gmail.com> wrote:
Because Koha's API and schema lack consistency, abstraction, and isolation of concerns, adding nearly anything substantial demands that those elements change in ways that affect other areas radically. The amount of resources required to rebase dozens of individual feature branches when half of them require meddling with the key internals in way that will affect others increases in a non-linear fashion with the passage of time.
I agree. Rather than forming comitees, Koha community has to deal with software engenering challenges. A dump (and informal) rule should impose to any entity adding to Koha a large new feature to do also substantial code rationalization and cleanup. (I don't say it's easy...) Doing so without sharing with community members or even working in pairs or groups with some of the parties interested on the way chosen is a waste of time for everyone. API changes would have to be done by concensus and could be eventually rejected and then the work done would be nullified. Comittee was maybe a misleading term or might not be what people think it is. What was finally proposed in the informal meeting on the last day was organising meetings on some technical issues in order to share work. Working on refactoring code or Data persistance, Plack usage or work on circular dependencies or DBIx::Class is a shared view and should be done in the open, could have consequences on the data structure and tasks could be quite easily be assigned to more than one company. Then charge could be also be shared communautary and become sustainable for everyone. a) multiple signoffs would become more evident. So integration into the code would be also more easy. and therefore, sustainability would be more achievable. b) the development cost would be shared by companies. drawbacks, it would require more time than working alone. Any library hiring a developer to work on Koha because they believe in
Le 10/11/2010 07:11, Frédéric Demians a écrit : the Free software,or any company (BibLibre but I think Tamil or Xercode, or even Catalyst) want their investment not to be lost. Even HLT have long "cried" on lost features from 1.0. Should we consider this is not an issue ? Friendly. -- Henri-Damien LAURENT _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Chris Nighswonger wrote:
As a vendor-neutral voice, I would like to encourage everyone who has an vested interest in these areas and the best interests of the Koha project at heart to actively participate and respond to these RFC's. It seems that often there is little dicussion, etc. on RFCs in the community. And even when there is discussion, etc. it is often unclear if a consensus is reached (at least publicly).
Why is there little discussion? I think low-comment RFCs are often posted in large batches. That is easier for the requestor but means that the same weekly average of commenter time gets spread between them all, resulting in a low level of discussion on each one. The current wiki isn't the easiest thing to track or edit and wikis are generally bad places for lengthy discussion. How might we remedy this? "RFC Corner" in the newsletter and meetings? How else?
[...] Clients of vendors should be educated during the RFQ process as to this aspect of open source, and their expectations managed accordingly, imho.
Well, we try, but free software vendors can't do this too much because some suppliers of "open source" disagree that these communities are good or even necessary. Community-friendly vendors would probably lose orders to them if we pushed the point harder because it would make us look slower. It needs to be done by vendor-neutral advisory services like www.oss-watch.ac.uk - anyone want to back one for Koha-Community.org? [...]
Secondly, I would suggest that we implement a strong recommendation that larger shops submit timely RFC's *prior to* beginning work on code and then promote discussion on those RFC's. This recommendation should with some lesser strength suggest that everyone submit timely RFC's to maximize productivity and usefulness of the resources of all concerned.
This will almost certainly not happen in some cases, such as where the ordering librarian wants the feature yesterday or as near as, so work is expected to start immediately when the order is placed, or in situations where the ordering librarian is new to FOSS and things like RFC processes. Although we don't do it and I hope no-one does, I also suggest there's a commercial incentive to hold back details in the hope of being paid twice for the same feature. If the RFCs ever helped gather orders (such as I'd like http://wiki.koha-community.org/wiki/NISO_CORE_protocol to do), then maybe some commercial incentive would push the other way, but how many examples of RFC-first development work have there been? Even when the RFC appears before coding, the order has already been placed. The new Template:RFC doesn't even have an option for an not-yet-ordered RFC. So while I feel the sentiment is good, I think it would be unrealistic to make this recommendation strong. I'd be delighted if we could work like that, but it's not what clients usually request (or pay for).
Thirdly, I would suggest a stated policy (and such a policy is presently in place practically) which requires all submissions to pass through a QA branch and receive at a minimum one sign-off prior to being pushed into master. [...]
I feel that this and the "review and consensus process" are both up to the RMs and should be part of their manifestoes (or not, as the case may be). It's a release management matter, not a general policy one. Do as you will. Hope that helps, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Past Koha Release Manager (2.0), LMS programmer, statistician, webmaster. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for Koha work http://www.software.coop/products/koha
On Mon, Nov 8, 2010 at 10:45 PM, MJ Ray <mjr@phonecoop.coop> wrote:
How might we remedy this? "RFC Corner" in the newsletter and meetings? How else?
I will gladly add a section of links to RFCs to the newsletter! I'm not sure how to start - should November newsletter include links to all RFCs? What do you all want to see? I can probably just put in a link to the RFC page for this newsletter ..
From now on if you submit a new RFC in a month please make sure you email me the link to the wiki page and I'll add it to the following newsletter.
Nicole C. Engard Documentation Manager
participants (12)
-
Chris Cormack -
Chris Nighswonger -
Clay Fouts -
Frédéric Demians -
Ian Walls -
Jared Camins-Esakov -
Joe Atzberger -
LAURENT Henri-Damien -
Linda Culberson -
MJ Ray -
Nicole Engard -
Paul Poulain