From nengard at gmail.com Tue Jun 1 18:39:30 2010 From: nengard at gmail.com (Nicole Engard) Date: Tue, 1 Jun 2010 12:39:30 -0400 Subject: [Koha-devel] Call for Koha Newsletter Articles Message-ID: Hello all, It's that time again, time to gather up all your Koha news (big and small) and send them to me for publication in our next issue of the Koha Newsletter. Articles can be as short as 1 line and as long as 20 (with links to read more content). They should include news about our Koha library, features you're working on, tips & tutorials you want to share. In short the sky's the limit. Please email me your articles by the 12th of June so that I can edit everything and get it organized for the June 15th release. Past newsletters can be found here: http://koha-community.org/category/koha-newsletter/ and you can subscribe to updates via RSS: http://feeds.feedburner.com/KohaNewsletter Thanks in advance, Nicole C. Engard Koha Documentation Manager/Newsletter Editor From nengard at gmail.com Tue Jun 1 19:38:28 2010 From: nengard at gmail.com (Nicole Engard) Date: Tue, 1 Jun 2010 13:38:28 -0400 Subject: [Koha-devel] dontmerge system preference Message-ID: Hi all, I want to update the template to show the right text, but I don't know for sure which is which and figure one of those of you who have done dev work will know: The dontmerge preference says "Don't automatically update attached biblios when changing an authority record. If this is off, please ask your administrator to enable the merge_authorities.pl cronjob". -- it shouldn't say OFF - what should it say? "if this is set to 'do'" - which is off? do or don't? Thanks Nicole From henridamien.laurent at gmail.com Tue Jun 1 21:49:11 2010 From: henridamien.laurent at gmail.com (LAURENT Henri-Damien) Date: Tue, 01 Jun 2010 21:49:11 +0200 Subject: [Koha-devel] dontmerge system preference In-Reply-To: References: Message-ID: <4C056437.7000209@gmail.com> Le 01/06/2010 19:38, Nicole Engard a ?crit : > Hi all, > > I want to update the template to show the right text, but I don't know > for sure which is which and figure one of those of you who have done > dev work will know: > > The dontmerge preference says "Don't automatically update attached > biblios when changing an authority record. If this is off, please ask > your administrator to enable the merge_authorities.pl cronjob". -- it > shouldn't say OFF - what should it say? "if this is set to 'do'" - > which is off? do or don't? > Hi Nicole, as far as that system preference is concerned, and in accordance to the Coding Guidelines which ask that NO systempreference should have a negative noun or effect, this preference is now replaced by MergeAuthoritiesOnUpdate which, if set edits all the related biblios and reports the heading into them. Hopes that helps. -- Henri-Damien LAURENT From nengard at gmail.com Tue Jun 1 21:55:07 2010 From: nengard at gmail.com (Nicole Engard) Date: Tue, 1 Jun 2010 15:55:07 -0400 Subject: [Koha-devel] dontmerge system preference In-Reply-To: <4C056437.7000209@gmail.com> References: <4C056437.7000209@gmail.com> Message-ID: On Tue, Jun 1, 2010 at 3:49 PM, LAURENT Henri-Damien wrote: > > Hi Nicole, > as far as that system preference is concerned, and in accordance to the > Coding Guidelines which ask that NO systempreference should have a > negative noun or effect, this preference is now replaced by > MergeAuthoritiesOnUpdate which, if set edits all the related biblios and > reports the heading into them. > Hopes that helps. This preference is not in HEAD as of yet - when it is there will dontmerge be removed? If so then it helps because we won't have that poorly worded preference summary anymore :) Nicole From gmcharlt at gmail.com Wed Jun 2 02:01:15 2010 From: gmcharlt at gmail.com (Galen Charlton) Date: Tue, 1 Jun 2010 20:01:15 -0400 Subject: [Koha-devel] Meeting reminder - general meeting, tomorrow, 2 June 2010, 10:00 UTC+0 Message-ID: Hi, The next general meeting of the Koha project is scheduled for tomorrow, 2 June 2010, at 10:00 UTC+0. The page for the meeting, including current agenda and links to time converters, can be found at: http://wiki.koha-community.org/wiki/General_Meeting,_June_2_2010 At present, the agenda is: 1. Update on Roadmap to 3.2. 2. Update on Roadmap to 3.0. 3. Follow-up on actions from General Meeting, May 5 2010. 4. Discuss switching licensing for all new code submissions to Affero General Public License (AGPL) 1. AGPL does not prevent unfriendly vendor lock-in (still achievable through access control, particularly to the databases) while introducing onerous burdens on friendly hosters. http://lists.katipo.co.nz/pipermail/koha/2010-May/023816.html 5. (If time permits) Revisit vendor listing requirements on koha-community.org. 1. Anti-privatisation requirement (private domains or trademarks, legal proceedings) http://lists.katipo.co.nz/pipermail/koha/2010-May/023785.html 2. Linkback requirement - also http://lists.katipo.co.nz/pipermail/koha/2010-May/023785.html 6. info at koha.org address: various places reference an info at koha.org address whose destination is unknown. Should we have an info at koha-community.org? If so, where should it go? 1. forward to the main Koha mailing list? 2. replace references to info at koha.org with themain Koha mailing list address? 3. establish a new koha-info mailing list that info at koha-community.org forwards to? 4. have it be an auto-responder that directs the inquiry to the project website? 7. Agree times of next meetings. Regards, Galen -- Galen Charlton gmcharlt at gmail.com From reed at catalyst.net.nz Wed Jun 2 03:49:16 2010 From: reed at catalyst.net.nz (Reed Wade) Date: Wed, 2 Jun 2010 13:49:16 +1200 Subject: [Koha-devel] generic Koha data export format? Message-ID: Maybe this is already solved but-- It seems like it would be fun if there was a nicely generic format defined for Koha data dumps. Is it correct that right now this is a DB specific task? I'm thinking some format that would be good for-- - working across different versions of Koha (within reason) - portable backups - a stable interchange format when migrating between Koha and other ILSs - the case where some library is running v2 and it's really, really finally time for them to move to v12 and they now have the resources to do it -reed From rick at praxis.com.au Wed Jun 2 04:18:50 2010 From: rick at praxis.com.au (Rick Welykochy) Date: Wed, 02 Jun 2010 12:18:50 +1000 Subject: [Koha-devel] generic Koha data export format? In-Reply-To: References: Message-ID: <4C05BF8A.50903@praxis.com.au> Reed Wade wrote: > It seems like it would be fun if there was a nicely generic format > defined for Koha data dumps. Is it correct that right now this is a DB > specific task? This is a good idea, esp. when migrating data. Two simple, popular and powerful formats come to mind: 1. CSV, directly importable into spreadsheet and database programmes 2. XML, easily handled by MySQL, more transparent than CSV cheers rickw -- _________________________________ Rick Welykochy || Praxis Services From paul.poulain at biblibre.com Wed Jun 2 09:59:24 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Wed, 02 Jun 2010 09:59:24 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions Message-ID: <4C060F5C.9010402@biblibre.com> Hi, In this mail, I want to share with you our situation, our problems, and our ideas. You'll be probably surprised by this mail. Please consider it's a good news. It's not a threat - not my cup of tea (frenchism suspected) - . But I think things must be done this way, so I share it with you, and I'm impatiently waiting for koha-community reaction & thoughts. BibLibre situation =========== We are very successful. Koha is really a success in France. One french university is now live, we are on the way to migrate a very large public library (500k items), 3 other universities should go live in September, and a lot of business coming. We are 11, are on our way to hire 3 new ppl, have on board Philippe Chabanon, that is very experienced in managing a company, is fully enthusiastic about free softwares and community, and knows a lot of people in the french library world. Is everything perfect ? nope. Our problems ========= Organizational problem ------------ Some of our clients sponsor new developments. Those are done under "march? public" rules, that usually define a very strict date (if we fail, then we have to pay fines, based on how many days late we are !). That's why we started 3 different branches for our 3 different major sponsored works (Aix-Marseille, N?mes, Lyon 3, you can see everything on git.biblibre.com). Everything that was in Aix-Marseille branch is now in koha 3.2 . But we have a lot of things done for Lyon 3 (that is live already) and Nimes (that will go live in early july) that are not in koha 3.2 Our main short-term problem is that koha 3.2 is very (very) late. It has been announced one year ago ! We did a lot of testing for Lyon 3 before putting them live. Some of them resulted in some bugfixes that could/should have been submitted to koha mainstream. But, we couldn't. Thus we face a situation where someone fixes in official 3.2 some things we already fixed in our Lyon3 branch (shame on us, but really, we couldn't find time. Delivery to Lyon3 was on time, but is was shorter than short) The situation is the same with Nimes : they will probably be live (with 3.2 + Lyon3 + Nimes sponsored stuff) before 3.2 is released -going live planned for july 1st, Galen, we challenge you ;-) -. So, frankly, dealing with our contracts and the official version of Koha has been a pain, and has resulted in a "virtual fork" for Lyon3 (and in 1 month for Nimes). We strongly want to submit our code to the main trunk, but dealing with a so-long-delayed version is really a nightmare (and I understand PTFS deploying "Harley" -even if i share community opinion about the version being advertised when released in a way it can be understood as an official version-). We must find a solution to this situation, or the "virtual fork" will result in a "real fork" that we strongly want to avoid ! As of today, we have something like 400 patches to integrate into koha main trunk ! Technical problem ----------- With some customers having more than 1 million items (Aix-Marseille : 1247686), and with more and more customers, we reach some limits with the structure of Koha, in term of performances and stability/maintainability. Chris suggested some improvements with koha 3.4, and we have some complementary ideas, I want to share with you, and would like to implement ASAP. Ideas/proposal ======= Technical : Koha architecture --------------------------------- At the begining of may 2010, we staff had a meeting at Marseille to know if it would be possible to make koha easier to maintain and evolve, we are now doing some experiment to split koha into several layers that provides completely independant set of tasks based on the MVC architecture (http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller). Koha can be seen as a stack with communication between the closest layers. 4 layers at all (does it make you remember TCP/IP? ;) ) VIEW: interactions with different clients ( browser, REST, XMPP, Z3950, ... ) ?? CONTROLLER: Controls the model by transmitting the CRUD tasks requested by the VIEW ?? MODEL: Business objects, as close as possible to the real world ?? PERSISTENCE: Make the Business objects persistent Plus, we expect to use external tools as often as possible to take experience of other communities and reduce the amount of code we have to debug by ourselves. We think tools exists for every layers, actually we just have to choose between them: View: ------- HTML::Template, Template Toolkit 2, Template::Declare, SOAP::Lite, any REST, XMPP, Z3950, SRU/SRW wrapper, ... (Infobot ;)) Controller: ---------- Dancer, Mojo, The Controller parts of existing MVC frameworks ( Jifty, Catalyst, ...), ... Model: ------- Any OO Framework ( prefer those which comes with persistence extension as MooseX::Storage, KiokuDB, ... choose one ) Persistence: ----------- DBIx::Class, Document DB clients, SQL::Abstract, Jifty::DBI, .... Ideally, the communication between each layers must be specified, provable by unit testing, and well documented ( pods in the t/* ?). Today, Koha works in CGI mode, so there is no persistence at all. And having no persistence means we can't implement the MVC model -some tests we did with OO frameworks show very poor performances-, and won't be able to improve performance heavily -the main loss of perf in koha as of today is loading datas that are almost static & should be available immediatly : systempreferences & all the admin tables -. So: 1st step => remove CGI, to have persistent code, enabling persistent DB connection. We did some tests with Dancer, that are very encouraring, we will send a mail about that soon. 2nd step => Once we will be OK with dancer to get rid of CGI => head to persistence & OO framework. We vote for Moose & KiokuDB, but we still have POC to write to confirm the idea. 3rd step =>view part, by moving to T::T (POC still to do to confirm the idea) It may look like a koha rewrite but it's not: every parts would be added step by step into koha by wrapping the new design behind the old API during a transition phase. An example of current code sub GetBranchesInCategory($) { my ($categorycode) = @_; my @branches; my $dbh = C4::Context->dbh(); my $sth = $dbh->prepare( "SELECT b.branchcode FROM branchrelations r, branches b where r.branchcode=b.branchcode and r.categorycode=?" ); $sth->execute($categorycode); while ( my $branch = $sth->fetchrow ) { push @branches, $branch; } $sth->finish(); return ( \@branches ); } would became transition code: sub GetBranchesInCategory($) { my $id = shift or return; # - Category would be the Business object with a member transition (which can be # deprecated at the end of transition) that encapsulate every # persistance specific data used in the C4::* api # - the find method could be delegated to the DBIx::Class find that returns the # record associated to the id # returns if ( my $category = Category->transition->find($id) ) { # codes to deal with branches comes de facto with tools as DBIx::Class # it returns the list of branches for the category $category->branches; } else { undef } } You can find in attachement a small graph with all our ideas, and the beginning of a timeline. ppl reading carefully will see a "changing indexing system (solr)". 4 years after starting with it, we still find zebra being a nightmare, with a lot of problems for unicode (with ICU). We did some tests with solr, and they are very encouraging. We will provide the results soon too. But i'm really enthusiast about what we did in a few hours ! Organizational : Koha team ------------------------------- Koha team organisation ............... If we (BibLibre & the Koha community) want to be able to implement such a roadmap (that also includes some new functionnality, as customers sponsors some...), we must have a heavily organised team. I think we must : * manage release by date (not by features). This is probably the more important part for us (see our "march? public" problem). The timing of the versions would be defined by the companies contributing the most and synched with sponsored devs roadmap. That will be tricky to organize when there are more than one company involved, but, for instance, BibLibre is the largest contributor ( PTFS: you're welcomed to enter the thread ! ). In fact, "a koha 3.4" could be released in a few weeks (look at out master branch, it contains 3.2 + Lyon3 + Nimes sponsored work). A "koha 3.6" could be released in december/february,... a regular release, that's something that is probably "non-negociable" for us. * define very clear coding guidelines & be very stricts on their use : everything that does not respect the guidelines must be politely but firmly rejected. * have a very efficient patch workflow : a patch can't be in the wild for weeks. The submitter must have an answer in a very short timeline (is 48H possible ?). ie : We can't rely on only one person to push patches. We must have back-ups for major roles like RM & "patch pusher". * we could divide Koha in functionnal (serials / acquisition / cataloguing / ...) or technical (perfs / SQL structure / view layer / indexing engine / ...) parts, with some working group for each of them, and something like a monthly report of the WG (with a weekly internal meeting) * write tests ! technical tests (.t) and functional tests (selenium ?) Our involvement (BibLibre) : .................... We want to improve our involvement in Koha, by dedicating Henri-Damien to work at 75% for Koha and be the link between the work done internally by BibLibre, and the work done by everyone else. This mean : * publish our RFPs as soon as possible * gather all the work done every month and communicate about that on the wiki and blog and translate on koha-fr.org * manage patches coming out from BibLibre, and coming out from everyone else (to deal as soon as possible with any problem) I know we have elected a team recently, and I know chris will do an awesome job as RM (I don't know for others, I didn't saw them in their responsability yet. For chris, i'm the team for 8 years, I know who he is ;-) ), but I think hdl should have a responsability more important than "bug wrangler" to reflect the time he (will) dedicate to the project. Is something like "co-RM" possible ? any other idea ? Any other idea ? Next step ----------- We are *really* willing to implement this roadmap and way of doing things. The goal of this mail is to ask everybody involved in Koha development about their opinion: we want our proposal to be a chance for Koha (new deal ?). On the technical side, stay tuned. Some POC will fall in the next weeks !!! On the organizational side, the ball is on your side: let's go to update/improve our organization ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -------------- section suivante -------------- Une pi?ce jointe HTML a ?t? nettoy?e... URL: From chris at bigballofwax.co.nz Wed Jun 2 10:25:05 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Wed, 2 Jun 2010 20:25:05 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C060F5C.9010402@biblibre.com> References: <4C060F5C.9010402@biblibre.com> Message-ID: Hi Paul There is an IRC meeting today, in 3 hours time .. but this topic is much to big for that, so I propose we leave it off the agenda for that, and let people think about the ideas raised for a bit, and then do replies. Speaking for myself nothing in the email strikes me as bad news, I do think time based releases cause their own set of problems, and as RM elect I think if we decided on that we would certainly have to become a LOT stricter about patches. IE, we would need to reject more and faster .. which might not be a bad thing anyway. I also don't think bug wrangler is a small job .. .anyone who thinks that hasnt looked in bugzilla recently ;-) Some of the main tasks of a wrangler are just what you describe Henri-Damien as doing, ie, managing integration of bug fixes and new features. One thing that would allow patches to be accepted a lot faster would be making every patch need to contain a working test. One thing I am considering is borrowing from the Java people and getting Hudson up and running http://wiki.hudson-ci.org/display/HUDSON/Meet+Hudson Of course, we need unit tests to make this useful. As for the technical part of managing many branches in git, we have significant experience of that at work, in the team I work on every single feature, or bug, is in its' own branch and they are merged into testing releases (we have literally hundreds of branches). Git makes this easy, so I think the use of many more feature branches at git.koha-community.org will make merging a lot easier, this means even virtual forks are only just a branch. As to the speed of pushing patches, test cases and good commit messages would improve that immensely. But I am confident that with the help of a QA manager, which poor Galen never had, we can accept or reject patches in a timely manner. The better your commit message the easier that gets, if we have to read all the code just to try and figure out what its trying to do, then to figure out if it is actually doing it .. that will slow things down immensely. So as RM I accept the challenge of faster releases, and throw back a challenge of better patches :) Chris 2010/6/2 Paul Poulain : > Hi, > > In this mail, I want to share with you our situation, our problems, and our > ideas. > You'll be probably surprised by this mail. Please consider it's a good news. > It's not a threat - not my cup of tea (frenchism suspected) - . But I think > things must be done this way, so I share it with you, and I'm impatiently > waiting for koha-community reaction & thoughts. > > BibLibre situation > =========== > We are very successful. Koha is really a success in France. One french > university is now live, we are on the way to migrate a very large public > library (500k items), 3 other universities should go live in September, and > a lot of business coming. > We are 11, are on our way to hire 3 new ppl, have on board Philippe > Chabanon, that is very experienced in managing a company, is fully > enthusiastic about free softwares and community, and knows a lot of people > in the french library world. > Is everything perfect ? nope. > > Our problems > ========= > Organizational problem > ------------ > Some of our clients sponsor new developments. Those are done under "march? > public" rules, that usually define a very strict date (if we fail, then we > have to pay fines, based on how many days late we are !). That's why we > started 3 different branches for our 3 different major sponsored works > (Aix-Marseille, N?mes, Lyon 3, you can see everything on git.biblibre.com). > Everything that was in Aix-Marseille branch is now in koha 3.2 . But we have > a lot of things done for Lyon 3 (that is live already) and Nimes (that will > go live in early july) that are not in koha 3.2 > > Our main short-term problem is that koha 3.2 is very (very) late. It has > been announced one year ago ! We did a lot of testing for Lyon 3 before > putting them live. Some of them resulted in some bugfixes that could/should > have been submitted to koha mainstream. But, we couldn't. Thus we face a > situation where someone fixes in official 3.2 some things we already fixed > in our Lyon3 branch (shame on us, but really, we couldn't find time. > Delivery to Lyon3 was on time, but is was shorter than short) > The situation is the same with Nimes : they will probably be live (with 3.2 > + Lyon3 + Nimes sponsored stuff) before 3.2 is released -going live planned > for july 1st, Galen, we challenge you ;-) -. > > So, frankly, dealing with our contracts and the official version of Koha has > been a pain, and has resulted in a "virtual fork" for Lyon3 (and in 1 month > for Nimes). We strongly want to submit our code to the main trunk, but > dealing with a so-long-delayed version is really a nightmare (and I > understand PTFS deploying "Harley" -even if i share community opinion about > the version being advertised when released in a way it can be understood as > an official version-). We must find a solution to this situation, or the > "virtual fork" will result in a "real fork" that we strongly want to avoid ! > > As of today, we have something like 400 patches to integrate into koha main > trunk ! > > Technical problem > ----------- > With some customers having more than 1 million items (Aix-Marseille : > 1247686), and with more and more customers, we reach some limits with the > structure of Koha, in term of performances and stability/maintainability. > Chris suggested some improvements with koha 3.4, and we have some > complementary ideas, I want to share with you, and would like to implement > ASAP. > > Ideas/proposal > ======= > Technical : Koha architecture > --------------------------------- > At the begining of may 2010, we staff had a meeting at Marseille to know if > it would be possible to make koha easier to maintain and evolve, we are now > doing some experiment to split koha into several layers that provides > completely independant set of tasks based on the MVC architecture > (http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller). > > Koha can be seen as a stack with communication between the closest layers. 4 > layers at all (does it make you remember TCP/IP? ;) ) > > VIEW: interactions with different clients ( browser, REST, XMPP, Z3950, ... > ) > ?? > CONTROLLER: Controls the model by transmitting the CRUD tasks requested by > the VIEW > ?? > MODEL: Business objects, as close as possible to the real world > ?? > PERSISTENCE: Make the Business objects persistent > > Plus, we expect to use external tools as often as possible to take > experience of other communities and reduce the amount of code we have to > debug by ourselves. We think tools exists for every layers, actually we just > have to > choose between them: > > View: > ------- > HTML::Template, Template Toolkit 2, Template::Declare, SOAP::Lite, > any REST, XMPP, Z3950, SRU/SRW wrapper, ... (Infobot ;)) > > Controller: > ---------- > Dancer, Mojo, The Controller parts of existing MVC frameworks ( > Jifty, Catalyst, ...), ... > > Model: > ------- > Any OO Framework ( prefer those which comes with persistence extension as > MooseX::Storage, KiokuDB, ... choose one ) > > Persistence: > ----------- > DBIx::Class, Document DB clients, SQL::Abstract, Jifty::DBI, .... > > Ideally, the communication between each layers must be specified, provable > by unit testing, and well documented ( pods in the t/* ?). > > Today, Koha works in CGI mode, so there is no persistence at all. And having > no persistence means we can't implement the MVC model -some tests we did > with OO frameworks show very poor performances-, and won't be able to > improve performance heavily -the main loss of perf in koha as of today is > loading datas that are almost static & should be available immediatly : > systempreferences & all the admin tables -. > > So: 1st step => remove CGI, to have persistent code, enabling persistent DB > connection. We did some tests with Dancer, that are very encouraring, we > will send a mail about that soon. > 2nd step => Once we will be OK with dancer to get rid of CGI => head to > persistence & OO framework. We vote for Moose & KiokuDB, but we still have > POC to write to confirm the idea. > 3rd step =>view part, by moving to T::T (POC still to do to confirm the > idea) > > It may look like a koha rewrite but it's not: every parts would be added > step by step into koha by wrapping the new design behind the old API during > a transition phase. An example of current code > > sub GetBranchesInCategory($) { > my ($categorycode) = @_; > my @branches; > my $dbh = C4::Context->dbh(); > my $sth = $dbh->prepare( > "SELECT b.branchcode > FROM branchrelations r, branches b > where r.branchcode=b.branchcode and r.categorycode=?" > ); > $sth->execute($categorycode); > while ( my $branch = $sth->fetchrow ) { > push @branches, $branch; > } > $sth->finish(); > return ( \@branches ); > } > > would became transition code: > > sub GetBranchesInCategory($) { > my $id = shift > or return; > > # - Category would be the Business object with a member transition (which > can be > # deprecated at the end of transition) that encapsulate every > # persistance specific data used in the C4::* api > # - the find method could be delegated to the DBIx::Class find that returns > the > # record associated to the id > > # returns > if ( my $category = Category->transition->find($id) ) { > # codes to deal with branches comes de facto with tools as DBIx::Class > # it returns the list of branches for the category > $category->branches; > } > else { undef } > } > > You can find in attachement a small graph with all our ideas, and the > beginning of a timeline. > > ppl reading carefully will see a "changing indexing system (solr)". 4 years > after starting with it, we still find zebra being a nightmare, with a lot of > problems for unicode (with ICU). We did some tests with solr, and they are > very encouraging. We will provide the results soon too. But i'm really > enthusiast about what we did in a few hours ! > > Organizational : Koha team > ------------------------------- > ??? Koha team organisation > ??? ............... > If we (BibLibre & the Koha community) want to be able to implement such a > roadmap (that also includes some new functionnality, as customers sponsors > some...), we must have a heavily organised team. I think we must : > * manage release by date (not by features). This is probably the more > important part for us (see our "march? public" problem). The timing of the > versions would be defined by the companies contributing the most and synched > with sponsored devs roadmap. That will be tricky to organize when there are > more than one company involved, but, for instance, BibLibre is the largest > contributor ( PTFS: you're welcomed to enter the thread ! ). In fact, "a > koha 3.4" could be released in a few weeks (look at out master branch, it > contains 3.2 + Lyon3 + Nimes sponsored work). A "koha 3.6" could be released > in december/february,... a regular release, that's something that is > probably "non-negociable" for us. > * define very clear coding guidelines & be very stricts on their use : > everything that does not respect the guidelines must be politely but firmly > rejected. > * have a very efficient patch workflow : a patch can't be in the wild for > weeks. The submitter must have an answer in a very short timeline (is 48H > possible ?). ie : We can't rely on only one person to push patches. We must > have back-ups for major roles like RM & "patch pusher". > * we could divide Koha in functionnal (serials / acquisition / cataloguing / > ...) or technical (perfs / SQL structure / view layer / indexing engine / > ...) parts, with some working group for each of them, and something like a > monthly report of the WG (with a weekly internal meeting) > * write tests ! technical tests (.t) and functional tests (selenium ?) > > ??? Our involvement (BibLibre) : > ?? ?.................... > We want to improve our involvement in Koha, by dedicating Henri-Damien to > work at 75% for Koha and be the link between the work done internally by > BibLibre, and the work done by everyone else. This mean : > > publish our RFPs as soon as possible > gather all the work done every month and communicate about that on the wiki > and blog and translate on koha-fr.org > manage patches coming out from BibLibre, and coming out from everyone else > (to deal as soon as possible with any problem) > > I know we have elected a team recently, and I know chris will do an awesome > job as RM (I don't know for others, I didn't saw them in their > responsability yet. For chris, i'm the team for 8 years, I know who he is > ;-) ), but I think hdl should have a responsability more important than "bug > wrangler" to reflect the time he (will) dedicate to the project. Is > something like "co-RM" possible ? any other idea ? Any other idea ? > > Next step > ----------- > We are *really* willing to implement this roadmap and way of doing things. > The goal of this mail is to ask everybody involved in Koha development about > their opinion: we want our proposal to be a chance for Koha (new deal ?). > > On the technical side, stay tuned. Some POC will fall in the next weeks !!! > On the organizational side, the ball is on your side: let's go to > update/improve our organization ! > > -- > Paul POULAIN > http://www.biblibre.com > Expert en Logiciels Libres pour l'info-doc > Tel : (33) 4 91 81 35 08 > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > From marc.chantreux at biblibre.com Wed Jun 2 11:38:54 2010 From: marc.chantreux at biblibre.com (Marc Chantreux) Date: Wed, 2 Jun 2010 11:38:54 +0200 Subject: [Koha-devel] generic Koha data export format? In-Reply-To: <4C05BF8A.50903@praxis.com.au> References: <4C05BF8A.50903@praxis.com.au> Message-ID: <20100602093854.GA863@auckland.lan> On Wed, Jun 02, 2010 at 12:18:50PM +1000, Rick Welykochy wrote: > 1. CSV, directly importable into spreadsheet and database programmes > 2. XML, easily handled by MySQL, more transparent than CSV I would choose YAML because - it can store tree (as xml) without the need of extra tags (required in xml) - it's human readable/editable - it supports pointed structures - it's now supported everywhere HTH -- Marc Chantreux BibLibre, expert en logiciels libres pour l'info-doc http://biblibre.com From colin.campbell at ptfs-europe.com Wed Jun 2 11:58:07 2010 From: colin.campbell at ptfs-europe.com (Colin Campbell) Date: Wed, 02 Jun 2010 10:58:07 +0100 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C060F5C.9010402@biblibre.com> References: <4C060F5C.9010402@biblibre.com> Message-ID: <4C062B2F.80407@ptfs-europe.com> On 02/06/10 08:59, Paul Poulain wrote: > Hi, > > In this mail, I want to share with you our situation, our problems, and > our ideas. Thanks to Paul for triggering a needed discussion. The opportunities for Koha seem to be very big at the moment throughout the world and we are facing a lot of big decisions. I think the issues of more frequent releases is becoming more pressing. Increasingly we're all deploying 3.1. somethings or pre 3.2s or alphas because we need to respond to the needs of libraries more swiftly. Looking at perl's experience of having regular scheduled releases has convinced me that this is a good idea. The traditional feature driven release is driven more by marketing than user needs. Even if we can get regular unstable release points that might enables us to progress more constructively. I think we need to start thinking about technical and organizational issues for beyond 3.2 now. Possibly we should start working toward a 3.4 technical summit on irc where we could see if we can thrash out a plan. Cheers Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell at ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com From henridamien.laurent at gmail.com Wed Jun 2 12:08:42 2010 From: henridamien.laurent at gmail.com (LAURENT Henri-Damien) Date: Wed, 02 Jun 2010 12:08:42 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> Message-ID: <4C062DAA.70607@gmail.com> Le 02/06/2010 10:25, Chris Cormack a ?crit : > Hi Paul > > There is an IRC meeting today, in 3 hours time .. but this topic is > much to big for that, so I propose we leave it off the agenda for > that, and let people think about the ideas raised for a bit, and then > do replies. Hi chris, this mail was sent today so that people can react, the intent was not to debate of that during today's meeting (agenda is already online) but to raise our questions and tell our point of view. Anyway, decisions and actions will have to be taken forward. > > Speaking for myself nothing in the email strikes me as bad news, I do > think time based releases cause their own set of problems, and as RM > elect I think if we decided on that we would certainly have to become > a LOT stricter about patches. IE, we would need to reject more and > faster .. which might not be a bad thing anyway. > I also don't think bug wrangler is a small job .. .anyone who thinks > that hasnt looked in bugzilla recently ;-) Some of the main tasks of a > wrangler are just what you describe Henri-Damien as doing, ie, > managing integration of bug fixes and new features. > > One thing that would allow patches to be accepted a lot faster would > be making every patch need to contain a working test. One thing I am > considering is borrowing from the Java people and getting Hudson up > and running http://wiki.hudson-ci.org/display/HUDSON/Meet+Hudson > Of course, we need unit tests to make this useful. > > As for the technical part of managing many branches in git, we have > significant experience of that at work, in the team I work on every > single feature, or bug, is in its' own branch and they are merged into > testing releases (we have literally hundreds of branches). Git makes > this easy, so I think the use of many more feature branches at > git.koha-community.org will make merging a lot easier, this means even > virtual forks are only just a branch. The way koha code is now is making that troublesome since when we fix some circulation stuff or holding stuff, we have to touch many files, so that when rebasing, we have many conflicts. Moreover, since our customers ask for features and not only bug fixes, and want a kind of relesase when they are delivered, we have to merge both devs and bugfixes in order to achieve global stability and coherence betweeen devs and bug fixes. Some devs we did also helped us in fixing bugs quicker, simpler and better. So that if we ported that to master back, on order to ensure that nothing is broken, i need more than just a few hours, would require me not just cherry-picking but also some kind of refactoring and then would introduce some risks both in koha-community master branch (overlooked bug) and in our branch when we rebase on koha-community master branch. This is why we also propose some new tools in order to improve coding guidelines and eventually have some way to automate testing. > > As to the speed of pushing patches, test cases and good commit > messages would improve that immensely. But I am confident that with > the help of a QA manager, which poor Galen never had, we can accept or > reject patches in a timely manner. The better your commit message the > easier that gets, if we have to read all the code just to try and > figure out what its trying to do, then to figure out if it is actually > doing it .. that will slow things down immensely. > > So as RM I accept the challenge of faster releases, and throw back a > challenge of better patches :) Well, there are some good exemples, and we are getting better, but it is also quite difficult to raise the cursor to the good point and when we do a quick followup, difficult to resist the temptation of using Followup.... And when we are having six months delay on integration of our branch, it is difficult to re process the whole history. It is the work of release manager, QA manager, Bug wrangler, and eventually, all the subscribers to koha-patches to ask for details. But what about branches to come ? quickly sent... my 2 cents. -- Henri-Damien LAURENT > > Chris > > 2010/6/2 Paul Poulain : >> Hi, >> >> In this mail, I want to share with you our situation, our problems, and our >> ideas. >> You'll be probably surprised by this mail. Please consider it's a good news. >> It's not a threat - not my cup of tea (frenchism suspected) - . But I think >> things must be done this way, so I share it with you, and I'm impatiently >> waiting for koha-community reaction & thoughts. >> >> BibLibre situation >> =========== >> We are very successful. Koha is really a success in France. One french >> university is now live, we are on the way to migrate a very large public >> library (500k items), 3 other universities should go live in September, and >> a lot of business coming. >> We are 11, are on our way to hire 3 new ppl, have on board Philippe >> Chabanon, that is very experienced in managing a company, is fully >> enthusiastic about free softwares and community, and knows a lot of people >> in the french library world. >> Is everything perfect ? nope. >> >> Our problems >> ========= >> Organizational problem >> ------------ >> Some of our clients sponsor new developments. Those are done under "march? >> public" rules, that usually define a very strict date (if we fail, then we >> have to pay fines, based on how many days late we are !). That's why we >> started 3 different branches for our 3 different major sponsored works >> (Aix-Marseille, N?mes, Lyon 3, you can see everything on git.biblibre.com). >> Everything that was in Aix-Marseille branch is now in koha 3.2 . But we have >> a lot of things done for Lyon 3 (that is live already) and Nimes (that will >> go live in early july) that are not in koha 3.2 >> >> Our main short-term problem is that koha 3.2 is very (very) late. It has >> been announced one year ago ! We did a lot of testing for Lyon 3 before >> putting them live. Some of them resulted in some bugfixes that could/should >> have been submitted to koha mainstream. But, we couldn't. Thus we face a >> situation where someone fixes in official 3.2 some things we already fixed >> in our Lyon3 branch (shame on us, but really, we couldn't find time. >> Delivery to Lyon3 was on time, but is was shorter than short) >> The situation is the same with Nimes : they will probably be live (with 3.2 >> + Lyon3 + Nimes sponsored stuff) before 3.2 is released -going live planned >> for july 1st, Galen, we challenge you ;-) -. >> >> So, frankly, dealing with our contracts and the official version of Koha has >> been a pain, and has resulted in a "virtual fork" for Lyon3 (and in 1 month >> for Nimes). We strongly want to submit our code to the main trunk, but >> dealing with a so-long-delayed version is really a nightmare (and I >> understand PTFS deploying "Harley" -even if i share community opinion about >> the version being advertised when released in a way it can be understood as >> an official version-). We must find a solution to this situation, or the >> "virtual fork" will result in a "real fork" that we strongly want to avoid ! >> >> As of today, we have something like 400 patches to integrate into koha main >> trunk ! >> >> Technical problem >> ----------- >> With some customers having more than 1 million items (Aix-Marseille : >> 1247686), and with more and more customers, we reach some limits with the >> structure of Koha, in term of performances and stability/maintainability. >> Chris suggested some improvements with koha 3.4, and we have some >> complementary ideas, I want to share with you, and would like to implement >> ASAP. >> >> Ideas/proposal >> ======= >> Technical : Koha architecture >> --------------------------------- >> At the begining of may 2010, we staff had a meeting at Marseille to know if >> it would be possible to make koha easier to maintain and evolve, we are now >> doing some experiment to split koha into several layers that provides >> completely independant set of tasks based on the MVC architecture >> (http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller). >> >> Koha can be seen as a stack with communication between the closest layers. 4 >> layers at all (does it make you remember TCP/IP? ;) ) >> >> VIEW: interactions with different clients ( browser, REST, XMPP, Z3950, ... >> ) >> ?? >> CONTROLLER: Controls the model by transmitting the CRUD tasks requested by >> the VIEW >> ?? >> MODEL: Business objects, as close as possible to the real world >> ?? >> PERSISTENCE: Make the Business objects persistent >> >> Plus, we expect to use external tools as often as possible to take >> experience of other communities and reduce the amount of code we have to >> debug by ourselves. We think tools exists for every layers, actually we just >> have to >> choose between them: >> >> View: >> ------- >> HTML::Template, Template Toolkit 2, Template::Declare, SOAP::Lite, >> any REST, XMPP, Z3950, SRU/SRW wrapper, ... (Infobot ;)) >> >> Controller: >> ---------- >> Dancer, Mojo, The Controller parts of existing MVC frameworks ( >> Jifty, Catalyst, ...), ... >> >> Model: >> ------- >> Any OO Framework ( prefer those which comes with persistence extension as >> MooseX::Storage, KiokuDB, ... choose one ) >> >> Persistence: >> ----------- >> DBIx::Class, Document DB clients, SQL::Abstract, Jifty::DBI, .... >> >> Ideally, the communication between each layers must be specified, provable >> by unit testing, and well documented ( pods in the t/* ?). >> >> Today, Koha works in CGI mode, so there is no persistence at all. And having >> no persistence means we can't implement the MVC model -some tests we did >> with OO frameworks show very poor performances-, and won't be able to >> improve performance heavily -the main loss of perf in koha as of today is >> loading datas that are almost static & should be available immediatly : >> systempreferences & all the admin tables -. >> >> So: 1st step => remove CGI, to have persistent code, enabling persistent DB >> connection. We did some tests with Dancer, that are very encouraring, we >> will send a mail about that soon. >> 2nd step => Once we will be OK with dancer to get rid of CGI => head to >> persistence & OO framework. We vote for Moose & KiokuDB, but we still have >> POC to write to confirm the idea. >> 3rd step =>view part, by moving to T::T (POC still to do to confirm the >> idea) >> >> It may look like a koha rewrite but it's not: every parts would be added >> step by step into koha by wrapping the new design behind the old API during >> a transition phase. An example of current code >> >> sub GetBranchesInCategory($) { >> my ($categorycode) = @_; >> my @branches; >> my $dbh = C4::Context->dbh(); >> my $sth = $dbh->prepare( >> "SELECT b.branchcode >> FROM branchrelations r, branches b >> where r.branchcode=b.branchcode and r.categorycode=?" >> ); >> $sth->execute($categorycode); >> while ( my $branch = $sth->fetchrow ) { >> push @branches, $branch; >> } >> $sth->finish(); >> return ( \@branches ); >> } >> >> would became transition code: >> >> sub GetBranchesInCategory($) { >> my $id = shift >> or return; >> >> # - Category would be the Business object with a member transition (which >> can be >> # deprecated at the end of transition) that encapsulate every >> # persistance specific data used in the C4::* api >> # - the find method could be delegated to the DBIx::Class find that returns >> the >> # record associated to the id >> >> # returns >> if ( my $category = Category->transition->find($id) ) { >> # codes to deal with branches comes de facto with tools as DBIx::Class >> # it returns the list of branches for the category >> $category->branches; >> } >> else { undef } >> } >> >> You can find in attachement a small graph with all our ideas, and the >> beginning of a timeline. >> >> ppl reading carefully will see a "changing indexing system (solr)". 4 years >> after starting with it, we still find zebra being a nightmare, with a lot of >> problems for unicode (with ICU). We did some tests with solr, and they are >> very encouraging. We will provide the results soon too. But i'm really >> enthusiast about what we did in a few hours ! >> >> Organizational : Koha team >> ------------------------------- >> Koha team organisation >> ............... >> If we (BibLibre & the Koha community) want to be able to implement such a >> roadmap (that also includes some new functionnality, as customers sponsors >> some...), we must have a heavily organised team. I think we must : >> * manage release by date (not by features). This is probably the more >> important part for us (see our "march? public" problem). The timing of the >> versions would be defined by the companies contributing the most and synched >> with sponsored devs roadmap. That will be tricky to organize when there are >> more than one company involved, but, for instance, BibLibre is the largest >> contributor ( PTFS: you're welcomed to enter the thread ! ). In fact, "a >> koha 3.4" could be released in a few weeks (look at out master branch, it >> contains 3.2 + Lyon3 + Nimes sponsored work). A "koha 3.6" could be released >> in december/february,... a regular release, that's something that is >> probably "non-negociable" for us. >> * define very clear coding guidelines & be very stricts on their use : >> everything that does not respect the guidelines must be politely but firmly >> rejected. >> * have a very efficient patch workflow : a patch can't be in the wild for >> weeks. The submitter must have an answer in a very short timeline (is 48H >> possible ?). ie : We can't rely on only one person to push patches. We must >> have back-ups for major roles like RM & "patch pusher". >> * we could divide Koha in functionnal (serials / acquisition / cataloguing / >> ...) or technical (perfs / SQL structure / view layer / indexing engine / >> ...) parts, with some working group for each of them, and something like a >> monthly report of the WG (with a weekly internal meeting) >> * write tests ! technical tests (.t) and functional tests (selenium ?) >> >> Our involvement (BibLibre) : >> .................... >> We want to improve our involvement in Koha, by dedicating Henri-Damien to >> work at 75% for Koha and be the link between the work done internally by >> BibLibre, and the work done by everyone else. This mean : >> >> publish our RFPs as soon as possible >> gather all the work done every month and communicate about that on the wiki >> and blog and translate on koha-fr.org >> manage patches coming out from BibLibre, and coming out from everyone else >> (to deal as soon as possible with any problem) >> >> I know we have elected a team recently, and I know chris will do an awesome >> job as RM (I don't know for others, I didn't saw them in their >> responsability yet. For chris, i'm the team for 8 years, I know who he is >> ;-) ), but I think hdl should have a responsability more important than "bug >> wrangler" to reflect the time he (will) dedicate to the project. Is >> something like "co-RM" possible ? any other idea ? Any other idea ? >> >> Next step >> ----------- >> We are *really* willing to implement this roadmap and way of doing things. >> The goal of this mail is to ask everybody involved in Koha development about >> their opinion: we want our proposal to be a chance for Koha (new deal ?). >> >> On the technical side, stay tuned. Some POC will fall in the next weeks !!! >> On the organizational side, the ball is on your side: let's go to >> update/improve our organization ! >> >> -- >> Paul POULAIN >> http://www.biblibre.com >> Expert en Logiciels Libres pour l'info-doc >> Tel : (33) 4 91 81 35 08 >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel From reed at catalyst.net.nz Wed Jun 2 12:22:42 2010 From: reed at catalyst.net.nz (Reed Wade) Date: Wed, 2 Jun 2010 22:22:42 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C062DAA.70607@gmail.com> References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> Message-ID: Any reason not to move to a 4 week release cycle? With alternate ones getting "better" testing. It would mean automating a lot of things -- but doable and worth it in any case. -reed From mdhafen at tech.washk12.org Wed Jun 2 16:23:14 2010 From: mdhafen at tech.washk12.org (Michael Hafen) Date: Wed, 02 Jun 2010 08:23:14 -0600 Subject: [Koha-devel] generic Koha data export format? In-Reply-To: References: Message-ID: <1275488594.21343.4.camel@koha-dev> There are a few possibilities I see here. One is to create an export tool. Some MARC format would be good for the biblio and copy information, but there would have to be some framework, similar to that which is in the works for imports, to map koha fields to MARC fields. The system prefered MARC format would probably be used. Then a seperate button to generate a csv of the borrower information. This could be amended easily to be imported into different version of Koha by adding or removing columns in a spreadsheet application. Finially, another botton to generate a csv/xml/yaml file with issues, reserves, fines, serials. Maybe even split those to seperate buttons/files too. That's just one idea. And one of those features is already in place in the biblio exports. It would just need to be expanded to handle copy information. On Wed, 2010-06-02 at 13:49 +1200, Reed Wade wrote: > Maybe this is already solved but-- > > It seems like it would be fun if there was a nicely generic format > defined for Koha data dumps. Is it correct that right now this is a DB > specific task? > > I'm thinking some format that would be good for-- > > - working across different versions of Koha (within reason) > - portable backups > - a stable interchange format when migrating between Koha and other ILSs > - the case where some library is running v2 and it's really, really > finally time for them to move to v12 and they now have the resources > to do it > > -reed > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel -- Michael Hafen Systems Analyst and Programmer Washington County School District Utah, USA for Koha checkout http://development.washk12.org/gitweb/ or git://development.washk12.org/koha From laurenthdl at alinto.com Wed Jun 2 17:07:56 2010 From: laurenthdl at alinto.com (LAURENT Henri-Damien) Date: Wed, 02 Jun 2010 17:07:56 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> Message-ID: <4C0673CC.8070903@alinto.com> Le 02/06/2010 12:22, Reed Wade a ?crit : > Any reason not to move to a 4 week release cycle? With alternate ones > getting "better" testing. > > It would mean automating a lot of things -- but doable and worth it in any case. ^^The main reason is this one. and no doubt that it IS BOTH doable AND worth.... But in what time and who would do that... BibLibre is willing to devote time for that... But this surely is a community effort with development and should be managed as such. maybe there should be teams along with the assignees of bugs which could make smaller patches, and meetings so as to report progess. -- Henri-Damien LAURENT From mjr at software.coop Wed Jun 2 19:16:16 2010 From: mjr at software.coop (MJ Ray) Date: Wed, 2 Jun 2010 18:16:16 +0100 (BST) Subject: [Koha-devel] generic Koha data export format? In-Reply-To: <20100602093854.GA863@auckland.lan> Message-ID: <20100602171616.42BABF71C1@nail.towers.org.uk> Marc Chantreux > On Wed, Jun 02, 2010 at 12:18:50PM +1000, Rick Welykochy wrote: > > 1. CSV, directly importable into spreadsheet and database programmes > > 2. XML, easily handled by MySQL, more transparent than CSV > > I would choose YAML because > - it can store tree (as xml) without the need of extra tags (required in > xml) > - it's human readable/editable Which is better on the above two points are debatable: YAML doesn't need closing tags (which are often better for clarity, like comments next to /TMPL_IF tags), but I think it has meaningful whitespace which I don't like because humans are poor at seeing whitespace. > - it supports pointed structures Do we need them? > - it's now supported everywhere Not everywhere, but probably comparable with XML. Does MySQL handle it now? I'm inclined towards XML because I know its tools better and my experience is that XML tools are more widely available, although I guess Koha requires both YAML and XML tools to be available. Hope that helps, -- MJ Ray (slef) Webmaster and LMS developer at | software www.software.coop http://mjr.towers.org.uk | .... co IMO only: see http://mjr.towers.org.uk/email.html | .... op From mjr at phonecoop.coop Wed Jun 2 19:38:27 2010 From: mjr at phonecoop.coop (MJ Ray) Date: Wed, 2 Jun 2010 18:38:27 +0100 (BST) Subject: [Koha-devel] bugs.koha-community.org Message-ID: <20100602143214.E2406F71C1@nail.towers.org.uk> Resend due to .org change.... Chris Cormack wrote: > About a week ago, I asked people to start registering at > bugs.koha-community.org in preparation for shifting there. I'm pleased > to say that a lot of you have. > > I'd like to make a deadline for the switch now, Friday 21 May, 9am > NZST (wolfram alpha is a good tool for converting that to your > timezone). > That is when I plan to shift all the bugs over, so if any of you > aren't registered now, can you please do it before then. And from that > time onwards please use bugs.koha-community.org. I'm a bit(!) behind with list emails, so I didn't see this until someone on IRC mentioned it to me. I feel it would have been a good inclusion in the newsletter, or the bugs database contained email addresses of reporters and assignees that probably could have been used to contact them. Anyway, I've registered on bugs.koha-community.org now. How do I get my bugs reattached, please? Thanks, -- MJ Ray (slef) Webmaster and LMS developer at | software www.software.coop http://mjr.towers.org.uk | .... co IMO only: see http://mjr.towers.org.uk/email.html | .... op From gmcharlt at gmail.com Wed Jun 2 19:43:42 2010 From: gmcharlt at gmail.com (Galen Charlton) Date: Wed, 2 Jun 2010 13:43:42 -0400 Subject: [Koha-devel] bugs.koha-community.org In-Reply-To: <20100602143214.E2406F71C1@nail.towers.org.uk> References: <20100602143214.E2406F71C1@nail.towers.org.uk> Message-ID: Hi, On Wed, Jun 2, 2010 at 1:38 PM, MJ Ray wrote: > Anyway, I've registered on bugs.koha-community.org now. ?How do I get > my bugs reattached, please? I will take care of this in the next day or two. Regards, Galen -- Galen Charlton gmcharlt at gmail.com From marc.chantreux at biblibre.com Wed Jun 2 19:59:20 2010 From: marc.chantreux at biblibre.com (Marc Chantreux) Date: Wed, 2 Jun 2010 19:59:20 +0200 Subject: [Koha-devel] generic Koha data export format? In-Reply-To: <20100602171616.42BABF71C1@nail.towers.org.uk> References: <20100602093854.GA863@auckland.lan> <20100602171616.42BABF71C1@nail.towers.org.uk> Message-ID: <20100602175920.GA19688@auckland.lan> On Wed, Jun 02, 2010 at 06:16:16PM +0100, MJ Ray wrote: > Which is better on the above two points are debatable I have experience reading and editing both and i have to admit i hope to never see xml again out of its real place ( spanned text description ). > xml need closing tags (which are often better for clarity, like comments > next to /TMPL_IF tags) I didn't meant end tags but extratags. In yaml, you can write authors: - John Doe - Octave R. Gebel - Agathe dablooz then you have an array. The equivalent XML would be something like John Doe Octave R. Gebel Agathe dablooz element tag is there only because of xml serialization so you need extra config in to make your xmlparser add dans remove tags correctly when reading/writing files. Too error-prone for me. > which I don't like because humans are poor at seeing whitespace. a point for xml ... but - as in xml: that parser will help you - errors are easier to fix by hand. By experience, it's easier to find a hash key without tailing space than tring to understand how an accidentaly removed char in a tag breaks all the tree. > > - it supports pointed structures > Do we need them? It depends if we can extract informations and links without having references to the database ids - it will be hard to do it in some cases. For exemples: authorities are storing DB ids inside the marc record. - it will be the good strategy if we want a tool that is not just another mysql dumper (updating versions, merging 2 bases, ...) > > - it's now supported everywhere > > Not everywhere, but probably comparable with XML. Does MySQL handle > it now? mysql comes with its own grammar (sql) so why must sql have to know something about xml or yaml ? regards -- Marc Chantreux BibLibre, expert en logiciels libres pour l'info-doc http://biblibre.com From reed at catalyst.net.nz Thu Jun 3 01:38:38 2010 From: reed at catalyst.net.nz (Reed Wade) Date: Thu, 3 Jun 2010 11:38:38 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C0673CC.8070903@alinto.com> References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> <4C0673CC.8070903@alinto.com> Message-ID: On Thu, Jun 3, 2010 at 3:07 AM, LAURENT Henri-Damien wrote: > Le 02/06/2010 12:22, Reed Wade a ?crit : >> Any reason not to move to a 4 week release cycle? With alternate ones >> getting "better" testing. >> >> It would mean automating a lot of things -- but doable and worth it in any case. > > ^^The main reason is this one. > and no doubt that it IS BOTH doable AND worth.... But in what time and > who would do that... BibLibre is willing to devote time for that... But > this surely is a community effort with development and should be managed > as such. > maybe there should be teams along with the assignees of bugs which could > make smaller patches, and meetings so as to report progess. (4 weeks is probably too short but...) Yes, getting the science and organisation correct would be key. Something ChrisC has alluded to in emails and IRC is the scheme we have developed at Catalyst for a project he and I and (5-8 others) work on. It is a long running development effort for a high scale news web site. It has a similar architecture and larger code base than Koha. We use git and there's a lot of perl. It seems likely there are some bits of code and techniques that could be applied to Koha release mgmt. We have a 4 (really 1-6) week release cycle which is feature and date driven (customer might need new thing in time for national elections or sporting event and they don't tend to plan too far ahead). We use debian packages for all code deployments and DB updates. We use a system similar to bugzilla for tracking code development. Each feature or bug fix gets a branch named according to the ticket number. We have an automated process which merges all branches for the next release (it queries the issue tracker to know what tickets are in what release) and emails the authors if there's a conflict. This runs several times a day. Instructions embedded in the ticket comments are used to sort dependencies and additional / alternate branches if needed for conflict resolution. We have a similar automated process which performs the merge then creates debian packages for our testing environment. It emails the results to the release manage who reviews and sets the status of updated features so QA staff know what they can now test (this should be automated but isn't just yet because I like to review these still). We have a pre-production autobuild set up which is a duplicate of all that but it only takes branches that QA have signed off on (examines ticket status). This scheme has been extraordinarily satisfying and effective and allows us to spend more time on real work instead of the mechanics of the process. Production deployments used to be long bad days and now they tend to be uneventful. Everyone is happier. Release management is now time spent considering product features instead of the mechanics of integration and testing. The key benefits we've gained are: - at any moment you can play with what would be the release if you decided to release at that moment - developers become responsible for fixing code conflicts or other blocks to their branch getting into a release and without hardly any time needed by the release manager (so very low latency for fixes) to wrangle that because it's automated emails telling the developer something went wrong -- How to apply to Koha? There are differences. We are a small group, mostly in the same office. Our customer is a 5 minute walk down the street from us. I was about to say that our trade-offs and priorities are clear -- but that wasn't true before we had the system described above. It's allowed us and the customer to focus on keeping things clear because they've seen the value of things like having a feature freeze date that doesn't change unless the release date changes. Some key things that could be taken-- - the auto-build and merge and report and package on a test machine for easy QA system -- this is something we should be able to adapt to bugzilla and make available for the project - the notion of naming branches according to bug/feature id -- we've found this to be very powerful Chris or Lars may have something to add. -reed From chris at bigballofwax.co.nz Thu Jun 3 01:55:43 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 3 Jun 2010 11:55:43 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> <4C0673CC.8070903@alinto.com> Message-ID: On 3 June 2010 11:38, Reed Wade wrote: > On Thu, Jun 3, 2010 at 3:07 AM, LAURENT Henri-Damien > wrote: >> Le 02/06/2010 12:22, Reed Wade a ?crit : >>> Any reason not to move to a 4 week release cycle? With alternate ones >>> getting "better" testing. >>> >>> It would mean automating a lot of things -- but doable and worth it in any case. >> >> ^^The main reason is this one. >> and no doubt that it IS BOTH doable AND worth.... But in what time and >> who would do that... BibLibre is willing to devote time for that... But >> this surely is a community effort with development and should be managed >> as such. >> maybe there should be teams along with the assignees of bugs which could >> make smaller patches, and meetings so as to report progess. > > > > (4 weeks is probably too short but...) > > Yes, getting the science and organisation correct would be key. > > Something ChrisC has alluded to in emails and IRC is the scheme we > have developed at Catalyst for a project he and I and (5-8 others) > work on. It is a long running development effort for a high scale news > web site. It has a similar architecture and larger code base than > Koha. We use git and there's a lot of perl. It seems likely there are > some bits of code and techniques that could be applied to Koha release > mgmt. > > We have a 4 (really 1-6) week release cycle which is feature and date > driven (customer might need new thing in time for national elections > or sporting event and they don't tend to plan too far ahead). > > We use debian packages for all code deployments and DB updates. > > We use a system similar to bugzilla for tracking code development. > Each feature or bug fix gets a branch named according to the ticket > number. > > We have an automated process which merges all branches for the next > release (it queries the issue tracker to know what tickets are in what > release) and emails the authors if there's a conflict. This runs > several times a day. Instructions embedded in the ticket comments are > used to sort dependencies and additional / alternate branches if > needed for conflict resolution. > > We have a similar automated process which performs the merge then > creates debian packages for our testing environment. It emails the > results to the release manage who reviews and sets the status of > updated features so QA staff know what they can now test (this should > be automated but isn't just yet because I like to review these still). > > We have a pre-production autobuild set up which is a duplicate of all > that but it only takes branches that QA have signed off on (examines > ticket status). > > This scheme has been extraordinarily satisfying and effective and > allows us to spend more time on real work instead of the mechanics of > the process. Production deployments used to be long bad days and now > they tend to be uneventful. Everyone is happier. Release management is > now time spent considering product features instead of the mechanics > of integration and testing. > > The key benefits we've gained are: > ?- at any moment you can play with what would be the release if you > decided to release at that moment > ?- developers become responsible for fixing code conflicts or other > blocks to their branch getting into a release and without hardly any > time needed by the release manager (so very low latency for fixes) to > wrangle that because it's automated emails telling the developer > something went wrong > > -- > > How to apply to Koha? > > There are differences. We are a small group, mostly in the same > office. Our customer is a 5 minute walk down the street from us. > > I was about to say that our trade-offs and priorities are clear -- but > that wasn't true before we had the system described above. It's > allowed us and the customer to focus on keeping things clear because > they've seen the value of things like having a feature freeze date > that doesn't change unless the release date changes. > > Some key things that could be taken-- > > ?- the auto-build and merge and report and package on a test machine > for easy QA system > ?-- this is something we should be able to adapt to bugzilla and make > available for the project > > ?- the notion of naming branches according to bug/feature id -- we've > found this to be very powerful > > Chris or Lars may have something to add. > I think the most important part of our system is indeed the separate branches per feature/bug This allows features to be removed and the release still happen, if they fail testing. Time based releases simply can not happen if we cant easily remove failing code. (Or not merge it as the case may be). Chris From nengard at gmail.com Thu Jun 3 02:30:24 2010 From: nengard at gmail.com (Nicole Engard) Date: Wed, 2 Jun 2010 20:30:24 -0400 Subject: [Koha-devel] dontmerge system preference In-Reply-To: References: <4C056437.7000209@gmail.com> Message-ID: See : http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3178 this patch was never complete which explains why I don't have the new preference. On Tue, Jun 1, 2010 at 3:55 PM, Nicole Engard wrote: > On Tue, Jun 1, 2010 at 3:49 PM, LAURENT Henri-Damien > wrote: > >> >> Hi Nicole, >> as far as that system preference is concerned, and in accordance to the >> Coding Guidelines which ask that NO systempreference should have a >> negative noun or effect, this preference is now replaced by >> MergeAuthoritiesOnUpdate which, if set edits all the related biblios and >> reports the heading into them. >> Hopes that helps. > > This preference is not in HEAD as of yet - when it is there will > dontmerge be removed? If so then it helps because we won't have that > poorly worded preference summary anymore :) > > Nicole > From lars at catalyst.net.nz Thu Jun 3 04:22:25 2010 From: lars at catalyst.net.nz (Lars Wirzenius) Date: Thu, 03 Jun 2010 14:22:25 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> <4C0673CC.8070903@alinto.com> Message-ID: <1275531745.2756.113.camel@saimaa.wgtn.cat-it.co.nz> On to, 2010-06-03 at 11:38 +1200, Reed Wade wrote: > Chris or Lars may have something to add. This whole issue of how to structure development processes for Koha is pretty big, and I think it might be good to avoid changing too much too fast. For Koha 3.4, off the top of my head, I would suggest: * automatically measure test coverage for the test suite * reject patches that decrease test coverage * reject patches early for obviously low quality - bad commit messages - too many changes in one patch - automatic test suite does not pass - other things that make it hard for RM to review * keep the master branch always ready to release, as far as possible * possibly discuss changes on koha-devel before they are accepted - might apply only to non-trivial things, in the RM's opinion - discussion might help to find the best way of implementing feature * set up a test machine running the current tip of master - Debian packages would make this easy to update - available to everyone (via the web interface) - easy way to see if things work or not, and if the current version exhibits a bug or not That might all already be too much change, so implementing changes more slowly might be a good idea. From marc.chantreux at biblibre.com Thu Jun 3 09:19:01 2010 From: marc.chantreux at biblibre.com (Marc Chantreux) Date: Thu, 3 Jun 2010 09:19:01 +0200 Subject: [Koha-devel] Make koha dance! Message-ID: <20100603071901.GC15039@auckland.lan> hello koha people, As Paul said in his last message, we try to know if it's possible to introduce a clear layer structure in koha (actually: we think it's possible). On Wed, Jun 02, 2010 at 09:59:24AM +0200, Paul Poulain wrote: > Koha can be seen as a stack with communication between the closest layers. 4 > layers at all (does it make you remember TCP/IP? ;) ) > > VIEW: interactions with different clients ( browser, REST, XMPP, Z3950, ... ) > ?? > CONTROLLER: Controls the model by transmitting the CRUD tasks requested by the > VIEW > ?? > MODEL: Business objects, as close as possible to the real world > ?? > PERSISTENCE: Make the Business objects persistent the most important layer is the MODEL and we would like to use very modern perl OO paradigms (as MooseX::Declare) to clean it. The problem there is that Moose is a bottleneck at start time so CGI scheme must be removed first. It leads us to the CONTROLLER part which is the part where we can introduce an MVC framework. So we tried to make koha run on top of Dancer ... this is my report on the WIP, we hope you'll enjoy: http://www.tinybox.net/2010/06/03/dance-koha-dance/ Regards -- Marc Chantreux BibLibre, expert en logiciels libres pour l'info-doc http://biblibre.com From cmurdock at ccfls.org Fri Jun 4 16:42:51 2010 From: cmurdock at ccfls.org (Cindy Murdock Ames) Date: Fri, 04 Jun 2010 10:42:51 -0400 Subject: [Koha-devel] memcached? Message-ID: <4C0910EB.7070104@ccfls.org> Hi all, The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: our transactions are *really* slow at times. I suspect the culprit is probably a ton of simultaneous mysql activity. I understand Chris has been working on adding memcached support to Koha, and I'm hoping that will help. I tried enabling it yesterday, but it didn't seem to do anything, so I must have been missing something, or maybe it's not implemented quite yet. Can anyone tell me what we need to do to get it going? Our system: 3.2 alpha2 (with zebra) on debian lenny (64bit), 16gb ram, two dual core opteron 2214 processors. We have nine libraries throughout the county with about 15 circ stations simultaneously circulating items at the busiest times. We have about 29,500 borrowers and 309,000 items. We just upgraded from a far less powerful dev_week system which didn't have these issues. What I've done: installed the debian packages for memcached and libcache-memcached-perl installed Cache::Memcached::Fast from CPAN (had to force it to install) Added these lines to the config section of koha-conf.xml 127.0.0.1:11211 KOHA I've also tried mod_perl, which does seem to improve page load time (from 0.8 sec to 0.1 sec on the login page, for example) but I've disabled it again for now because it seemed to cause some errors. Obviously I need to work on that some more. Thanks in advance!! Cindy -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS http://meadvillelibrary.org | http://ccfls.org From colin.campbell at ptfs-europe.com Fri Jun 4 17:35:51 2010 From: colin.campbell at ptfs-europe.com (Colin Campbell) Date: Fri, 04 Jun 2010 16:35:51 +0100 Subject: [Koha-devel] memcached? In-Reply-To: <4C0910EB.7070104@ccfls.org> References: <4C0910EB.7070104@ccfls.org> Message-ID: <4C091D57.2020702@ptfs-europe.com> On 04/06/10 15:42, Cindy Murdock Ames wrote: > Hi all, > > The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: > our transactions are *really* slow at times. I suspect the culprit is > probably a ton of simultaneous mysql activity. Have you done any tuning of mysql? Out of the box it probably is using less memory than it needs for efficient running. A lot of the my.cnf parameters can be tuned to improve throughput. Memcache can work wonders but it needs to be caching the data thats causing you to thrash the db. I'm not sure we've reached that point yet. Cheers Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell at ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com From dpavlin at rot13.org Fri Jun 4 18:40:33 2010 From: dpavlin at rot13.org (Dobrica Pavlinusic) Date: Fri, 4 Jun 2010 18:40:33 +0200 Subject: [Koha-devel] memcached? In-Reply-To: <4C091D57.2020702@ptfs-europe.com> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> Message-ID: <20100604164033.GB599@rot13.org> On Fri, Jun 04, 2010 at 04:35:51PM +0100, Colin Campbell wrote: > On 04/06/10 15:42, Cindy Murdock Ames wrote: > >Hi all, > > > >The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: > >our transactions are *really* slow at times. I suspect the culprit is > >probably a ton of simultaneous mysql activity. > Have you done any tuning of mysql? Out of the box it probably is > using less memory than it needs for efficient running. A lot of the > my.cnf parameters can be tuned to improve throughput. > Memcache can work wonders but it needs to be caching the data thats > causing you to thrash the db. I'm not sure we've reached that point > yet. I documented my journy over mysql tuning for Koha at: http://blog.rot13.org/2010/04/mysql_is_slow_did_you_tune_your_settings.html Just running mysqltuner.pl or tuning-primer.sh will probably be enough :-) I would highly recommend putting session files at filesystem as opposed to database, especially if you have semi-high load on your website (web crawlers and what not) because it cut down database load by 75% for us. Testing current memcache in Koha, it seemed to be used only for localization caching, and it was a bit slower on our system after some basic profiling. YMMV. -- Dobrica Pavlinusic 2share!2flame dpavlin at rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin From cmurdock at ccfls.org Fri Jun 4 18:51:48 2010 From: cmurdock at ccfls.org (Cindy Murdock Ames) Date: Fri, 04 Jun 2010 12:51:48 -0400 Subject: [Koha-devel] memcached? In-Reply-To: <20100604164033.GB599@rot13.org> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: <4C092F24.6010600@ccfls.org> Thanks Dobrica. I've seen your post before when looking for a solution to those innodb problems. I'll give mysqltuner a shot this weekend while we're closed, as well as the session files. Cheers, c. Dobrica Pavlinusic wrote: > http://blog.rot13.org/2010/04/mysql_is_slow_did_you_tune_your_settings.html > > Just running mysqltuner.pl or tuning-primer.sh will probably be enough > :-) > > I would highly recommend putting session files at filesystem as opposed > to database, especially if you have semi-high load on your website (web > crawlers and what not) because it cut down database load by 75% for us. > > Testing current memcache in Koha, it seemed to be used only for > localization caching, and it was a bit slower on our system after some > basic profiling. YMMV. > > > I documented my journy over mysql tuning for Koha at: -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS http://meadvillelibrary.org | http://ccfls.org From laurenthdl at alinto.com Fri Jun 4 18:58:25 2010 From: laurenthdl at alinto.com (LAURENT Henri-Damien) Date: Fri, 04 Jun 2010 18:58:25 +0200 Subject: [Koha-devel] memcached? In-Reply-To: <20100604164033.GB599@rot13.org> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: <4C0930B1.3010909@alinto.com> Le 04/06/2010 18:40, Dobrica Pavlinusic a ?crit : > On Fri, Jun 04, 2010 at 04:35:51PM +0100, Colin Campbell wrote: >> On 04/06/10 15:42, Cindy Murdock Ames wrote: >>> Hi all, >>> >>> The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: >>> our transactions are *really* slow at times. I suspect the culprit is >>> probably a ton of simultaneous mysql activity. >> Have you done any tuning of mysql? Out of the box it probably is >> using less memory than it needs for efficient running. A lot of the >> my.cnf parameters can be tuned to improve throughput. >> Memcache can work wonders but it needs to be caching the data thats >> causing you to thrash the db. I'm not sure we've reached that point >> yet. > > I documented my journy over mysql tuning for Koha at: > > http://blog.rot13.org/2010/04/mysql_is_slow_did_you_tune_your_settings.html > > Just running mysqltuner.pl or tuning-primer.sh will probably be enough > :-) > > I would highly recommend putting session files at filesystem as opposed > to database, especially if you have semi-high load on your website (web > crawlers and what not) because it cut down database load by 75% for us. > > Testing current memcache in Koha, it seemed to be used only for > localization caching, and it was a bit slower on our system after some > basic profiling. YMMV. > memcached is not really meant for speed but for scaling. Speed improvement is a +. So that compared to db access, it is not really faster. But if you have many access and network slowness then it is really interesting. Chris Cormack also did some POCs on Caching whole pages. And not only function results. And results were quite encouraging. But that never hit code base for 3.2. What is really strange is the way Context is evaluated multiple times. And we come to thousands of SQL queries for a simple page. -- Henri-Damien LAURENT BibLibre From tajoli at cilea.it Fri Jun 4 19:01:18 2010 From: tajoli at cilea.it (Zeno Tajoli) Date: Fri, 04 Jun 2010 19:01:18 +0200 Subject: [Koha-devel] memcached? In-Reply-To: <20100604164033.GB599@rot13.org> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: <20100604170119.31715473A9@eliot.biblibre.com> Hi Dobrica, >I would highly recommend putting session files at filesystem as opposed >to database, especially if you have semi-high load on your website (web >crawlers and what not) because it cut down database load by 75% for us. do you speak about preference 'SessionStorage' ? To insert 'tmp' instead of 'mysql' ? Bye Zeno Tajoli Zeno Tajoli CILEA - Segrate (MI) tajoliAT_SPAM_no_prendiATcilea.it (Indirizzo mascherato anti-spam; sostituisci quanto tra AT con @) From dpavlin at rot13.org Sat Jun 5 12:13:27 2010 From: dpavlin at rot13.org (Dobrica Pavlinusic) Date: Sat, 5 Jun 2010 12:13:27 +0200 Subject: [Koha-devel] memcached? In-Reply-To: <20100604170119.31715473A9@eliot.biblibre.com> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> <20100604170119.31715473A9@eliot.biblibre.com> Message-ID: <20100605101327.GA3346@rot13.org> On Fri, Jun 04, 2010 at 07:01:18PM +0200, Zeno Tajoli wrote: > Hi Dobrica, > > >I would highly recommend putting session files at filesystem as opposed > >to database, especially if you have semi-high load on your website (web > >crawlers and what not) because it cut down database load by 75% for us. > > do you speak about preference 'SessionStorage' ? > To insert 'tmp' instead of 'mysql' ? Yes, sorry for vague description. I would also suggest to change C4::Auth around like 1344 to something like: session = new CGI::Session("driver:File;serializer:yaml;id:md5", $sessionID, {Directory=>'/dev/shm'}); In this line, /tmp was changed to /dev/shm which is ram-based filesystem on linux, making session serialization free in term of disk i/o. -- Dobrica Pavlinusic 2share!2flame dpavlin at rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin From cfouts at ptfs.com Fri Jun 4 19:28:34 2010 From: cfouts at ptfs.com (Fouts, Clay) Date: Fri, 4 Jun 2010 10:28:34 -0700 Subject: [Koha-devel] memcached? In-Reply-To: <20100604164033.GB599@rot13.org> References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: I can also attest to the benefit of storing session data somewhere other than in the MySQL database. LLEK has the ability to store its session data through memcached instead of writing to a file (which can create problems in a multi-server configuration) or MySQL. This frees up the database more than one might expect. The constant stream of tiny, non-contiguous INSERTs and UPDATEs contributes heavily to fragmentation of InnoDB writes which in turn lengthens transaction times. (Not to mention Koha's inability to selectively cull stale sessions, causing your table to have millions of rows over time). The drawback is of course that if memcached goes away you lose all your current session data since it's not a permanent storage medium, but this is not a significant loss in most circumstances. The MySQL "slow query" log is also very valuable for tracking down queries that hog DB time. However, the single most effective tool I've seen for finding performance bottlenecks is the NYTProf execution profiler. It can be a bit cumbersome to use, but the results give crystal clear insight into which functions are taking up the most time (though it doesn't necessarily give information about why they take up so much time). I have been surprised at how often MySQL is *not* to blame when it comes to Koha's performance. Clay On Fri, Jun 4, 2010 at 9:40 AM, Dobrica Pavlinusic wrote: > On Fri, Jun 04, 2010 at 04:35:51PM +0100, Colin Campbell wrote: > > On 04/06/10 15:42, Cindy Murdock Ames wrote: > > >Hi all, > > > > > >The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: > > >our transactions are *really* slow at times. I suspect the culprit is > > >probably a ton of simultaneous mysql activity. > > Have you done any tuning of mysql? Out of the box it probably is > > using less memory than it needs for efficient running. A lot of the > > my.cnf parameters can be tuned to improve throughput. > > Memcache can work wonders but it needs to be caching the data thats > > causing you to thrash the db. I'm not sure we've reached that point > > yet. > > I documented my journy over mysql tuning for Koha at: > > http://blog.rot13.org/2010/04/mysql_is_slow_did_you_tune_your_settings.html > > Just running mysqltuner.pl or tuning-primer.sh will probably be enough > :-) > > I would highly recommend putting session files at filesystem as opposed > to database, especially if you have semi-high load on your website (web > crawlers and what not) because it cut down database load by 75% for us. > > Testing current memcache in Koha, it seemed to be used only for > localization caching, and it was a bit slower on our system after some > basic profiling. YMMV. > > -- > Dobrica Pavlinusic 2share!2flame > dpavlin at rot13.org > Unix addict. Internet consultant. > http://www.rot13.org/~dpavlin > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mason at kohaaloha.com Sat Jun 5 01:51:16 2010 From: mason at kohaaloha.com (JAMES Mason) Date: Sat, 5 Jun 2010 11:51:16 +1200 Subject: [Koha-devel] memcached - koha tuning In-Reply-To: <4C0910EB.7070104@ccfls.org> References: <4C0910EB.7070104@ccfls.org> Message-ID: <34189D00-3A5F-444B-A596-6E5E25B1BC0F@KohaAloha.com> On 2010-06-5, at 2:42 AM, Cindy Murdock Ames wrote: > Hi all, > > The good news: CCFLS went live with 3.2alpha2 on Tuesday! The bad news: our transactions are *really* slow at times. Heya Cindy, Think about moving your zebra and mysql DB's to different physical disks, any decreasing your zebra-rebuild frequency. This one helped us a lot! Does your box get slow only when a zeb-rebuild is happening? Do some mysql/inno tuning to take advantage of most of your delicious RAM ;) innodb_buffer_pool_size = 10000M install munin/munin-node on you box to get some decent stats on what specifically is causing the slow-down. http://packages.debian.org/lenny/munin these additional munin plugins are great for mysql-tuning, i run them on my prod systems http://wiki.github.com/kjellm/munin-mysql/ set good cacheing rules in your apache.conf for css/js/images ------------------------------------------------------------ Header set Pragma "no-cache" Header set Cache-Control "no-cache, must-revalidate, post-check=0, pre-check=0" Header set Expires 1 ExpiresDefault "access plus 7 days" Header set Cache-Control "public" ------------------------------------------------------------ force all *non* circ-station koha-requests thru a squid-box, to prioritize checkin/checkout requests on circ-stations Cheers, Mason. -- KohaAloha, NZ. From cmurdock at ccfls.org Sat Jun 5 23:36:24 2010 From: cmurdock at ccfls.org (Cindy Murdock Ames) Date: Sat, 05 Jun 2010 17:36:24 -0400 Subject: [Koha-devel] memcached - koha tuning In-Reply-To: <34189D00-3A5F-444B-A596-6E5E25B1BC0F@KohaAloha.com> References: <4C0910EB.7070104@ccfls.org> <34189D00-3A5F-444B-A596-6E5E25B1BC0F@KohaAloha.com> Message-ID: <4C0AC358.9040904@ccfls.org> Thanks everyone for the pointers on tuning Koha. I'm working on implementing many of them right now. I'll let you know how things work out. I'm thinking this would be a good topic for the wiki. ;) Cheers! Cindy -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS http://meadvillelibrary.org | http://ccfls.org From chris at bigballofwax.co.nz Sun Jun 6 03:38:12 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Sun, 6 Jun 2010 13:38:12 +1200 Subject: [Koha-devel] memcached? In-Reply-To: References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: 2010/6/5 Fouts, Clay : > I can also attest to the benefit of storing session data somewhere other > than in the MySQL database.?LLEK has the ability to store its session data > through memcached instead of writing to a file (which can create problems in > a multi-server configuration) or MySQL. This frees up the database more than > one might expect. The constant stream of tiny, non-contiguous INSERTs and > UPDATEs contributes heavily to fragmentation of InnoDB writes which in turn > lengthens transaction times. (Not to mention Koha's inability to selectively > cull stale sessions, causing your table to have millions of rows over time). > The drawback is of course that if memcached goes away you lose all your > current session data since it's not a permanent storage medium, but this is > not a significant loss in most circumstances. Storing as temp files on a ram based disk as has already been suggested works much the same way too. Of course a patch for using memcached for session storage would be gratefully accepted and would save on once again doubling up on work already done. We use memcached to a huge extent at work, including storing session data, and have yet to have any real problems occur. > The MySQL "slow query" log is also very valuable for tracking down queries > that hog DB time. However, the single most effective tool I've seen for > finding performance bottlenecks is the NYTProf execution profiler. It can be > a bit cumbersome to use, but the results give crystal clear insight into > which functions are taking up the most time (though it doesn't necessarily > give information about why they take up so much time). I have been surprised > at how often MySQL is *not* to blame when it comes to Koha's performance. > Clay > I concur, MySQL is usually not the bottleneck, unless the site is under extreme load, the startup cost of perl is a major factor, as Mason has suggested mysql, zebra and in fact apache2 can fight each other for I/O and CPU. Different partitions/disks for mysql and zebra can provide an easy win. One thing that the Profiler won't tell you, is how fast your browser can render the pages, Koha 3.2 (and 3.0) before it have a lot more css and js on the intranet that previous versions. Different browsers can render the page quite a bit faster, also the expires headers suggested by Mason will stop the browser refetching things that it doesn' t need too. We found at HLT with the new site, that using Chrome instead of Firefox on the circ desk provided a 2-3 second speed up in page render. Something worth trying. Chris From ohiocore at gmail.com Sun Jun 6 07:45:55 2010 From: ohiocore at gmail.com (Joe Atzberger) Date: Sun, 6 Jun 2010 01:45:55 -0400 Subject: [Koha-devel] memcached? In-Reply-To: References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: NYTprof combined with firebug (or chrome dev tools) can give you the comprehensive picture. The former atomizes and itemizes the server side costs. The latter details the client side, including graphs of time spent downloading vs. rendering, per file. On Jun 5, 2010 9:38 PM, "Chris Cormack" wrote: 2010/6/5 Fouts, Clay : > I can also attest to the benefit of storing session data somewhere other > than in the MySQL datab... Storing as temp files on a ram based disk as has already been suggested works much the same way too. Of course a patch for using memcached for session storage would be gratefully accepted and would save on once again doubling up on work already done. We use memcached to a huge extent at work, including storing session data, and have yet to have any real problems occur. > The MySQL "slow query" log is also very valuable for tracking down queries > that hog DB time. Ho... I concur, MySQL is usually not the bottleneck, unless the site is under extreme load, the startup cost of perl is a major factor, as Mason has suggested mysql, zebra and in fact apache2 can fight each other for I/O and CPU. Different partitions/disks for mysql and zebra can provide an easy win. One thing that the Profiler won't tell you, is how fast your browser can render the pages, Koha 3.2 (and 3.0) before it have a lot more css and js on the intranet that previous versions. Different browsers can render the page quite a bit faster, also the expires headers suggested by Mason will stop the browser refetching things that it doesn' t need too. We found at HLT with the new site, that using Chrome instead of Firefox on the circ desk provided a 2-3 second speed up in page render. Something worth trying. Chris _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-commun... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmurdock at ccfls.org Mon Jun 7 19:16:16 2010 From: cmurdock at ccfls.org (Cindy Murdock Ames) Date: Mon, 07 Jun 2010 13:16:16 -0400 Subject: [Koha-devel] memcached - koha tuning In-Reply-To: <4C0AC358.9040904@ccfls.org> References: <4C0910EB.7070104@ccfls.org> <34189D00-3A5F-444B-A596-6E5E25B1BC0F@KohaAloha.com> <4C0AC358.9040904@ccfls.org> Message-ID: <4C0D2960.3010702@ccfls.org> Hi all, It looks like we've resolved our transaction speed problem--knock on wood! Many, many, many thanks to everyone who provided suggestions for improvement. Here's what Kyle and I did--I'm not sure what exactly helped, if it was one particular thing or a combo of many. * Mysql config: Increased innodb buffer pool size to 10000M (was 1024M); followed mysqltuner recommendations to set join_buffer_size (32M), query_cache_limit (16M), query_cache_size (16M), tmp_table_size (8M) and max_heap_table_size (8m). Mysqltuner still suggests more for those values, but the speed issue seems to be resolved so I'm not going to worry about it for now. The max amount of system memory it can currently use is 85%. * Moved the zebra db to another drive * Optimized mysql tables; mysqltuner reported that one table was fragmented. * Applied Mason's apache2 caching config recommendations * Storing session files on a ramdisk (/dev/shm) (Dobrica's suggestion) * mod_perl --Kyle got it to work by adding the following to httpd.conf: SetHandler perl-script PerlResponseHandler ModPerl::PerlRunPrefork PerlOptions +ParseHeaders PerlSendHeader On Options +ExecCGI PerlWarn On PerlRequire /home/koha/startup.pl Cheers! Cindy Cindy Murdock Ames wrote: > Thanks everyone for the pointers on tuning Koha. I'm working on > implementing many of them right now. I'll let you know how things > work out. I'm thinking this would be a good topic for the wiki. ;) > > Cheers! > Cindy > -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS http://meadvillelibrary.org | http://ccfls.org From nengard at gmail.com Tue Jun 8 00:53:21 2010 From: nengard at gmail.com (Nicole Engard) Date: Mon, 7 Jun 2010 18:53:21 -0400 Subject: [Koha-devel] Call for Koha Newsletter Articles In-Reply-To: References: Message-ID: Just a little nudge to remind you all that I am still looking for articles for this month's newsletter. Thanks Nicole On Tue, Jun 1, 2010 at 12:39 PM, Nicole Engard wrote: > Hello all, > > It's that time again, time to gather up all your Koha news (big and > small) and send ?them to me for publication in our next issue of the > Koha Newsletter. ?Articles can be as short as 1 line and as long as 20 > (with links to read more content). ?They should include news about our > Koha library, features you're working on, tips & tutorials you want to > share. ?In short the sky's the limit. ?Please email me your articles > by the 12th of June so that I can edit everything and get it organized > for the June 15th release. > > Past newsletters can be found here: > http://koha-community.org/category/koha-newsletter/ and you can > subscribe to updates via RSS: > http://feeds.feedburner.com/KohaNewsletter > > Thanks in advance, > Nicole C. Engard > Koha Documentation Manager/Newsletter Editor > From kmkale at anantcorp.com Tue Jun 8 07:24:57 2010 From: kmkale at anantcorp.com (Koustubha Kale) Date: Tue, 8 Jun 2010 10:54:57 +0530 Subject: [Koha-devel] Granthalaya.org announcement of Koha hosting service Message-ID: Friends, Granthalaya.org ( www.granthalaya.org ) and Anant Corporation ( www.anantcorp.com ) are very pleased to announce general availability of Koha hosting on granthalaya.org server located in VPM, Thane ( www.vpmthane.org ) datacenter. Our first user of this service is already live, please see opac.kalibindia.com This initiative is to provide a revenue stream for sustained operation and maintenance of the Granthalaya.org project ( which is a project of Vidya Prasarak Mandal, Thane; an educational trust.) which intends to create bibliographic union database of holdings of all the public libraries in Konkan area including Thane in India. Granthalaya.org also intends to help libraries in computerizing their holdings by providing them hosted / local installation of Koha, user training and data migration services. Key features of this offering are.. * Independent install of Koha version of your choice on a shared Ubuntu server. * Full access to your home directory and Koha code. * Separate mysql database. Full access via phpmyadmin. * Separate Zebra database * Installation, Initial setup and NBD Annual Support included. * Can offer customization, data migration and user training services if required. * Nightly full dump of database in your home directory with backup to disk and tape. * We are open to discuss any additional requirements Initially we are offering this service at USD 250 for initial setup (one time charge) + USD 350 as Annual Recurring fees. Please contact me if you would like to avail this offer or to help the Granthalaya.org project in any manner. Regards, Koustubha Kale Anant Corporation Contact Details : Address : 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), Maharashtra, India, Pin : 400601. TeleFax : +91-22-21720108, +91-22-21720109 Mobile : +919820715876 Website : http://www.anantcorp.com Blog : http://www.anantcorp.com/blog/?author=2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.bernier at free.fr Wed Jun 9 10:05:30 2010 From: alex.bernier at free.fr (Alex Bernier) Date: Wed, 9 Jun 2010 10:05:30 +0200 Subject: [Koha-devel] Koha 3.0 and mod_perl Message-ID: <20100609080530.GA5577@Pinson> Hello, I am planning to run Koha 3.0 with the mod_perl Apache 2 module (using the PerlRunPrefork response handler). My question : has someone noticed problems with such a configuration ? (On wiki.koha.org , there was a page about Koha and mod_perl but I don't find it anymore on the Wiki of koha-community.org.) Best regards, Alex From chrisc at catalyst.net.nz Wed Jun 9 10:18:56 2010 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Wed, 9 Jun 2010 20:18:56 +1200 Subject: [Koha-devel] Koha 3.0 and mod_perl In-Reply-To: <20100609080530.GA5577@Pinson> References: <20100609080530.GA5577@Pinson> Message-ID: <20100609081856.GI380@rorohiko> * Alex Bernier (alex.bernier at free.fr) wrote: > Hello, > > I am planning to run Koha 3.0 with the mod_perl Apache 2 module (using the PerlRunPrefork response handler). My question : has someone noticed problems with such a configuration ? > > (On wiki.koha.org , there was a page about Koha and mod_perl but I don't find it anymore on the Wiki of koha-community.org.) As far as I know, no one has tested it in a serious way, so its yet to be seen if it has problems. I wouldn't be suprised if there were some problems with the librarian interface. 3.4 is going to be a performance based release, one of the goals is to make sure the code is mod_perl safe. (As well as potentially other persistant code mechanisms). Chris > > Best regards, > > Alex > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand From alex.bernier at free.fr Wed Jun 9 10:55:40 2010 From: alex.bernier at free.fr (Alex Bernier) Date: Wed, 9 Jun 2010 10:55:40 +0200 Subject: [Koha-devel] Koha 3.0 and mod_perl In-Reply-To: <20100609081856.GI380@rorohiko> References: <20100609080530.GA5577@Pinson> <20100609081856.GI380@rorohiko> Message-ID: <20100609085540.GA6528@Pinson> On Wed, Jun 09, 2010 at 08:18:56PM +1200, Chris Cormack wrote: > * Alex Bernier (alex.bernier at free.fr) wrote: > > Hello, > > > > I am planning to run Koha 3.0 with the mod_perl Apache 2 module (using the PerlRunPrefork response handler). My question : has someone noticed problems with such a configuration ? > > > > (On wiki.koha.org , there was a page about Koha and mod_perl but I don't find it anymore on the Wiki of koha-community.org.) > As far as I know, no one has tested it in a serious way, so its yet to > be seen if it has problems. > > I wouldn't be suprised if there were some problems with the librarian > interface. Even with PerlRunPrefork response handler ? I don't find documentation regarding differences between PerlRunPrefork and Registry handlers, but it seems to me that PerlRunPrefork is more "flexible" than Registry. Regards, Alex From chrisc at catalyst.net.nz Wed Jun 9 11:03:04 2010 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Wed, 9 Jun 2010 21:03:04 +1200 Subject: [Koha-devel] Koha 3.0 and mod_perl In-Reply-To: <20100609085540.GA6528@Pinson> References: <20100609080530.GA5577@Pinson> <20100609081856.GI380@rorohiko> <20100609085540.GA6528@Pinson> Message-ID: <20100609090304.GJ380@rorohiko> * Alex Bernier (alex.bernier at free.fr) wrote: > On Wed, Jun 09, 2010 at 08:18:56PM +1200, Chris Cormack wrote: > > * Alex Bernier (alex.bernier at free.fr) wrote: > > > Hello, > > > > > > I am planning to run Koha 3.0 with the mod_perl Apache 2 module (using the PerlRunPrefork response handler). My question : has someone noticed problems with such a configuration ? > > > > > > (On wiki.koha.org , there was a page about Koha and mod_perl but I don't find it anymore on the Wiki of koha-community.org.) > > As far as I know, no one has tested it in a serious way, so its yet to > > be seen if it has problems. > > > > I wouldn't be suprised if there were some problems with the librarian > > interface. > > Even with PerlRunPrefork response handler ? I don't find documentation regarding differences between PerlRunPrefork and Registry handlers, but it seems to me that PerlRunPrefork is more "flexible" than Registry. > Yep, it is certainly more flexible, but there is quite a few instances in Koha where we don't clean up as well as we should. With CGI it doesn't matter so match, but with persistance my fear would be ram leakage. So i'd want to keep an eye on the apache child processes and make sure they don't grow out of control. Chris -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand From charl at prograbiz.com Thu Jun 10 01:10:10 2010 From: charl at prograbiz.com (charl at prograbiz.com) Date: Thu, 10 Jun 2010 01:10:10 +0200 (SAST) Subject: [Koha-devel] Wide character in null operation Message-ID: <61706.99.241.31.141.1276125010.squirrel@webmail.lantic.net> Can anyone please advise what may cause this problem? I get underneath error when I try to save the following in tag 100 a: "Bstan-ʼdzin-rgya-mtsho," Koha error The following fatal error has occurred: Wide character in null operation at /usr/local/share/perl/5.10.0/MARC/Charset/Table.pm line 96. Apache Server version: Apache/2.2.12 (Ubuntu) Server built: Mar 9 2010 21:20:44 Koha 3.00.06.010 Koha DB 3.0006010 MySQL mysql Ver 14.14 Distrib 5.1.37, for debian-linux-gnu (i486) using EditLine wrapper OS Linux wax.sabinet.co.za 2.6.31-22-generic #60-Ubuntu SMP Thu May 27 00:22:23 UTC 2010 i686 Perl 5.010000 From M.de.Rooy at rijksmuseum.nl Thu Jun 10 12:30:30 2010 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Thu, 10 Jun 2010 10:30:30 +0000 Subject: [Koha-devel] New preferences editor Message-ID: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> Hi all, I was just testing a small patch for 3.2 with a new preference not listed in the pref templates. Although the pref exists, you cannot find it or add it. (The manual does not yet explain how to add preferences too, or I could not find it.) I think it should be interesting to include (possibly older) local preferences that are not yet in the templates on a separate tab or so. The same holds for searching the prefs; the new version does not find preferences that are not listed in the pref files, but are in the database and can be used via C4::Context. This is somewhat confusing (to me)? Would it be better to allow finding them at least? Any thoughts? Regards, Marcel -------------- next part -------------- An HTML attachment was scrubbed... URL: From nengard at gmail.com Thu Jun 10 14:10:59 2010 From: nengard at gmail.com (Nicole Engard) Date: Thu, 10 Jun 2010 08:10:59 -0400 Subject: [Koha-devel] New preferences editor In-Reply-To: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> Message-ID: Marcel, I have reported a lot of bugs about missing preferences. Most are found with this search: http://bugs.koha-community.org/bugzilla3/buglist.cgi?quicksearch=sys+prefs As for instructions on how to add them, I can write that up, but the reason I haven't added some of these is because the implication is that they are not used in the code anymore or because I don't know how to add that particular preference. Thanks Nicole 2010/6/10 Marcel de Rooy : > Hi all, > > > > I was just testing a small patch for 3.2 with a new preference not listed in > the pref templates. Although the pref exists, you cannot find it or add it. > (The manual does not yet explain how to add preferences too, or I could not > find it.) > > > > I think it should be interesting to include (possibly older) local > preferences that are not yet in the templates on a separate tab or so. The > same holds for searching the prefs; the new version does not find > preferences that are not listed in the pref files, but are in the database > and can be used via C4::Context. This is somewhat confusing (to me)? Would > it be better to allow finding them at least? > > > > Any thoughts? > > > > Regards, > > > > Marcel > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > From kmkale at anantcorp.com Sat Jun 12 12:38:23 2010 From: kmkale at anantcorp.com (Koustubha Kale) Date: Sat, 12 Jun 2010 16:08:23 +0530 Subject: [Koha-devel] User gets logged out while adding items. Message-ID: Hi, When we add a bib and go on to add an item the user gets logged out and we get below errors in the koha-error.log --------------------------------------------------log---------------------------------------------- [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] :7: parser error : Opening and ending tag mismatch: record line 6 and datafield, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] , referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] ^, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] :8: parser error : Opening and ending tag mismatch: collection line 2 and record, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] , referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] ^, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] :9: parser error : Extra content at the end of the document, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] , referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] ^, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl [Sat Jun 12 15:30:32 2010] [error] [client 10.1.1.4] Premature end of script headers: additem.pl, referer: http://10.1.1.4/cgi-bin/koha/cataloguing/additem.pl -----------------------------------------end log---------------------------------------------------- This does not happen with super librarian user. The concerned user has following privileges.. Set Privileges for dataentry, headoffice circulate Circulate books catalogue View Catalog (Librarian Interface) parameters Set Koha system parameters borrow Borrow books editcatalogue Edit Catalog (Modify bibliographic/holdings data) updatecharges Update borrower charges acquisition Acquisition and/or suggestion management serials Allow to manage serials subscriptions Koha version details : Koha version: 3.01.00.136 OS version ('uname -a'): Linux koha 2.6.24-19-generic #1 SMP Wed Jun 18 14:43:41 UTC 2008 i686 GNU/Linux Perl interpreter: /usr/bin/perl Perl version: 5.010000 Perl @INC: /home/kk/kohaclone /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . MySQL version: mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (i486) using readline 5.2 Apache version: Server version: Apache/2.2.8 (Ubuntu) Zebra version: Zebra 2.0.34 (C) 1994-2007, Index Data ApS Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Configured as: i486-pc-linux-gnu Using ICU Can some please throw some light on this problem? Regards, Koustubha Kale Anant Corporation Contact Details : Address ?: 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), ? ? ? ? Maharashtra, India, Pin : 400601. TeleFax ?: +91-22-21720108, +91-22-21720109 Mobile ? ? : +919820715876 Website ?: http://www.anantcorp.com Blog : http://www.anantcorp.com/blog/?author=2 From bibliwho at gmail.com Mon Jun 14 17:37:28 2010 From: bibliwho at gmail.com (Cab Vinton) Date: Mon, 14 Jun 2010 11:37:28 -0400 Subject: [Koha-devel] NoSQL Message-ID: Have come across references to this a few times recently & am curious enough to ask the development community whether the NoSQL movement holds any relevance for Koha's future. According to that font of universal wisdom, Wikipedia, "Typical modern relational databases have shown poor performance on data-intensive applications including indexing a large number of documents [= any ILS?], serving pages on high-traffic websites and delivering streaming media. They can be efficient only when they are tuned either for small but frequent read/write transactions or for large batch transactions with rare write accesses, while there are demands for the data stores capable of heavy workloads with frequent updates." Thoughts? How wedded is Koha to SQL in the long run? Cheers, Cab Vinton, Director Sanbornton Public Library Sanbornton, NH From ohiocore at gmail.com Mon Jun 14 17:53:36 2010 From: ohiocore at gmail.com (Joe Atzberger) Date: Mon, 14 Jun 2010 11:53:36 -0400 Subject: [Koha-devel] NoSQL In-Reply-To: References: Message-ID: On Mon, Jun 14, 2010 at 11:37 AM, Cab Vinton wrote: > Have come across references to this a few times recently & am curious > enough to ask the development community whether the NoSQL movement > holds any relevance for Koha's future. > > According to that font of universal wisdom, Wikipedia, "Typical modern > relational databases have shown poor performance on data-intensive > applications including indexing a large number of documents [= any > ILS?], serving pages on high-traffic websites and delivering streaming > media. They can be efficient only when they are tuned either for small > but frequent read/write transactions or for large batch transactions > with rare write accesses, while there are demands for the data stores > capable of heavy workloads with frequent updates." > > Thoughts? How wedded is Koha to SQL in the long run? > > Very wedded for now, and logically so, imho. We don't use SQL for indexing large numbers of (MARCXML) documents. We use zebra for that. Zebra, despite whatever complaints we might have about ease of configuration, certainly is highly scalable. Our documents are still pretty small by rest-of-the-world standards, though I could see this question coming back if features around digital repository functionality are submitted. --Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From rick at praxis.com.au Mon Jun 14 18:05:07 2010 From: rick at praxis.com.au (Rick Welykochy) Date: Tue, 15 Jun 2010 02:05:07 +1000 Subject: [Koha-devel] NoSQL In-Reply-To: References: Message-ID: <4C165333.5090104@praxis.com.au> Cab Vinton wrote: > According to that font of universal wisdom, Wikipedia, "Typical modern > relational databases have shown poor performance on data-intensive > applications including indexing a large number of documents [= any > ILS?], serving pages on high-traffic websites and delivering streaming > media. They can be efficient only when they are tuned either for small > but frequent read/write transactions or for large batch transactions > with rare write accesses, while there are demands for the data stores > capable of heavy workloads with frequent updates." I checked the date and it isn't April 1st! The wikipedia article indicates that RDB performance can start to flag when doing joins on databases that are in the TB and PB sizes (tera- and peta-byte). This is not the case for Koha, yet! Nor for the foreseeable future. > Thoughts? How wedded is Koha to SQL in the long run? Quite wedded in the data model. Actually, Koha is quite wedded to MySQL specifically. It would be a lot of work to convert the existing code to use a different SQL database, let aone a different style of data store. NoSQL is a storm in a teacup unless you are a Facebook type application dealing with truly enormous datasets. cheers rickw -- _________________________________ Rick Welykochy || Praxis Services "Most folk are about as happy as they make up their minds to be." -- anon From bibliwho at gmail.com Mon Jun 14 18:22:44 2010 From: bibliwho at gmail.com (Cab Vinton) Date: Mon, 14 Jun 2010 12:22:44 -0400 Subject: [Koha-devel] NoSQL In-Reply-To: References: Message-ID: OK, thanks, guys. I suspected as much based on not having seen any discussion of this before, but good to have confirmation from folks who actually know something :-) Curiosity satisfied! Cheers, Cab Vinton, Director Sanbornton Public Library Sanbornton, NH From paul.poulain at biblibre.com Mon Jun 14 18:29:32 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Mon, 14 Jun 2010 18:29:32 +0200 Subject: [Koha-devel] NoSQL In-Reply-To: References: Message-ID: <4C1658EC.5000500@biblibre.com> Le 14/06/2010 17:53, Joe Atzberger a ?crit : > > > Thoughts? How wedded is Koha to SQL in the long run? > > > Very wedded for now, and logically so, imho. ++ > > We don't use SQL for indexing large numbers of (MARCXML) documents. > We use zebra for that. Zebra, despite whatever complaints we might > have about ease of configuration, certainly is highly scalable. ++ > > Our documents are still pretty small by rest-of-the-world standards, > though I could see this question coming back if features around > digital repository functionality are submitted. > ++ no need to say more, Jo is 100% right ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -------------- section suivante -------------- Une pi?ce jointe HTML a ?t? nettoy?e... URL: From lars at catalyst.net.nz Mon Jun 14 23:54:26 2010 From: lars at catalyst.net.nz (Lars Wirzenius) Date: Tue, 15 Jun 2010 09:54:26 +1200 Subject: [Koha-devel] NoSQL In-Reply-To: References: Message-ID: <1276552466.921.1.camel@saimaa.wgtn.cat-it.co.nz> On ma, 2010-06-14 at 11:53 -0400, Joe Atzberger wrote: > Zebra, despite whatever complaints we might have about ease of > configuration [- - -] *cough*http://debian.koha-community.org/*cough* :) (This is what distributions of free software operating systems, e.g., Debian, are meant for: to take the pain out of installing powerful, complicated software. For whatever reason, most people who write the kinds of sophisticated software that makes you realize you live in the future and reduces near-term sci-fi authors to tears[1] are not so good at making it easy to install, configure, upgrade, and otherwise manage their software. That's where distributions help. To build something like Debian is not just a matter of having a fast machine to compile a lot of source code provided by other people. You also have to put in a lot of hard work to make everything work out of the box, and also get everything to work together.) [1] http://www.antipope.org/charlie/blog-static/2008/03/blindsided-by-the-future.html From mjr at phonecoop.coop Tue Jun 15 00:08:38 2010 From: mjr at phonecoop.coop (MJ Ray) Date: Mon, 14 Jun 2010 23:08:38 +0100 Subject: [Koha-devel] [Koha] Proposal To Switch Koha's License to GPLv3 and AGPLv3 or AGPLv3 In-Reply-To: References: <20100511015407.3BAE2F724D@nail.towers.org.uk> Message-ID: <4c16a866.XIWfuaNFoEeGVobK%mjr@phonecoop.coop> (Koha main list dropped from cc) Chris Nighswonger wrote: > On Mon, May 10, 2010 at 9:54 PM, MJ Ray wrote: > > Secondly, this leads to another of what I think is still one of the > > Great Unknowns of AGPLv3: if you don't host the source alongside, must > > the app go offline if it thinks the source has gone offline? > > Where in the license does it say anything about hosting source > alongside the application? Nowhere, but it does say "your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source". If the source has gone offline, there is no opportunity to receive it, is there? > > There are so many of these lawyerbombs around AGPLv3 that I feel the > > whole thing is best avoided by sticking with GPLv2, at least until > > others have trod on some of the big ones, in the absence of any > > pressing need to switch. > > GPLvX was virtually untested in court until Progress Software Corp. v. > MySQL began back in 2002. Apparently that line of reasoning did not > stop the originators of Koha from selecting GPLv2+ when they released > it in 2000. There are very few bits of language in GPLv2 which are anything like as vague and confusing as the AGPLv3 clause. [...] > > I don't see how that's true, unless "cooperation of some sort" means > > something other than cooperation, such as mere trading. > > Free software licensing is usually just setting out the terms of trade, > > but AGPLv3's clod-handed attempts to force public sharing go beyond it. > > View it how you will. In the case you propose, we are simply > establishing an additional clarification to the terms of trade. The > intent of GPL licenses is to preserve the open nature of code released > as free and open source. Not only is it the right of the receiver to > have access to the code, etc. but it is also the right of the author > to have their intent that the code be free and open respected. The > terms of the trade are such that in return for the right to use, etc. > this code, you agree to release any changes you make to said parties. Forced publication was never part of the free and open nature before. That is not a clarification. It's a fundamental shift between user freedoms and author freedoms, driven by a reactionary approach to unfriendly users who lend their computing resources to others. > As I pointed out previously: All licensing agreements "force" some > points. Just violate one and see how quickly it is en-forced. (Pun > intended.) It's actually quite rare. When it does happen, it is painful, as I discovered to my cost in a past job, but it is still rare. > It actually seems to me that the matter of not forcing cooperation is > in contradiction to the expressed desire to force cooperation in the > matter of trademarks, etc. by de-listing, etc. Cooperation is, after > all, cooperation. Trademarks are not copyright and trying to treat them exactly the same will lead to some very unhealthy conclusions. We can look for the same freedoms (use, adapt, study and share) but in different ways. Also, I'm pretty sure I've never expressed any desire to force cooperation. [...] > > So what is the burning desire for AGPLv3? > [...] It is > not a panacea for all ills, only another hopeful obstacle in the path > of misbehavior. This community has so far been relatively unwilling to activate the obstacles it already has placed in that path. Another one will only serve to deter and scare its friends further, because the only people who heed the obstacles are those who respect its wishes. Furthermore, this isn't even an obstacle in the path of the bad hosters: "One problem which the GNU Affero GPL does not address is the problem of Software as a Service (SaaS). It is impossible, as far as we know, to address this problem with a software license." -- http://www.gnu.org/licenses/why-affero-gpl.html > > > > So may we postpone the rest of this discussion to post-3.2.0? > > > > > > As I stated in my original proposal: We are already very active atm, and now is > > > the time to at least begin discussing this change. > > > > I am disappointed by this desire to press ahead with holding a > > discussion of such a complex topic at such a busy time. ?It will limit > > participation and likely leave the discussion incomplete. > > Participation will only be limited if we want it to. No one involved > in this project is not busy. If we believe that we have a good thing > going in Koha, we must make the time to do the not so nice parts of > the project too. There is no point adding not so nice parts that bring no benefit unless you think this is an S+M community or something. Instead, let's find ways to make Koha more fun and more general! Regards, -- MJ Ray (slef) Webmaster and LMS developer at | software www.software.coop http://mjr.towers.org.uk | .... co IMO only: see http://mjr.towers.org.uk/email.html | .... op From chris at bigballofwax.co.nz Wed Jun 16 06:50:54 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Wed, 16 Jun 2010 16:50:54 +1200 Subject: [Koha-devel] memcached? In-Reply-To: References: <4C0910EB.7070104@ccfls.org> <4C091D57.2020702@ptfs-europe.com> <20100604164033.GB599@rot13.org> Message-ID: On 6 June 2010 13:38, Chris Cormack wrote: > 2010/6/5 Fouts, Clay : >> I can also attest to the benefit of storing session data somewhere other >> than in the MySQL database.?LLEK has the ability to store its session data >> through memcached instead of writing to a file (which can create problems in >> a multi-server configuration) or MySQL. This frees up the database more than >> one might expect. The constant stream of tiny, non-contiguous INSERTs and >> UPDATEs contributes heavily to fragmentation of InnoDB writes which in turn >> lengthens transaction times. (Not to mention Koha's inability to selectively >> cull stale sessions, causing your table to have millions of rows over time). >> The drawback is of course that if memcached goes away you lose all your >> current session data since it's not a permanent storage medium, but this is >> not a significant loss in most circumstances. > > Storing as temp files on a ram based disk as has already been > suggested works much the same way too. Of course a patch for using > memcached for session storage would be gratefully accepted and would > save on once again doubling up on work already done. > This code is now in the Koha git repository, I'm not sure if it is implemented the same way in LEK, but it's now in Koha anyway. http://git.koha-community.org/gitweb/?p=koha.git;a=shortlog;h=refs/heads/new/memcached Chris From henridamien.laurent at gmail.com Wed Jun 16 23:29:28 2010 From: henridamien.laurent at gmail.com (LAURENT Henri-Damien) Date: Wed, 16 Jun 2010 23:29:28 +0200 Subject: [Koha-devel] New preferences editor In-Reply-To: References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> Message-ID: <4C194238.2080405@gmail.com> Le 10/06/2010 14:10, Nicole Engard a ?crit : > Marcel, > > I have reported a lot of bugs about missing preferences. Most are > found with this search: > http://bugs.koha-community.org/bugzilla3/buglist.cgi?quicksearch=sys+prefs > > As for instructions on how to add them, I can write that up, but the > reason I haven't added some of these is because the implication is > that they are not used in the code anymore or because I don't know how > to add that particular preference. > > Thanks > Nicole Hi, about that new preferences editor, I really can appreciate the work Jesse Weaver did. I like the javascript and the hierarchy and the search and the way to update the whole page at once... Really. But it appears that to us, it is more messy than the previous system : - requires the edition of many files (sql and pref) rather than centralizing things. - the translation of the yaml file will be clumsy (think of arabic ppl who donot have the same way to tell Do don't) - And the Boolean approach in that system is quite inadapted to PERL : No and Yes for 0 and 1... Problem is that PERL interprets No as 1... unfortunately. We are therefore compelled not to use that system.... Anyone here having the same problems ? -- Henri-Damien LAURENT From frederic at tamil.fr Thu Jun 17 07:39:54 2010 From: frederic at tamil.fr (Frederic Demians) Date: Thu, 17 Jun 2010 07:39:54 +0200 Subject: [Koha-devel] New preferences editor In-Reply-To: <4C194238.2080405@gmail.com> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> Message-ID: <4C19B52A.3070402@tamil.fr> > about that new preferences editor, I really can appreciate the work > Jesse Weaver did. I like the javascript > and the hierarchy and the search and the way to update the whole page > at once... Really. I agree. The idea is good. The implementation is arguable. > - requires the edition of many files (sql and pref) rather than > centralizing things. The procedure for adding a new syspref has changed : * Before: o add an entry in systempreferences.pl file to distribute the syspref to an editor tab (no so good!) o add a revision to updatedatabase.pl to append the new syspref to an installed Koha o add an entry into kohastructure.sql * Now, with the new editor: o add a revision to updatedatabase.pl to append the new syspref to an installed Koha o add a entry into kohastructure.sql o add the syspref in a YAML file I'd had preferred system preferences to be outsided from the DB and put into a single config file. > - the translation of the yaml file will be clumsy (think of arabic ppl > who donot have the same way to tell Do don't) +1 -- It's stil an issue with French. I can't imagine what non-latin languages translator will have to cope with. The syntax is different and so the way a syspref can be embedded into a sentence. With the current syspref editor, a preference (and 2 sometime) is expressed in a sentence which is cut in several parts. The syspref is inserted somewhere in the sentence, roughly at the beginning, the middle or the end. This position which make sense in English has to be preserved by the translator. Very often it will oblige the translator to construct sentences which are syntactically ill-formed in his language. > We are therefore compelled not to use that system.... What does it mean? -- Fr?d?ric DEMIANS From henridamien.laurent at gmail.com Thu Jun 17 09:21:55 2010 From: henridamien.laurent at gmail.com (LAURENT Henri-Damien) Date: Thu, 17 Jun 2010 09:21:55 +0200 Subject: [Koha-devel] New preferences editor In-Reply-To: <4C19B52A.3070402@tamil.fr> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> <4C19B52A.3070402@tamil.fr> Message-ID: <4C19CD13.3000003@gmail.com> Le 17/06/2010 07:39, Frederic Demians a ?crit : > > >> We are therefore compelled not to use that system.... > > What does it mean? > As it stands now, and since some customers are going live soon or already have. We replaced all the occurences of preferences.pl in admin-home and main page with systempreferences.pl since that script was still to be used in order to add new preferences and was still usable even though not very user-friendly. -- Henri-Damien LAURENT From chris at bigballofwax.co.nz Thu Jun 17 10:51:26 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 17 Jun 2010 20:51:26 +1200 Subject: [Koha-devel] New preferences editor In-Reply-To: <4C19B52A.3070402@tamil.fr> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> <4C19B52A.3070402@tamil.fr> Message-ID: On 17 June 2010 17:39, Frederic Demians wrote: > >> about that new preferences editor, I really can appreciate the work Jesse >> Weaver did. I like the javascript >> and the hierarchy and the search and the way to update the whole page at >> once... Really. > > I agree. The idea is good. The implementation is arguable. > >> - requires the edition of many files (sql and pref) rather than >> centralizing things. > > The procedure for adding a new syspref has changed : > > ? * Before: > ? ? ? ? o add an entry in systempreferences.pl file to distribute the > ? ? ? ? ? syspref to an editor tab (no so good!) > ? ? ? ? o add a revision to updatedatabase.pl to append the new > ? ? ? ? ? syspref to an installed Koha > ? ? ? ? o add an entry into kohastructure.sql > ? * Now, with the new editor: > ? ? ? ? o add a revision to updatedatabase.pl to append the new > ? ? ? ? ? syspref to an installed Koha > ? ? ? ? o add a entry into kohastructure.sql > ? ? ? ? o add the syspref in a YAML file > > I'd had preferred system preferences to be outsided from the DB and put into > a single config file. > >> - the translation of the yaml file will be clumsy (think of arabic ppl who >> donot have the same way to tell Do don't) > > +1 -- It's stil an issue with French. I can't imagine what non-latin > languages translator will have to cope with. The syntax is different and so > the way a syspref can be embedded into a sentence. ?With the current syspref > editor, a preference (and 2 sometime) is expressed in a sentence which is > cut in several parts. The syspref is inserted somewhere in the sentence, > roughly at the beginning, the middle or the end. This position which make > sense in English has to be preserved by the translator. Very often it will > oblige the translator to construct sentences which are syntactically > ill-formed in his language. > >> We mustn't forget that before we couldn't translate the systempreferences at all, or we could only do it into one language you certainly couldn't have it translated in more than one, from that respect its a huge improvement. So before we dump on it too much, lets remember that. Lets also remember that Jesse did it, while others talked about it, someone went and wrote some code, lets do more of that and less of denigrating others work. Chris From frederic at tamil.fr Thu Jun 17 12:27:00 2010 From: frederic at tamil.fr (Frederic Demians) Date: Thu, 17 Jun 2010 12:27:00 +0200 Subject: [Koha-devel] New preferences editor In-Reply-To: References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> <4C19B52A.3070402@tamil.fr> Message-ID: <4C19F874.3020307@tamil.fr> > We mustn't forget that before we couldn't translate the > systempreferences at all, or we could only do it into one language you > certainly couldn't have it translated in more than one, from that > respect its a huge improvement. You get a point. Before, sysprefs were translatable but only by updating .sql files. Now, and I agree it's a huge improvement, translation can be delegated to translator rather than to developers. My concern about difficulties for translators to get syntactic well-formed sentences can have a solution if syspref description are well chosen and avoid English idiotisms. We must get feedback from translators about issues they encounter and then define rules for syspref descriptions. > So before we dump on it too much, lets remember that. Lets also > remember that Jesse did it, while others talked about it, someone went > and wrote some code, lets do more of that and less of denigrating > others work. It wasn't my intent and so I apologize if it was perceived this way. I add a page in the wiki to go ahead in the right direction for developers and translators: http://wiki.koha-community.org/wiki/System_Preferences Native English speakers are invited to fix my misspelling and gallicisms. -- Fr?d?ric DEMIANS From chris at bigballofwax.co.nz Thu Jun 17 12:35:00 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 17 Jun 2010 22:35:00 +1200 Subject: [Koha-devel] New preferences editor In-Reply-To: <4C19F874.3020307@tamil.fr> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> <4C19B52A.3070402@tamil.fr> <4C19F874.3020307@tamil.fr> Message-ID: On 17 June 2010 22:27, Frederic Demians wrote: > >> We mustn't forget that before we couldn't translate the systempreferences >> at all, or we could only do it into one language you certainly couldn't have >> it translated in more than one, from that respect its a huge improvement. > > You get a point. Before, sysprefs were translatable but only by updating > .sql files. Now, and I agree it's a huge improvement, translation can be > delegated to translator rather than to developers. My concern about > difficulties for translators to get syntactic well-formed sentences can have > a solution if syspref description are well chosen and avoid English > idiotisms. We must get feedback from translators about issues they encounter > and then define rules for syspref descriptions. > I think that's a great idea. >> So before we dump on it too much, lets remember that. Lets also remember >> that Jesse did it, while others talked about it, someone went and wrote some >> code, lets do more of that and less of denigrating others work. > > It wasn't my intent and so I apologize if it was perceived this way. It was probably an overreaction on my part and I apologise also. > > I add a page in the wiki to go ahead in the right direction for developers > and translators: > > ?http://wiki.koha-community.org/wiki/System_Preferences > > Native English speakers are invited to fix my misspelling and gallicisms. Fabulous, looking now. Chris From rick at praxis.com.au Thu Jun 17 14:03:57 2010 From: rick at praxis.com.au (Rick Welykochy) Date: Thu, 17 Jun 2010 22:03:57 +1000 Subject: [Koha-devel] New preferences editor In-Reply-To: <4C19F874.3020307@tamil.fr> References: <32857065C79E3F4E8FDFBB0CAC6897F79A76@S-MAIL-1B.rijksmuseum.intra> <4C194238.2080405@gmail.com> <4C19B52A.3070402@tamil.fr> <4C19F874.3020307@tamil.fr> Message-ID: <4C1A0F2D.3090907@praxis.com.au> Frederic Demians wrote: > well chosen and avoid English idiotisms. [SNIP] > Native English speakers are invited to fix my misspelling and gallicisms. I had to smile with your use of the above malapropism above. Many English idioms, and French idioms for that matter, are indeed idiotisms :) cheers rick -- _________________________________ Rick Welykochy || Praxis Services "Most folk are about as happy as they make up their minds to be." -- anon From ian.walls at bywatersolutions.com Fri Jun 18 14:48:20 2010 From: ian.walls at bywatersolutions.com (Ian Walls) Date: Fri, 18 Jun 2010 08:48:20 -0400 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> <4C0673CC.8070903@alinto.com> Message-ID: Had a thought randomly in the night about this thread, so I thought I'd resurrect it briefly. There had been some discussion about whether to base releases on features or fixed time schedules, or a combination thereof. I think a combination is probably the best, and would suggest the following: X.Y releases (like 3.2 or 3.4) being released on a features basis. That is, when a sufficient number of the RFCs for the release are met, the Release Manager says its time to publish the new version. Hopefully this will happen on a fairly regular basis, but we all know how development can go... X.Y.Z releases (like 3.2.1 or 3.2.2) being released on a fixed schedule. These updates include all the new features and bug fixes that the Quality Assurance Manager and Release Maintainer agree are stable and solid. There will be as many of these updates as there is time for before the next major release. I'm completely open to the periodicity of these releases. As Chris points out, this is only possible if bug fixes and features can be isolated from one another for integration into the timed releases. Developers publishing early and often will help with this, as well as maintaining Git repos with separate branches for various developments. The new git.koha-community.org site has the potential to allow various development groups to push code to their own branches in a safe and secure way, so that may be one way for teams working on longer-term projects to more easily integrate separate features. I believe this could also improve collaboration across development teams. Does this sound reasonable to folks? Am I missing any major points of consideration? Cheers, -Ian On Wed, Jun 2, 2010 at 7:38 PM, Reed Wade wrote: > On Thu, Jun 3, 2010 at 3:07 AM, LAURENT Henri-Damien > wrote: > > Le 02/06/2010 12:22, Reed Wade a ?crit : > >> Any reason not to move to a 4 week release cycle? With alternate ones > >> getting "better" testing. > >> > >> It would mean automating a lot of things -- but doable and worth it in > any case. > > > > ^^The main reason is this one. > > and no doubt that it IS BOTH doable AND worth.... But in what time and > > who would do that... BibLibre is willing to devote time for that... But > > this surely is a community effort with development and should be managed > > as such. > > maybe there should be teams along with the assignees of bugs which could > > make smaller patches, and meetings so as to report progess. > > > > (4 weeks is probably too short but...) > > Yes, getting the science and organisation correct would be key. > > Something ChrisC has alluded to in emails and IRC is the scheme we > have developed at Catalyst for a project he and I and (5-8 others) > work on. It is a long running development effort for a high scale news > web site. It has a similar architecture and larger code base than > Koha. We use git and there's a lot of perl. It seems likely there are > some bits of code and techniques that could be applied to Koha release > mgmt. > > We have a 4 (really 1-6) week release cycle which is feature and date > driven (customer might need new thing in time for national elections > or sporting event and they don't tend to plan too far ahead). > > We use debian packages for all code deployments and DB updates. > > We use a system similar to bugzilla for tracking code development. > Each feature or bug fix gets a branch named according to the ticket > number. > > We have an automated process which merges all branches for the next > release (it queries the issue tracker to know what tickets are in what > release) and emails the authors if there's a conflict. This runs > several times a day. Instructions embedded in the ticket comments are > used to sort dependencies and additional / alternate branches if > needed for conflict resolution. > > We have a similar automated process which performs the merge then > creates debian packages for our testing environment. It emails the > results to the release manage who reviews and sets the status of > updated features so QA staff know what they can now test (this should > be automated but isn't just yet because I like to review these still). > > We have a pre-production autobuild set up which is a duplicate of all > that but it only takes branches that QA have signed off on (examines > ticket status). > > This scheme has been extraordinarily satisfying and effective and > allows us to spend more time on real work instead of the mechanics of > the process. Production deployments used to be long bad days and now > they tend to be uneventful. Everyone is happier. Release management is > now time spent considering product features instead of the mechanics > of integration and testing. > > The key benefits we've gained are: > - at any moment you can play with what would be the release if you > decided to release at that moment > - developers become responsible for fixing code conflicts or other > blocks to their branch getting into a release and without hardly any > time needed by the release manager (so very low latency for fixes) to > wrangle that because it's automated emails telling the developer > something went wrong > > -- > > How to apply to Koha? > > There are differences. We are a small group, mostly in the same > office. Our customer is a 5 minute walk down the street from us. > > I was about to say that our trade-offs and priorities are clear -- but > that wasn't true before we had the system described above. It's > allowed us and the customer to focus on keeping things clear because > they've seen the value of things like having a feature freeze date > that doesn't change unless the release date changes. > > Some key things that could be taken-- > > - the auto-build and merge and report and package on a test machine > for easy QA system > -- this is something we should be able to adapt to bugzilla and make > available for the project > > - the notion of naming branches according to bug/feature id -- we've > found this to be very powerful > > Chris or Lars may have something to add. > > -reed > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > -- Ian Walls Lead Development Specialist ByWater Solutions ALA Booth # 817 Phone # (888) 900-8944 http://bywatersolutions.com ian.walls at bywatersolutions.com Twitter: @sekjal -------------- next part -------------- An HTML attachment was scrubbed... URL: From nengard at gmail.com Mon Jun 21 19:07:07 2010 From: nengard at gmail.com (Nicole Engard) Date: Mon, 21 Jun 2010 13:07:07 -0400 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C062DAA.70607@gmail.com> <4C0673CC.8070903@alinto.com> Message-ID: I was actually thinking the other day (when WP 3.0 was released) and this model is probably something like what they do. Lots of little releases between each major release keeps me excited for the next time WordPress prompts me to upgrade :) Nicole 2010/6/18 Ian Walls > Had a thought randomly in the night about this thread, so I thought I'd > resurrect it briefly. > > There had been some discussion about whether to base releases on features > or fixed time schedules, or a combination thereof. I think a combination is > probably the best, and would suggest the following: > > X.Y releases (like 3.2 or 3.4) being released on a features basis. That > is, when a sufficient number of the RFCs for the release are met, the > Release Manager says its time to publish the new version. Hopefully this > will happen on a fairly regular basis, but we all know how development can > go... > > X.Y.Z releases (like 3.2.1 or 3.2.2) being released on a fixed schedule. > These updates include all the new features and bug fixes that the Quality > Assurance Manager and Release Maintainer agree are stable and solid. There > will be as many of these updates as there is time for before the next major > release. I'm completely open to the periodicity of these releases. > > As Chris points out, this is only possible if bug fixes and features can be > isolated from one another for integration into the timed releases. > Developers publishing early and often will help with this, as well as > maintaining Git repos with separate branches for various developments. The > new git.koha-community.org site has the potential to allow various > development groups to push code to their own branches in a safe and secure > way, so that may be one way for teams working on longer-term projects to > more easily integrate separate features. I believe this could also improve > collaboration across development teams. > > Does this sound reasonable to folks? Am I missing any major points of > consideration? > > Cheers, > > > -Ian > > On Wed, Jun 2, 2010 at 7:38 PM, Reed Wade wrote: > >> On Thu, Jun 3, 2010 at 3:07 AM, LAURENT Henri-Damien >> wrote: >> > Le 02/06/2010 12:22, Reed Wade a ?crit : >> >> Any reason not to move to a 4 week release cycle? With alternate ones >> >> getting "better" testing. >> >> >> >> It would mean automating a lot of things -- but doable and worth it in >> any case. >> > >> > ^^The main reason is this one. >> > and no doubt that it IS BOTH doable AND worth.... But in what time and >> > who would do that... BibLibre is willing to devote time for that... But >> > this surely is a community effort with development and should be managed >> > as such. >> > maybe there should be teams along with the assignees of bugs which could >> > make smaller patches, and meetings so as to report progess. >> >> >> >> (4 weeks is probably too short but...) >> >> Yes, getting the science and organisation correct would be key. >> >> Something ChrisC has alluded to in emails and IRC is the scheme we >> have developed at Catalyst for a project he and I and (5-8 others) >> work on. It is a long running development effort for a high scale news >> web site. It has a similar architecture and larger code base than >> Koha. We use git and there's a lot of perl. It seems likely there are >> some bits of code and techniques that could be applied to Koha release >> mgmt. >> >> We have a 4 (really 1-6) week release cycle which is feature and date >> driven (customer might need new thing in time for national elections >> or sporting event and they don't tend to plan too far ahead). >> >> We use debian packages for all code deployments and DB updates. >> >> We use a system similar to bugzilla for tracking code development. >> Each feature or bug fix gets a branch named according to the ticket >> number. >> >> We have an automated process which merges all branches for the next >> release (it queries the issue tracker to know what tickets are in what >> release) and emails the authors if there's a conflict. This runs >> several times a day. Instructions embedded in the ticket comments are >> used to sort dependencies and additional / alternate branches if >> needed for conflict resolution. >> >> We have a similar automated process which performs the merge then >> creates debian packages for our testing environment. It emails the >> results to the release manage who reviews and sets the status of >> updated features so QA staff know what they can now test (this should >> be automated but isn't just yet because I like to review these still). >> >> We have a pre-production autobuild set up which is a duplicate of all >> that but it only takes branches that QA have signed off on (examines >> ticket status). >> >> This scheme has been extraordinarily satisfying and effective and >> allows us to spend more time on real work instead of the mechanics of >> the process. Production deployments used to be long bad days and now >> they tend to be uneventful. Everyone is happier. Release management is >> now time spent considering product features instead of the mechanics >> of integration and testing. >> >> The key benefits we've gained are: >> - at any moment you can play with what would be the release if you >> decided to release at that moment >> - developers become responsible for fixing code conflicts or other >> blocks to their branch getting into a release and without hardly any >> time needed by the release manager (so very low latency for fixes) to >> wrangle that because it's automated emails telling the developer >> something went wrong >> >> -- >> >> How to apply to Koha? >> >> There are differences. We are a small group, mostly in the same >> office. Our customer is a 5 minute walk down the street from us. >> >> I was about to say that our trade-offs and priorities are clear -- but >> that wasn't true before we had the system described above. It's >> allowed us and the customer to focus on keeping things clear because >> they've seen the value of things like having a feature freeze date >> that doesn't change unless the release date changes. >> >> Some key things that could be taken-- >> >> - the auto-build and merge and report and package on a test machine >> for easy QA system >> -- this is something we should be able to adapt to bugzilla and make >> available for the project >> >> - the notion of naming branches according to bug/feature id -- we've >> found this to be very powerful >> >> Chris or Lars may have something to add. >> >> -reed >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> > > > > -- > Ian Walls > Lead Development Specialist > ByWater Solutions > ALA Booth # 817 > Phone # (888) 900-8944 > http://bywatersolutions.com > ian.walls at bywatersolutions.com > Twitter: @sekjal > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.poulain at biblibre.com Mon Jun 21 19:47:01 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Mon, 21 Jun 2010 19:47:01 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C060F5C.9010402@biblibre.com> References: <4C060F5C.9010402@biblibre.com> Message-ID: <4C1FA595.9000700@biblibre.com> Le 02/06/2010 10:25, Chris Cormack a ?crit : > > So as RM I accept the challenge of faster releases, and throw back a > > challenge of better patches :) > Chris & al, We (BibLibre & the community) face a big problem, and we must deal with it here and now : we (BibLibre) are deploying biblibre/master. It's live in Lyon3 university, it'll be live in Nimes next weeks, in SAN-OP in a few more weeks (migrating from 3.0). It contains a lot of new features (circ rules rewritten, and other new features described in wiki.koha-community.org, RFC for 3.4 from BibLibre) And we have a lot of new contracts falling. The technical fact : - we (BibLibre) have 450+ patches on git.biblibre.com, master branch, that are not in git.koha-community.org (new features & bugfixes) - koha-community has 100 patches that are not on biblibre/master So : if we do nothing, in a few *weeks*, we will have a fork that we want to avoid ! Here is our proposal : 1- release 3.2 2- we (BibLibre) rebase our master to 3.2 (that's 450+ patches) 3- we (BibLibre) submit them to master/koha 4- you (chris) push them into main trunk (as they are) 5- we (community) define the rules for all the next devs. 6- start working with new rules => september 1-Release 3.2 : ============= hdl will take care this next two weeks of the 2 BLO related to updatedatabase & affected to him 2 - rebase biblibre/master on koha/master ============== We (BibLibre) take care of rebasing our master at a date we define altogether. As that will be a HUGE work, you (chris) don't apply any other patches while we rebase to avoid breaking our work. We think we would need one week (maybe two) to rebase & test properly. 3/4 - push biblibre patches on koha/master ============== You (chris) push our 450+ patches on koha/master. Since the new rules were not defined when we wrote all this code, those patches don't abide by those rules and can't. 5 - define rules ============== Once 3.2 is released, we (community) set all rules (Including coding style, test cases, choosing OO pattern, standard commit notes,...) and everybody use them - including us (BibLibre) we don't request to be favored - 6 - work on 3.4 ============== All RFCs we wrote on the wiki are now implemented for our Nimes & Lyon 3 customers. If something new comes, we will respect fully the new rules. There may be some patches fixing Lyon3/Nimes features that won't respect them, but, hopefully, there will be only a few of them (Lyon3 is live, Nimes go live next week, so it's really stable) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -------------- section suivante -------------- Une pi?ce jointe HTML a ?t? nettoy?e... URL: From chris at bigballofwax.co.nz Mon Jun 21 20:44:22 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Tue, 22 Jun 2010 06:44:22 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C1FA595.9000700@biblibre.com> References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> Message-ID: 2010/6/22 Paul Poulain : > Le 02/06/2010 10:25, Chris Cormack a ?crit : > >> So as RM I accept the challenge of faster releases, and throw back a >> challenge of better patches :) > > > Chris & al, > > We (BibLibre & the community) face a big problem, and we must deal with > it here and now : we (BibLibre) are deploying biblibre/master. It's live > in Lyon3 university, it'll be live in Nimes next weeks, in SAN-OP in a > few more weeks (migrating from 3.0). > It contains a lot of new features (circ rules rewritten, and other new > features described in wiki.koha-community.org, RFC for 3.4 from BibLibre) > > And we have a lot of new contracts falling. > > The technical fact : > - we (BibLibre) have 450+ patches on git.biblibre.com, master branch, > that are not in git.koha-community.org (new features & bugfixes) > - koha-community has 100 patches that are not on biblibre/master > > So : if we do nothing, in a few *weeks*, we will have a fork that we > want to avoid ! > > Here is our proposal : > 1- release 3.2 > 2- we (BibLibre) rebase our master to 3.2 (that's 450+ patches) > 3- we (BibLibre) submit them to master/koha > 4- you (chris) push them into main trunk (as they are) > 5- we (community) define the rules for all the next devs. > 6- start working with new rules => september > > 1-Release 3.2 : > ============= > hdl will take care this next two weeks of the 2 BLO related to > updatedatabase & affected to him > > 2 - rebase biblibre/master on koha/master > ============== > We (BibLibre) take care of rebasing our master at a date we define > altogether. As that will be a HUGE work, you (chris) don't apply any other > patches while we rebase to avoid breaking our work. We think we would need > one week (maybe two) to rebase & test properly. > > 3/4 - push biblibre patches on koha/master > ============== > You (chris) push our 450+ patches on koha/master. Since the new rules were > not defined when we wrote all this code, those patches don't abide by those > rules and can't. > > 5 - define rules > ============== > Once 3.2 is released, we (community) set all rules (Including coding style, > test cases, choosing OO pattern, standard commit notes,...) and everybody > use them - including us (BibLibre) we don't request to be favored - > > 6 - work on 3.4 > ============== > All RFCs we wrote on the wiki are now implemented for our Nimes & Lyon 3 > customers. If something new comes, we will respect fully the new rules. > There may be some patches fixing Lyon3/Nimes features that won't respect > them, but, hopefully, there will be only a few of them (Lyon3 is live, Nimes > go live next week, so it's really stable) > -- This is certainly a possible plan of action, but no, I won't be pushing any patches to master without them going through QA. Certainly not 450 in one go. They will need to be branched into smaller feature sets and each one tested and merged. What I am willing to do, is be less strict about accompanying tests as the patches were written before that, what I am not willing to do is merge 450 patches that haven't been looked at by QA or the Release Manager into master. These aren't new rules. Splitting them up into featuresets will make their integration much faster. Chris From nengard at gmail.com Mon Jun 21 21:08:29 2010 From: nengard at gmail.com (Nicole Engard) Date: Mon, 21 Jun 2010 15:08:29 -0400 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> Message-ID: On Mon, Jun 21, 2010 at 2:44 PM, Chris Cormack wrote: > This is certainly a possible plan of action, but no, I won't be > pushing any patches to master without them going through QA. > Certainly not 450 in one go. > I don't know a lot about the programming aspects - but I have to agree with Chris on this. The idea of 450 patches going in without testing seems like a bit of a nightmare waiting to happen. It would mean finding bugs (which I'm famous for) and not being able to track down what caused it as easily. I'd feel more comfortable as a tester/doc manager/user if I knew thing were being tested in smaller chunks along the way. > > Chris > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cnighswonger at foundations.edu Mon Jun 21 21:34:17 2010 From: cnighswonger at foundations.edu (Chris Nighswonger) Date: Mon, 21 Jun 2010 15:34:17 -0400 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C1FA595.9000700@biblibre.com> References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> Message-ID: Hi Paul, 2010/6/21 Paul Poulain : > The technical fact : > - we (BibLibre) have 450+ patches on git.biblibre.com, master branch, > that are not in git.koha-community.org (new features & bugfixes) > - koha-community has 100 patches that are not on biblibre/master Ouch! > > So : if we do nothing, in a few *weeks*, we will have a fork that we > want to avoid ! Great! > > 1-Release 3.2 : > ============= > hdl will take care this next two weeks of the 2 BLO related to > updatedatabase & affected to him > > 2 - rebase biblibre/master on koha/master > ============== > We (BibLibre) take care of rebasing our master at a date we define > altogether. As that will be a HUGE work, you (chris) don't apply any other > patches while we rebase to avoid breaking our work. We think we would need > one week (maybe two) to rebase & test properly. I think, this would be the place to implement the first of Chris' suggestions: Have hdl (or whoever) do this merging in such a way as to produce smaller feature-centric branches. (Think "bite sized" pieces.) However, I think this must be done concurrent to development on 3.4. We must also consider that if we are ever to have schedule based releases, we *must* take this opportunity to hold on to a stable master and use branching along with merging to facilitate ongoing development while maintaining that stable master. So here is a good place to begin. As BibLibre finishes up merging on a particular feature branch, QA can immediately pickup with testing and then the RM take care of merging the tested branch into master, thus keeping master stable. I think if everyone pitches in that this could all happen in a month time frame. As you say, these new features are in production on several of your large clients, so they should not be too buggy. > > 3/4 - push biblibre patches on koha/master > ============== > You (chris) push our 450+ patches on koha/master. Since the new rules were > not defined when we wrote all this code, those patches don't abide by those > rules and can't. As Chris and Nicole have pointed out, I don't think a direct push into a stable master of this large of a patch set will prove beneficial to anyone. > > 5 - define rules > ============== > Once 3.2 is released, we (community) set all rules (Including coding style, > test cases, choosing OO pattern, standard commit notes,...) and everybody > use them - including us (BibLibre) we don't request to be favored - > > 6 - work on 3.4 > ============== > All RFCs we wrote on the wiki are now implemented for our Nimes & Lyon 3 > customers. If something new comes, we will respect fully the new rules. > There may be some patches fixing Lyon3/Nimes features that won't respect > them, but, hopefully, there will be only a few of them (Lyon3 is live, Nimes > go live next week, so it's really stable) The rest of this naturally follows along of course. We need some enforcement of rules especially as the project gets bigger and larger support companies are pushing the development at a greater pace. It will be especially imperative that those who have the resources to push development faster adhere more strictly to good coding/testing/etc practices as messy code in large quantities will quickly produce chaos and be of little use in the long haul. Kudos to BbLibre for being open and transparent in all of this and for requesting community input as they undergo growing pains. Kind Regards, Chris From paul.poulain at biblibre.com Mon Jun 21 21:40:23 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Mon, 21 Jun 2010 21:40:23 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> Message-ID: <4C1FC027.3010407@biblibre.com> Le 21/06/2010 20:44, Chris Cormack a ?crit : > This is certainly a possible plan of action, but no, I won't be > pushing any patches to master without them going through QA. > Certainly not 450 in one go. They will need to be branched into > smaller feature sets and each one tested and merged. What I am willing > to do, is be less strict about accompanying tests as the patches were > written before that, what I am not willing to do is merge 450 patches > that haven't been looked at by QA or the Release Manager into master. > These aren't new rules. 1- we did a lot of QA on them, and although there may be some remaining bugs, we are live with them on 1 site, very soon on a second one, and many more. 2- we proposed to submit those patches many months ago, but galen decision was : "feature freeze for 3.2". We had to go ahead. 3- we will deal with any problem that may arise. And if 3.4 has to be released in something like 6 months, that's enough, undoubtfully 4- splitting them in smaller feature set is a more than huge work. It's probably almost impossible. So, even if your proposition is a good idea, I think it's not a possible way to go for us. You're reaching our (BibLibre) limit. So, if someone is volunteering for this more-than-huge job, then, fair, head to git.biblibre.com/master, it's here, available, all commits comments are in english. But you can't count on us to do that, even if I'm sorry, really. Friendly. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 From chris at bigballofwax.co.nz Mon Jun 21 21:48:43 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Tue, 22 Jun 2010 07:48:43 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C1FC027.3010407@biblibre.com> References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> <4C1FC027.3010407@biblibre.com> Message-ID: On 22 June 2010 07:40, Paul Poulain wrote: > Le 21/06/2010 20:44, Chris Cormack a ?crit : >> This is certainly a possible plan of action, but no, I won't be >> pushing any patches to master without them going through QA. >> Certainly not 450 in one go. They will need to be branched into >> smaller feature sets and each one tested and merged. What I am willing >> to do, is be less strict about accompanying tests as the patches were >> written before that, what I am not willing to do is merge 450 patches >> that haven't been looked at by QA or the Release Manager into master. >> These aren't new rules. > 1- we did a lot of QA on them, and although there may be some remaining > bugs, we are live with them on 1 site, very soon on a second one, and > many more. > 2- we proposed to submit those patches many months ago, but galen > decision was : "feature freeze for 3.2". We had to go ahead. > 3- we will deal with any problem that may arise. And if 3.4 has to be > released in something like 6 months, that's enough, undoubtfully > 4- splitting them in smaller feature set is a more than huge work. It's > probably almost impossible. > > So, even if your proposition is a good idea, I think it's not a possible > way to go for us. You're reaching our (BibLibre) limit. So, if someone > is volunteering for this more-than-huge job, then, fair, head to > git.biblibre.com/master, it's here, available, all commits comments are > in english. But you can't count on us to do that, even if I'm sorry, really. > > Well we will have to come up with some solution, they have not been tested with current master (like you say, they are 100 patches behind) and they will undoubtedly introduce bugs, this is no reflection on Biblibre, everyones code contains bugs, it is almost a mathematical impossibility that 450 patches will merge bug free. I will push them up into a new/biblibre-patches branch, and from there work can be done to isolate, test and merge them in managable chunks. We simply can't merge 450 patches now and not expect to make our lives much much harder in the future. I'm willing to work on making smaller feature sets, it will help a lot if the commit messages relate to bugs in bugzilla, that will make making them much easier, if not I will be asking Biblibre lots of questions. Also friendly :) Chris From cnighswonger at foundations.edu Mon Jun 21 22:04:22 2010 From: cnighswonger at foundations.edu (Chris Nighswonger) Date: Mon, 21 Jun 2010 16:04:22 -0400 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C1FC027.3010407@biblibre.com> References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> <4C1FC027.3010407@biblibre.com> Message-ID: On Mon, Jun 21, 2010 at 3:40 PM, Paul Poulain wrote: > Le 21/06/2010 20:44, Chris Cormack a ?crit : >> This is certainly a possible plan of action, but no, I won't be >> pushing any patches to master without them going through QA. >> Certainly not 450 in one go. They will need to be branched into >> smaller feature sets and each one tested and merged. What I am willing >> to do, is be less strict about accompanying tests as the patches were >> written before that, what I am not willing to do is merge 450 patches >> that haven't been looked at by QA or the Release Manager into master. >> These aren't new rules. > 1- we did a lot of QA on them, and although there may be some remaining > bugs, we are live with them on 1 site, very soon on a second one, and > many more. But this approach is not acceptable in the community at large as evidenced by the BibLibre acquisitions work introducing four blockers. What you and your clients may be able to live with, others may not be able to live with. So not only do we need good QA at the vendor level, but also at the community level as well. > 2- we proposed to submit those patches many months ago, but galen > decision was : "feature freeze for 3.2". We had to go ahead. This had to happen some time. I would suggest that the problem is not with the feature freeze, but perhaps with not keeping in sync with the main repo master. > 3- we will deal with any problem that may arise. And if 3.4 has to be > released in something like 6 months, that's enough, undoubtfully I think there is a better way. One which requires less resources on the part of one company. I cannot imagine agreeing to take on every possible problem that may arise from the merging of 400+ patches into a branch that is already 100 patches behind. I would not expect any company to take that upon themselves when we can do it in a much cleaner manner. > 4- splitting them in smaller feature set is a more than huge work. It's > probably almost impossible. I think that at a minimum the present BibLIbre work would need to be merged into a topic branch off of the stable 3.2 master and then testing/debugging be done on that topic branch while keeping it in sync with the stable master. In no case does it appear that applying 400+ patches to a stable master would be possible. *That* would make for huge work. > > So, even if your proposition is a good idea, I think it's not a possible > way to go for us. You're reaching our (BibLibre) limit. So, if someone > is volunteering for this more-than-huge job, then, fair, head to > git.biblibre.com/master, it's here, available, all commits comments are > in english. But you can't count on us to do that, even if I'm sorry, really. My only observation here would be that the burden of keeping development work in sync with the official HEAD *must* always fall primarily on the developer and not the community. This is a relatively easy task seeing we are using git version control and not svn or some other less friendly sort. Again, I think the new features are great. Its the logistics that are the fly in the ointment so to speak. Kind Regards, Chris From paul.poulain at biblibre.com Tue Jun 22 08:50:50 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Tue, 22 Jun 2010 08:50:50 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> <4C1FC027.3010407@biblibre.com> Message-ID: <4C205D4A.1060002@biblibre.com> Le 21/06/2010 21:48, Chris Cormack a ?crit : (answer written after a restfull night) > Well we will have to come up with some solution, yes, we have ! > They have not been > tested with current master (like you say, they are 100 patches behind) > and they will undoubtedly introduce bugs, this is no reflection on > Biblibre, everyones code contains bugs, it is almost a mathematical > impossibility that 450 patches will merge bug free. > I agree with you (the next question being : is it easier to fix some bugs introduced by a so big merge, or to split our work into small pieces, test them,... not sure of the answer) > I will push them up into a new/biblibre-patches branch, and from there > work can be done to isolate, test and merge them in managable chunks. > We simply can't merge 450 patches now and not expect to make our lives > much much harder in the future. I'm willing to work on making smaller > feature sets, it will help a lot if the commit messages relate to bugs > in bugzilla, that will make making them much easier, if not I will be > asking Biblibre lots of questions. > There is a "good" news I forget to speak of yesterday (it was 10PM, & i'm very busy, so very tired, those days) : Most (if not all) commits are related to a BibLibre mantis entry. Features and bugfixes. The bad news : it's all in french. The good news : 1- it should help seeing what is related to what 2- we could open our mantis repository to some of you. Note it contains a lot of private informations (about data migrations, comments,...), but we have sub-projects for every curstomer & split migration / feature devs, so we should be able to open only what needs to be open. I'll speak of that with hdl asap (not today, i'm training Aix-Marseille Universities) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 From paul.poulain at biblibre.com Tue Jun 22 08:56:35 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Tue, 22 Jun 2010 08:56:35 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C1FA595.9000700@biblibre.com> <4C1FC027.3010407@biblibre.com> Message-ID: <4C205EA3.8010306@biblibre.com> Le 21/06/2010 22:04, Chris Nighswonger a ?crit : > On Mon, Jun 21, 2010 at 3:40 PM, Paul Poulain wrote: > >> Le 21/06/2010 20:44, Chris Cormack a ?crit : >> >>> This is certainly a possible plan of action, but no, I won't be >>> pushing any patches to master without them going through QA. >>> Certainly not 450 in one go. They will need to be branched into >>> smaller feature sets and each one tested and merged. What I am willing >>> to do, is be less strict about accompanying tests as the patches were >>> written before that, what I am not willing to do is merge 450 patches >>> that haven't been looked at by QA or the Release Manager into master. >>> These aren't new rules. >>> >> 1- we did a lot of QA on them, and although there may be some remaining >> bugs, we are live with them on 1 site, very soon on a second one, and >> many more. >> > But this approach is not acceptable in the community at large as > evidenced by the BibLibre acquisitions work introducing four blockers. > I never wrote that publicly, but I must say i'm not very proud of the new acq code. it can/must be improved a lot. > What you and your clients may be able to live with, others may not be > able to live with. So not only do we need good QA at the vendor level, > but also at the community level as well. > of course. the idea being that merging very early (reminder : everything is written) means we would have 6 months to find & fix bugs. It's not the same as merging 10 days before releasing imo. >> 2- we proposed to submit those patches many months ago, but galen >> decision was : "feature freeze for 3.2". We had to go ahead. >> > This had to happen some time. > agreed. The problem being that 3.2 is very late. Today, we have to do something to merge our work. > I would suggest that the problem is not with the feature freeze, but > perhaps with not keeping in sync with the main repo master. > we tried, but had some bugs introduced by the merge + we were too short on time. Again : today, we (BibLibre) are willing to do something because in a few weeks, it'll be too late, unfortunately > > I think that at a minimum the present BibLIbre work would need to be > merged into a topic branch off of the stable 3.2 master and then > testing/debugging be done on that topic branch while keeping it in > sync with the stable master. > see answer to the other chris for the rest of your mail ;-) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 From mjr at phonecoop.coop Tue Jun 22 12:01:53 2010 From: mjr at phonecoop.coop (MJ Ray) Date: Tue, 22 Jun 2010 11:01:53 +0100 (BST) Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C205D4A.1060002@biblibre.com> Message-ID: <20100622100153.9E7A0F7159@nail.towers.org.uk> Paul Poulain wrote: > Le 21/06/2010 21:48, Chris Cormack a ?crit : > > We simply can't merge 450 patches now and not expect to make our lives > > much much harder in the future. I'm willing to work on making smaller > > feature sets, it will help a lot if the commit messages relate to bugs > > in bugzilla, that will make making them much easier, if not I will be > > asking Biblibre lots of questions. A small thing to flag up: software.coop is in a similar position. We have had a fork since LibLime pushed new features into HEAD which didn't work yet on platforms we were contracted to support. So, we had a similar fork-or-fail choice to BibLibre and reached the same unhappy decision. I've been trying to tidy+push changes and move us towards merging, but that work has slowed this year because something was up with email pull requests and I had some large urgent paid projects. It's also been complicated by conflicting changes from LibLime being submitted as part of Harley. And merging gets no easier as more time passes. I'm sure BibLibre will appreciate this: it sounds like it's hard enough for them merging a fork after some months - ours has existed almost two years or so now :-( It's a double whammy: it means we're not testing and fixing the next Koha release. If big players like BibLibre are also in a similar situation, how much of the Koha 3.2 slip has this caused? > There is a "good" news I forget to speak of yesterday (it was 10PM, & > i'm very busy, so very tired, those days) : > Most (if not all) commits are related to a BibLibre mantis entry. > Features and bugfixes. > > The bad news : it's all in french. I don't care. Submit the French. Then either us franglais speakers can translate on demand, or people can use machine translation to get the gist of it. It's far better than having a cryptic "MT" reference to something most of us can't read. > The good news : > 1- it should help seeing what is related to what > 2- we could open our mantis repository to some of you. Note it contains > a lot of private informations (about data migrations, comments,...), but > we have sub-projects for every curstomer & split migration / feature > devs, so we should be able to open only what needs to be open. Aha! So BibLibre made the same mistake we did with private information polluting our koha development work! I'm sure I remember being flamed about how simple it was to avoid or clean that. It's not. Nevertheless, it's good to see BibLibre following in our footsteps. I hope we can find some better solution than either the unhappy forker doing all the clean-up, or the confusing competing-release approach of PTFS Harley which leaves the community doing all the work as well as answering questions from users about the new Koha "release". Regards, -- MJ Ray (slef) Webmaster and LMS developer at | software www.software.coop http://mjr.towers.org.uk | .... co IMO only: see http://mjr.towers.org.uk/email.html | .... op From mjr at phonecoop.coop Tue Jun 22 12:08:20 2010 From: mjr at phonecoop.coop (MJ Ray) Date: Tue, 22 Jun 2010 11:08:20 +0100 (BST) Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C205EA3.8010306@biblibre.com> Message-ID: <20100622100820.3F050F7159@nail.towers.org.uk> Paul Poulain skribis: > Le 21/06/2010 22:04, Chris Nighswonger a ?crit : > > On Mon, Jun 21, 2010 at 3:40 PM, Paul Poulain wrote: > >> 2- we proposed to submit those patches many months ago, but galen > >> decision was : "feature freeze for 3.2". We had to go ahead. > >> > > This had to happen some time. > > > agreed. The problem being that 3.2 is very late. Today, we have to do > something to merge our work. OK, so problems in 3.0's release process caused software.coop to fork; and now problems in 3.2's release process caused BibLibre to fork. Can we do anything to stop 3.4 causing another vendor to fork? > > I would suggest that the problem is not with the feature freeze, but > > perhaps with not keeping in sync with the main repo master. > > > we tried, but had some bugs introduced by the merge + we were too short > on time. Just a side note: I think the co-op fork has most patches from master integrated, but it's still non-trivial to clean and push back. > Again : today, we (BibLibre) are willing to do something because in a > few weeks, it'll be too late, unfortunately Great, so if giving it a free pass through community QA isn't an option, what's the next best choice? Regards, -- MJ Ray (slef) Webmaster and LMS developer at | software www.software.coop http://mjr.towers.org.uk | .... co IMO only: see http://mjr.towers.org.uk/email.html | .... op From chris at bigballofwax.co.nz Tue Jun 22 12:09:18 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Tue, 22 Jun 2010 22:09:18 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <20100622100153.9E7A0F7159@nail.towers.org.uk> References: <4C205D4A.1060002@biblibre.com> <20100622100153.9E7A0F7159@nail.towers.org.uk> Message-ID: On 22 June 2010 22:01, MJ Ray wrote: > Paul Poulain wrote: >> Le 21/06/2010 21:48, Chris Cormack a ?crit : >> > We simply can't merge 450 patches now and not expect to make our lives >> > much much harder in the future. I'm willing to work on making smaller >> > feature sets, it will help a lot if the commit messages relate to bugs >> > in bugzilla, that will make making them much easier, if not I will be >> > asking Biblibre lots of questions. > > A small thing to flag up: software.coop is in a similar position. ?We > have had a fork since LibLime pushed new features into HEAD which > didn't work yet on platforms we were contracted to support. ?So, we > had a similar fork-or-fail choice to BibLibre and reached the same > unhappy decision. > > I've been trying to tidy+push changes and move us towards merging, but > that work has slowed this year because something was up with email > pull requests and I had some large urgent paid projects. ?It's also > been complicated by conflicting changes from LibLime being submitted > as part of Harley. ?And merging gets no easier as more time passes. > I'm sure BibLibre will appreciate this: it sounds like it's hard > enough for them merging a fork after some months - ours has existed > almost two years or so now :-( > > It's a double whammy: it means we're not testing and fixing the next > Koha release. ?If big players like BibLibre are also in a similar > situation, how much of the Koha 3.2 slip has this caused? > >> There is a "good" news I forget to speak of yesterday (it was 10PM, & >> i'm very busy, so very tired, those days) : >> Most (if not all) commits are related to a BibLibre mantis entry. >> Features and bugfixes. >> >> The bad news : it's all in french. > > I don't care. ?Submit the French. ?Then either us franglais speakers > can translate on demand, or people can use machine translation to get > the gist of it. ?It's far better than having a cryptic "MT" reference > to something most of us can't read. > >> The good news : >> 1- it should help seeing what is related to what >> 2- we could open our mantis repository to some of you. Note it contains >> a lot of private informations (about data migrations, comments,...), but >> we have sub-projects for every curstomer & split migration / feature >> devs, so we should be able to open only what needs to be open. > > Aha! ?So BibLibre made the same mistake we did with private > information polluting our koha development work! ?I'm sure I remember > being flamed about how simple it was to avoid or clean that. ?It's not. > Mantis is their Bug tracker, so I dont think Paul was saying the private data is in the code, more that it is in their bug tracker. Which seems fine. But just in case they do mean in git. Here is a good tutorial on removing sensitive data. http://help.github.com/removing-sensitive-data/ It isn't actually that bad (no flame intended :)). > Nevertheless, it's good to see BibLibre following in our footsteps. > I hope we can find some better solution than either the unhappy > forker doing all the clean-up, or the confusing competing-release > approach of PTFS Harley which leaves the community doing all the work > as well as answering questions from users about the new Koha "release". > I don't think there is a need for a fork in this case, once the patches are rebased on master, and pushed into their own branch, we can divide them up, test and merge. But in the meantime once that branch is based on master, and rebased regularly, it's very simple for people to merge that into a branch in their own repository, until its all in master. Chris From paul.poulain at biblibre.com Tue Jun 22 12:18:18 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Tue, 22 Jun 2010 12:18:18 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C205D4A.1060002@biblibre.com> <20100622100153.9E7A0F7159@nail.towers.org.uk> Message-ID: <4C208DEA.2090107@biblibre.com> Le 22/06/2010 12:09, Chris Cormack a ?crit : >> Aha! So BibLibre made the same mistake we did with private >> information polluting our koha development work! I'm sure I remember >> being flamed about how simple it was to avoid or clean that. It's not. >> >> > Mantis is their Bug tracker, so I dont think Paul was saying the > private data is in the code, more that it is in their bug tracker. > Which seems fine. > > But just in case they do mean in git. Here is a good tutorial on > removing sensitive data. > http://help.github.com/removing-sensitive-data/ > It isn't actually that bad (no flame intended :)). > > chris is right : it's just our mantis that has some sensitive information. git is clean (as it is and has always been 100% available on git.biblibre.com) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 From colin.campbell at ptfs-europe.com Tue Jun 22 15:33:17 2010 From: colin.campbell at ptfs-europe.com (Colin Campbell) Date: Tue, 22 Jun 2010 14:33:17 +0100 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C060F5C.9010402@biblibre.com> References: <4C060F5C.9010402@biblibre.com> Message-ID: <4C20BB9D.8030604@ptfs-europe.com> As everyone has pointed out 400+ is a lot and liable to generate a bout of indigestion, but as Paul pointed out in his original email we need to up our game when it comes to managing the product of all that work. Small improvements could make life easier for all. If the patch introduces a feature or changes how one behaves is it documented? The pod should answer the puzzled why of anyone reading the code. Are there tests? (And if so does it pass them) We really need to take testing much more seriously, coverage of the code is minimal, (and we still don't have a code base that runs cleanly with use warnings!!). There are some trivial things you could do, before submitting the patch make sure that you're not adding any meaningless whitespace at the end of lines ( git diff shows it) it helps keep future diffs, merges etc less labour-intensive. Does your code show its intention clearly. Trivial points like variable names (data and results don't convey much more than using random strings of hex digits), don't be obscure (changing e.g. hldct to hold_count removes ambiguity especially to a fellow programmer who may not be a native speaker of your dialect of English). Does the code need breaking up into subroutines? (Programmers have very short attention spans - about a screen and a bit and even shorter in if else blocks). Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell at ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com From reed at catalyst.net.nz Tue Jun 22 23:27:15 2010 From: reed at catalyst.net.nz (Reed Wade) Date: Wed, 23 Jun 2010 09:27:15 +1200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: <4C20BB9D.8030604@ptfs-europe.com> References: <4C060F5C.9010402@biblibre.com> <4C20BB9D.8030604@ptfs-europe.com> Message-ID: > 400 patches Chris and I needed to do some checking before making a public commitment but-- Catalyst will be able supply a couple of weeks of someone from our QA team to help resolve these. -reed From paul.poulain at biblibre.com Wed Jun 23 10:36:07 2010 From: paul.poulain at biblibre.com (Paul Poulain) Date: Wed, 23 Jun 2010 10:36:07 +0200 Subject: [Koha-devel] BibLibre strategy for Koha 3.4 and next versions In-Reply-To: References: <4C060F5C.9010402@biblibre.com> <4C20BB9D.8030604@ptfs-europe.com> Message-ID: <4C21C777.2010103@biblibre.com> Le 22/06/2010 23:27, Reed Wade a ?crit : >> 400 patches >> > Chris and I needed to do some checking before making a public commitment but-- > > Catalyst will be able supply a couple of weeks of someone from our QA > team to help resolve these. > That would be awesome ! you can count on us to work closely with you on this matter ! Hurray for catalyst ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 From gmcharlt at gmail.com Fri Jun 25 03:34:27 2010 From: gmcharlt at gmail.com (Galen Charlton) Date: Thu, 24 Jun 2010 21:34:27 -0400 Subject: [Koha-devel] Koha 3.2 beta released Message-ID: Hi, I am pleased to announce the release of the beta version of 3.2. The tarball can be downloaded from: http://download.koha-community.org/koha-3.02.00-beta.tar.gz Checksums and signatures to verify the download can be retrieved from: http://download.koha-community.org/koha-3.02.00-beta.tar.gz.sig http://download.koha-community.org/koha-3.02.00-beta.tar.gz.MD5.asc http://download.koha-community.org/koha-3.02.00-beta.tar.gz.MD5 Bugfixes since 3.2 alpha2 include: bug 4312 change default for tagsmoderation to 0 from NULL (BUG #4857) aqplan.pl: consideration of Planning categories with authorised values MT3652 : Unifying the search of neworder with Search (BUG #4811) suggestion.pl: display borrowers name in suggestion information filters (Suggested By, Managed by, Accepted by) (BUG #4810) parcel.pl: Fix a bug with applying a filter on pending orders displaying (bug #4523) possibility to show / hide the filters menu Bug 3217 Impossible to change biblio record FW to Default (MT2371) basket.tmpl, in basket details, change Open on => Opened on (BUG #4356) Basket.pl: adding a link to the basketgroup (BUG #4521) aqbudgets.pl - Transform undefined budget spent value to 0.00 value Bug 4905 Runtime errors in about.pl fix two broken calls to _FixPriority bug 3344: display in-transit status on holds priority list bug 4224: explicitly mark hold requests as being handled by item in transit Fix for Bug 4532, Use include file for bibliodefaultview logic Fix for Bug 4821, With multiple 5XX fields, the font display gets progressively smaller Bug 4423, Staff Client XSLT is just a copy of the OPAC one Followup fix for Bug 4453: Removing invalid template fragment. Fix for Bug 4278, canceling vendor add refreshes wrong Fix for Bug 3895 - Menu on left of Contracts is for Admin Pages Fix for Bug 4886, missing 404 redirection on wrong biblionumber for MARC and ISBD details Fix for Bug 4884, opac-showmarc.pl can't find compact.xsl Fix for Bug 4859, Formatting cleanup for merge biblio record interface Fix for Bug 4869 - Non-staff patrons logging into the OPAC don't have option to place holds from Public Lists Bug3916 : Batch Modify tool overwrites branches fields Bug 4474 swap options for sys pref singleBranchMode Bug 4902: Add missing b_phone and b_email to borrower details bug 4900: add in missing value for usedaysmode pref Bug 4897: Invalid XHTML in opac-userupdate.tmpl. fixes to xt/sysprefs.t test case fixed permissions for item batch deletion and modification for French fixed problems in xt/permissions.t test case Bug 4895: Add missing granular permissions Bug 3682: change message_attributes.message_name from varchar(20) to varchar(40) Fix for Bug 4868 - Improve controls for cloning and deleting MARC subfields Page structure correction, helps Bug 3850 (Page Need Design Work) Fixes bug 3619: _send_message_by_email not respecting AutoEmailPrimaryAddress = 'OFF' BUG 4883: Staff - remove \n from strings for translation Bug 4833: OPAC - remove \n from strings for translation Bug 4895: Add missing granular permissions to ru-RU, uk-UA Bug 4895: Add missing granular permissions to de-DE, pl-PL But 4890: Fixes invalid XHTML in basket.tmpl Bug 4889: Fixes minor XHTML error in booksellers.tmpl. Bug 4827: Fixes library drop-down list in aqbudgets.tmpl bug 4896: granular permissions now always on (DB rev 138) MT 1816: Granular permissions for the serials module Bug 4472 - Missing / in img tags breaking xslt (and other img tags) Bug 2789 Fix UNIMAC leader plugin (bug #4853) change rights needed to renew loans Bug #4864 Add SIPServer Perl Module Dependencies MT2938 : Adds a permission for editing items Fix for Bug 3081, Url's contain spaces Fix for Bug 3770, 'Add to list' page only allows adding to private lists Fix for Bug 3722 - Branch deletion results in incorrect message Fix for Bug 3926, inadvertantly resurrected by me. Fix for Bug 3992 - New Sys Prefs Branch - Local Use Tab not Highlighted bug 4205 remove extra 'plan by' option Bug 4844 Remove a circular dependency in koha_perl_deps.pl Fix for Bug 4504, Confirmation messages in opac account not translated bug 4845 change language from reserves to holds German web installer files, including translation of MARC21 frameworks bug 4445: Upping the daily limit for XISBN bug 4445 update OCLC text tips on preferences bug 4834 split joined preferences bug 4027 fix testStatus typo to textStatus (MT #2565) fix aqplan csv export, and turn off debug bug 4508: fix crash when editing patron attributes or message prefs bug 4816: require authentication for placerequest.pl bug 4386: fix email hold filled notifications removed needless imports of the YAML module Bug 4199: Adds ability to print routing slip. Patch 2. Bug 4199: Adds ability to print routing slip from serials-collection. Bug 4805: Fixes multiple subscriptionid's being passed to serials-collection. bug 4377: ensure HOLD_PRINT letter added during installation and upgrade (DBrev 135) bug 4311: respect OPACXSLTResultsDisplay bug 4018: remove duplicate unAPI link when XSLT bib details display on (bug #4487) permit - and . in callnumber plugin Correct position of non-sorted table column Corrected: Fix for Bug 4529, columns on circ history all screwy (BUG #4521) aqbudgets.pl - Transform undefined budget spent value to 0.00 value Bug 4510 Script processes single supplier not an array (bug #4522) fix plugin unimarc 210$c Bug 4517 - add authentication to reorder_members.pl update sample news on new install with koha-community.org urls Fix for Bug 4534, Box for SQL not visible in "Create Report from SQL" moved INSTALL.debian-lenny to INSTALL.debian bug 4464: properly check if a subfield is populated if it is in a textarea Bug 2889: Adds highlighting to serials-collection.tmpl Bug 3910: Removes an erroneous heading from the serials search results. bug 4505: updated release notes Bug 4505 - Bumping the perl version to 5.8.8 or greater bug 4802 - fix swapped files in acq help bug 4802 add missing order search help file bug 4801: fix paging in display of staged bibs and import batches bug 4802 add new acq help files update main page help file fix XHTML error in bib details print page (bug #4519) fix record printing Bug 4525: Invalid XHTML in currency.tmpl. bug 4151: fix migration_tools typo Bug 4516 (Character shift in MARC21 Field 008) Backing down the required version of Graphics::Magick Enhancement Bug 4444: Centralize Code Handling Perl Dependencies bug 4509: remove references to PINESISBN system preference Bug 4507 Don't return null vendor to claims processing Bug 3768 (Bibliographic Framework Test) fix XHTML error in about.tmpl BUG 4499: Javascript error messages not translatable Bug 4141 Reconcile 3.0.x and HEAD database updates for 3.2.0 Correct URL for Baker & Taylor ContentCafe (MT3318) RSS OPAC: Adding CDATA term in title and description tags. Fix for Bug 4485, System preferences page uses non-standard warning style A couple of CSS and markup corrections Bug 4300 Display 866z summary holdings public note in OPAC stocknumber not saved properly MT2582: Fix user deletion without permission wr-70205 Rental discount not being respected Bug 3093 add syspref to turn off multiholds button Bug 3928: Modified date should follow syspref Bug 3928: Modification of date for serials. Display current selected category when editing lists. Fix for Bug 4475, AmazonLocale not used consistently Markup corrections for validity and to eliminate horizontal scrollbar Fix for Bug 4400, BIBTEX export from OPAC results in empty file Bug 4470 Patron search result pagination bar Do not declare variables within (bogus) conditionals Reserve and onloan were reporting the same cardnumber, due to a little bug in detail.pl fixed by this patch Fix for Bug 3666, Overriding renewal limit means negative count in OPAC Bug 3671 Workaround for font display problem bug 4403: look in appropriate theme/lang for bib display XSL quell 'non-initialized variable defaulttab' warning fix malformed call of XSLTParse4Display Further corrections for Bug 4244, Use "checkouts" instead of "issues" Style changes to cart popup window more compact display: - compact holdings list - adds list bullet image from the staff client (MT #2966) fix opac-user.pl function problem Bug 4465 Fixes translated boolean syspref inversion exclude TinyMCE from non-UTF8 file checks move misc source file test and fix scripts to xt/ Further fixes for Bug 4208, Many submit buttons are not translatable in 3.2 fixed Syndetics breakage Fixes bug 4448: &'s in itemcallnumber Fix for Bug 4416, renew all and return all buttons too close together Fix for Bug 4453, Improve formatting of batch item operations Fix for Bug 2375, Serials holdings data does not display in opac-detail Fixes bug 4452: CircControl syspref listed as ItemHomeBranch rather than ItemHomeLibrayr Changes to kohastructure.sql to add some missing drop if exists + change in install.pl mysql show tables Web Installer -> Step 3 --> ERROR 1050 (42S01) at line 301: Table 'branch_item_rules' already exists fix template translation issues Fixed Patron Search Results Use CSS3 box-shadow property for Cart tooltip In addition, various improvements to the Debian packaging were made. At this point, I am declaring a string freeze with the following exceptions: * forward ports from 3.0.x (for which translated strings can presumably be grabbed from the 3.0.x translations) * remaining 3.2 release blockers. Regards, Galen -- Galen Charlton gmcharlt at gmail.com From nengard at gmail.com Mon Jun 28 15:27:40 2010 From: nengard at gmail.com (Nicole Engard) Date: Mon, 28 Jun 2010 09:27:40 -0400 Subject: [Koha-devel] Koha Newsletter: Call for Articles Message-ID: It's that time again! The newsletter will be released on the 15th of next month and that means I need some submissions from users and developers alike! Articles can be as short as 2 sentences - so send along any news/tips/guides/ideas you want to share with the community. Thanks Nicole From psm_vu at india.com Wed Jun 30 14:43:40 2010 From: psm_vu at india.com (Partha Mukhopadhyay) Date: Wed, 30 Jun 2010 20:43:40 +0800 Subject: [Koha-devel] Graphics::Magick 1.3.5 Message-ID: <20100630124340.7EBCE44B6C@ws5-1.us4.outblaze.com> Dear all I have attempted an upgrade from Koha 3.06 to Koha 3.2 Beta (early bird!!! ha ha). Got stuck in the middle. CPAN failed to found Graphics::Magick 1.3.5. The package is available although. Should I try apt-get? Dr. Parthasarathi Mukhopadhyay Lecturer (Sr.), Department of Library and Information Science University of Burdwan, Burdwan, West Bengal, India -- _______________________________________________ Search for products and services at: http://search.mail.com From kmkale at anantcorp.com Wed Jun 30 14:50:17 2010 From: kmkale at anantcorp.com (Koustubha Kale) Date: Wed, 30 Jun 2010 18:20:17 +0530 Subject: [Koha-devel] Graphics::Magick 1.3.5 In-Reply-To: <20100630124340.7EBCE44B6C@ws5-1.us4.outblaze.com> References: <20100630124340.7EBCE44B6C@ws5-1.us4.outblaze.com> Message-ID: On Wed, Jun 30, 2010 at 6:13 PM, Partha Mukhopadhyay wrote: > Dear all > > I have attempted an upgrade from Koha 3.06 to Koha 3.2 Beta (early bird!!! > ha ha). Got stuck in the middle. CPAN failed to found Graphics::Magick > 1.3.5. The package is available although. Should I try apt-get? > > > Dr. Parthasarathi Mukhopadhyay > Please see http://lists.katipo.co.nz/public/koha/2010-March/022977.html Regards, Koustubha Kale Anant Corporation Contact Details : Address : 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), Maharashtra, India, Pin : 400601. TeleFax : +91-22-21720108, +91-22-21720109 Mobile : +919820715876 Website : http://www.anantcorp.com Blog : http://www.anantcorp.com/blog/?author=2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From psm_vu at india.com Wed Jun 30 15:00:25 2010 From: psm_vu at india.com (Partha Mukhopadhyay) Date: Wed, 30 Jun 2010 21:00:25 +0800 Subject: [Koha-devel] Graphics::Magick 1.3.5 Message-ID: <20100630130025.CB35E44B6C@ws5-1.us4.outblaze.com> Dear Mr. Koustubha Thanks for a quick response. Please see the result..... ----------------------------------------------------------------------------------------------------- psm at libra25:~$ sudo apt-get install libimage-magick-perl [sudo] password for psm: Reading package lists... Done Building dependency tree?????? Reading state information... Done Package libimage-magick-perl is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package libimage-magick-perl has no installation candidate psm at libra25:~$ ---------------------------------------------------------------------------- ----- Original Message ----- From: "Koustubha Kale" To: "Partha Mukhopadhyay" Cc: koha-devel at lists.koha-community.org Subject: Re: [Koha-devel] Graphics::Magick 1.3.5 Date: Wed, 30 Jun 2010 18:20:17 +0530 On Wed, Jun 30, 2010 at 6:13 PM, Partha Mukhopadhyay wrote: Dear all I have attempted an upgrade from Koha 3.06 to Koha 3.2 Beta (early bird!!! ha ha). Got stuck in the middle. CPAN failed to found Graphics::Magick 1.3.5. The package is available although. Should I try apt-get? Dr. Parthasarathi Mukhopadhyay Please see http://lists.katipo.co.nz/public/koha/2010-March/022977.html ? Regards, Koustubha Kale Anant Corporation Contact Details : Address ?: 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), ? ? ? ? Maharashtra, India, Pin : 400601. TeleFax ?: +91-22-21720108, +91-22-21720109 Mobile ? ? : +919820715876 Website ?: http://www.anantcorp.com Blog : http://www.anantcorp.com/blog/?author=2 Parthasarathi Mukhopadhyay Department of Library and Information Science University of Burdwan, Burdwan, West Bengal, India -- _______________________________________________ Search for products and services at: http://search.mail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From kmkale at anantcorp.com Wed Jun 30 15:06:36 2010 From: kmkale at anantcorp.com (Koustubha Kale) Date: Wed, 30 Jun 2010 18:36:36 +0530 Subject: [Koha-devel] Graphics::Magick 1.3.5 In-Reply-To: <20100630130025.CB35E44B6C@ws5-1.us4.outblaze.com> References: <20100630130025.CB35E44B6C@ws5-1.us4.outblaze.com> Message-ID: On Wed, Jun 30, 2010 at 6:30 PM, Partha Mukhopadhyay wrote: > > Dear Mr. Koustubha > > Thanks for a quick response. Please see the result..... > > > ----------------------------------------------------------------------------------------------------- > psm at libra25:~$ sudo apt-get install libimage-magick-perl > [sudo] password for psm: > Reading package lists... Done > Building dependency tree > Reading state information... Done > Package libimage-magick-perl is not available, but is referred to by > another package. > This may mean that the package is missing, has been obsoleted, or > is only available from another source > E: Package libimage-magick-perl has no installation candidate > psm at libra25:~$ > > ---------------------------------------------------------------------------- > > > OS? Version? Regards, Koustubha Kale Anant Corporation Contact Details : Address : 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), Maharashtra, India, Pin : 400601. TeleFax : +91-22-21720108, +91-22-21720109 Mobile : +919820715876 Website : http://www.anantcorp.com Blog : http://www.anantcorp.com/blog/?author=2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From cnighswonger at foundations.edu Wed Jun 30 15:21:46 2010 From: cnighswonger at foundations.edu (Chris Nighswonger) Date: Wed, 30 Jun 2010 09:21:46 -0400 Subject: [Koha-devel] Graphics::Magick 1.3.5 In-Reply-To: <20100630130025.CB35E44B6C@ws5-1.us4.outblaze.com> References: <20100630130025.CB35E44B6C@ws5-1.us4.outblaze.com> Message-ID: 2010/6/30 Partha Mukhopadhyay > > Dear Mr. Koustubha > > Thanks for a quick response. Please see the result..... > > > ----------------------------------------------------------------------------------------------------- > psm at libra25:~$ sudo apt-get install libimage-magick-perl > [sudo] password for psm: > Reading package lists... Done > Building dependency tree > Reading state information... Done > Package libimage-magick-perl is not available, but is referred to by > another package. > This may mean that the package is missing, has been obsoleted, or > is only available from another source > E: Package libimage-magick-perl has no installation candidate > psm at libra25:~$ > > ---------------------------------------------------------------------------- > > As mentioned the mailing list archive, the proper package name is libgraphics-magick-perl and not libimage-magick-perl. $ sudo apt-cache search libgraphics-magick-perl libgraphics-magick-perl - format-independent image processing - perl interface Kind Regards, Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From psm_vu at india.com Wed Jun 30 15:39:13 2010 From: psm_vu at india.com (Partha Mukhopadhyay) Date: Wed, 30 Jun 2010 21:39:13 +0800 Subject: [Koha-devel] Graphics::Magick 1.3.5 Message-ID: <20100630133913.DD01644B6C@ws5-1.us4.outblaze.com> Dear Chris I downloaded the package and it goes on this way......... -------------------------------------------------------------------------------------------------- psm at libra25:~/Desktop/koha_sw$ sudo dpkg -i libgraphics-magick-perl_1.3.5-6_i386.deb ----------------------------------------------------------------------------------- Selecting previously deselected package libgraphics-magick-perl. (Reading database ... 98452 files and directories currently installed.) Unpacking libgraphics-magick-perl (from libgraphics-magick-perl_1.3.5-6_i386.deb) ... dpkg: dependency problems prevent configuration of libgraphics-magick-perl: ?libgraphics-magick-perl depends on libgraphicsmagick3 (>= 1.3.5); however: ? Package libgraphicsmagick3 is not installed. ?libgraphics-magick-perl depends on libwmf0.2-7 (>= 0.2.8.4); however: ? Package libwmf0.2-7 is not installed. dpkg: error processing libgraphics-magick-perl (--install): ?dependency problems - leaving unconfigured Processing triggers for man-db ... Errors were encountered while processing: ?libgraphics-magick-perl -------------------------------------------------------------------------------------------------------------- psm at libra25:~/Desktop/koha_sw$ sudo apt-get install libwmf0.2-7 --------------------------------------------------------------------------------------------------------- Reading package lists... Done Building dependency tree?????? Reading state information... Done Package libwmf0.2-7 is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package libwmf0.2-7 has no installation candidate -=---------------------------------------------------------------------------------------------------------------------- psm at libra25:~/Desktop/koha_sw$ sudo apt-get install libgraphicsmagick3 --------------------------------------------------------------------------------------------------------------------------- Reading package lists... Done Building dependency tree?????? Reading state information... Done Package libgraphicsmagick3 is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package libgraphicsmagick3 has no installation candidate psm at libra25:~/Desktop/koha_sw$ sudo apt-get install libgraphicsmagick Reading package lists... Done Building dependency tree?????? Reading state information... Done E: Couldn't find package libgraphicsmagick psm at libra25:~/Desktop/koha_sw$ sudo apt-get install libgraphicsmagick3 Reading package lists... Done Building dependency tree?????? Reading state information... Done Package libgraphicsmagick3 is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package libgraphicsmagick3 has no installation candidate psm at libra25:~/Desktop/koha_sw$ partha As mentioned the mailing list archive, the proper package name is libgraphics-magick-perl and not libimage-magick-perl. $ sudo apt-cache search libgraphics-magick-perl libgraphics-magick-perl - format-independent image processing - perl interface Kind Regards, Chris Parthasarathi Mukhopadhyay Department of Library and Information Science University of Burdwan, Burdwan, West Bengal, India -- _______________________________________________ Search for products and services at: http://search.mail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Wed Jun 30 15:44:40 2010 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 1 Jul 2010 01:44:40 +1200 Subject: [Koha-devel] Graphics::Magick 1.3.5 In-Reply-To: <20100630133913.DD01644B6C@ws5-1.us4.outblaze.com> References: <20100630133913.DD01644B6C@ws5-1.us4.outblaze.com> Message-ID: 2010/7/1 Partha Mukhopadhyay > > Dear Chris > > I downloaded the package and it goes on this way......... > Why download it? why not simply sudo apt-get install libgraphics-magick-perl What versiion of debian/ubuntu are you running? Chris From mdhafen at tech.washk12.org Wed Jun 30 17:01:11 2010 From: mdhafen at tech.washk12.org (Michael Hafen) Date: Wed, 30 Jun 2010 09:01:11 -0600 Subject: [Koha-devel] Graphics::Magick 1.3.5 In-Reply-To: References: <20100630133913.DD01644B6C@ws5-1.us4.outblaze.com> Message-ID: <1277910071.2361.0.camel@koha-dev> This is in the Universe repository on Ubuntu (Lucid). If the Universe repository isn't enabled you may not be able to install this package with it's dependencies through apt. On Thu, 2010-07-01 at 01:44 +1200, Chris Cormack wrote: > 2010/7/1 Partha Mukhopadhyay > > > > Dear Chris > > > > I downloaded the package and it goes on this way......... > > > > Why download it? why not simply > sudo apt-get install libgraphics-magick-perl > > What versiion of debian/ubuntu are you running? > Chris > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel -- Michael Hafen Systems Analyst and Programmer Washington County School District Utah, USA for Koha checkout http://development.washk12.org/gitweb/ or git://development.washk12.org/koha