February 2012 - Koha-devel - lists.koha-community.org

[Discussion tech] Generalise the use Template Toolkit plugins
by Paul Poulain 21 Feb '12

21 Feb '12

I've started another discussion page: http://wiki.koha-community.org/wiki/Generalise_the_use_Template_Toolkit_plu… Chris_c recently has added a plugin to manage the display of dates into Koha (see http://wiki.koha-community.org/wiki/Coding_Guidelines#Displaying_dates) that is very interesting. Do we want to generalise this mechanism ? When do we want to use it ? For example, the patch for bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4078 could nicely use this mechanism let's start the discussion ! (reading the page and the possible solutions, you'll see I made 2, but am not happy with them. Probably need others ideas...) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

3 3

[Discussion Tech] Cache handling in koha
by Tomas Cohen Arazi 21 Feb '12

21 Feb '12

Hi, I'm just starting a thread to discuss this topic so we make some decisions and we can continue improving Koha in this aspect. As far as I can remember there are a few issues with this: 1) The use of ENV vs. koha-conf.xml vs. DB/systempreferences to store memcached configuration. I introduced the use of ENV (from Apache vhost definition) for storing memcached servers configuration data on Bug 6193. This was done for being able to cache the koha-conf.xml configuration variables (prior to that, memcached info was saved in koha-conf.xml itself and thus not-cacheable). When coding that I realized that memcached was being initialized everywhere we needed it. And namespaces where not always consistent (KOHA and koha alternatively, this might not be a problem if this happens to be case insensitive, anyway, it looked bad to me). I then exported the object memcached and the ismemcached variable in C4::Context. This first patch (koha-conf.xml in memcache, and memcached object available for every other lib that includes C4::Context) has been pushed. Then there's a second patch, that removes several memcached configuration validation and initialization code from the rest of the libs, and reuses that variables from C4::Context. I tested it to work flawlessly, but things around it halted when other proposals raised. I like mi approach. I like how the resulting code looked, I wont hide it. But I don't need it to be accepted at all, maybe the other proposals are far better. The only thing I missed on this cache thing was more feedback. Back to the ENV use I think we must make a desicion: - Do we want to cache koha-conf.xml ? If the answer is no, then we can revert that patch. And we are done. If the answer is yes, we can deal with it is storing the settings in the database, but this will make it difficult to memoize the function reading. Or leave it as it is now. I don't agree it is a problem to move those settings to the vhost definition, it could be even moved into a separate file with an include sentence in the vhost definition for convenience (/etc/koha/memcached.conf and Include in vhost definition...) 2) Memoization in Koha Do we plan to use memoization in the mid/long-term? Or are we thinking of a more abstract set of 'cache' utilities which provide an interface to memcached? Using memoization is nice, clean and simple. It has the disadvantage that caching is done in an implicit way, and there are hidden problems (read side-effects) that have to be taken into account. For example, someone writes a method, other memoizes it, but doesn't know that a third method relies on data returned by the first mehotd being 'online'. As-is, memoize_memcached doesn't provide a proper way of invalidating data automatically. As said in a previous email: we will end up hooking all the data writing methods so they invalidate all the memcached store, or those methods the writer knows have to do with the current. Almost like making it explicit, but just for writers. In which case it is better to implement the second option, making everything explicit: The second approach means we have to hook every function to handle its caching, and cache invalidation. Its advantage is that being explicit, but requieres a lot of work and we will end up having business logic mixed with cache handling code, which is not a problem per-se. Just thinking. If we are planning to go on with memoization, I made a proposal on IRC (which had zero responses) to add memoize_memcached an array of references to data invalidation methods, so we don't have to hook every method. The Memoize::Memcached dev told me to take over the lib as he isn't maintaining it nor working on perl anymore. I'd like to do that, but I halted my work on that until it is more clear that it is not a waste of time. Ok. I'd like to re-read this words I wrote, but I'm just back from holidays and my bosses need a lot of answers for a lot of stuff I didn't finish last year, right now, so... Regards To+

6 11

Improving permissions on lists (virtual shelves)
by Marcel de Rooy 20 Feb '12

20 Feb '12

Hi all, Following up on the discussion on list permissions in December, I updated the wiki page http://wiki.koha-community.org/wiki/List_permissions and submitted the first patches for report 7310 (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7310) These first patches form the foundation for further changes but still stay very close to current functionality. One new point (not discussed before): While developing and refactoring some code, I removed the increasing lists information from session storage. This simplified the code, makes the sessions much smaller, ensures actual list information and did not make a big difference in performance (depending on factors as session count, number of lists, etc.; could even be faster). Next step in the development is now test and sign-off. Anyone willing is welcome to test! A detailed test plan is included on the report and on the wiki. Hope to hear from you! Marcel

1 0

Call for News for the February Newsletter
by Daniel Grobani 16 Feb '12

16 Feb '12

Dear Koha Kommunitarians, I'm harvesting news for the February newsletter. Please send me by the 25th anything you think your fellow community members might like to know about. "News" can be as short as a sentence or as long as a paper. I especially encourage you to send me a line or two about what you're currently working on for publication in the gossip/society column. And if you know of a go-live not announced on the list, please be sure to let me know about it. Thanks, Daniel Grobani

1 0

Re: [Koha-devel] link_bibs_to_authorities.pl can corrupt records
by Paul 16 Feb '12

16 Feb '12

Chris, Many thanks - picking up on you points below: At 09:57 AM 1/7/2012 +1300, Chris Cormack wrote: >I have successfully use that script to link authorities to biblio records >after doing a migration. But the way it works is to remove all links and >recreate. Not sure about the "recreate" bit... >Course I didn't do this on a server people were using, which it sounds >like you are, to have lost 100 biblios. "People" are using it. After nearly a year of using Koha 3.2, 3.6.1 is up and running (kudos to the development team, average LAN response time is down from 4.7 secs to well < 1 sec.) I asked for assistance with zebra for authorities, and the assistance from the community allowed me to resolve the difficulties. Then I asked our volunteers to start cataloguing on 3.6.1 to sort out any remaining "update" glitches -- hence the ~100 biblios in the last 24 hours (if you want the nitty-gritty on why we've got to do so many per day, I'll oblige off-list) >Its included for that use case. Jared is working on a much better linking >script and he gave you the information how to change it. Respectfully, it's included in the "current stable release" with no warnings (3.6.1 was "current" when I installed it -- I haven't verified 3.6.2.) and Jared's information came after "horses and barn doors." >Its not a total screwup script, and it didn't demolish the whole db, I >think you would find people more willing to help you, in their own free >time, in their weekends if you didn't come across so aggressive and >accusatory in your tone. I humbly apologize to anyone who finds me "aggressive and accusatory" -- it's probably the way I was taught to write (concisely, briefly, etc) rather than my feelings. Most people who know me well find me to be a cuddly rather than a grizzly bear. Again my apologies. Best - Paul >Chris >On 7 Jan 2012 09:48, "Archives and Collections Society" ><<mailto:info@aandc.org>info(a)aandc.org> wrote: >At 03:02 PM 1/6/2012 -0500, Ian Walls wrote: >>Paul, >>The <http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl >>script is hardcoded to ALWAYS erase manual links.ÃÂ I'm not sure what >>the reasoning is behind that.ÃÂ But, every time you run it, any link >>you've done manually that the script can't automatically figure out will >>be removed.ÃÂ Nature of the beast (for now, at least). > >What is "any link you've done manually"?, please.Â More than 90% of our >input is Z39.50 and "default" item cataloguing. > >OK - a total screw-up that is included without any warning whatsoever in >the standard distribution of 3.6.1 ???Â From the bug report ><<http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5683>http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5683> > >Chris Cormack 2011-06-01 21:13:23 UTC >Pushed to master > >Jared Camins-Esakov 2011-07-08 14:37:02 UTC >Seems to be working now. Closing. > >If it's "nature of the beast" why isn't the "beast" mentioned >somewhere?Â What on earth is the reason for this .pl to be included in >any distribution? > >More... >>2012/1/6 Chris Cormack >><<mailto:chris@bigballofwax.co.nz>chris(a)bigballofwax.co.nz> >>Im assuming you did this on your testing/staging server right? >>Id restore that from backup, then try applying the patch and see if you >>get better behaviour. >>If so, then do it on your production machine. > >what "patch"??? > >And yes -- I can restore from a previous sqldump, and only lose +/- 100 >biblios.Â Â We're in pre-production and when we got the zebra indexing >working properly, I told our volunteers to start inputting to 3.6.1 *only* >(not duplicate on 3.2) so that I could look into any difficulties they >experienced as cataloguers. > >I had no way at all of knowing that a pl script included in the "latest >stable distribution" could possibly demolish the whole db. > >Again, what "patch" and where do I find it? > >Might I suggest that someone re-opens the bug (I have attempted to >annotate it) and removes this dangerous script from public distribution >(or at least includes a very visible warning.) > >Best - Paul > >>Chris >>On 7 Jan 2012 08:47, "Paul" <<mailto:paul.a@aandc.org>paul.a(a)aandc.org> >>wrote: >>Bug 5683 (<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl >>can corrupt records) was signed off some months ago, but is perhaps a >>little too cryptic for me to follow in detail. Ã Can someone help? As >>far as I can see, without remedial action, this is catastrophic. >>Following difficulties we have had migrating 3.2 to 3.6.1, someone on the >>users mailing list suggested we should run >><http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl -- (db >>has ~10k authorities and ~15k biblios) -- so I did just that, and have >>APPARENTLY DEMOLISHED *EVERY* LINK IN THE DB. Ã From my notes: >>paul@nelson:/usr/share/koha$ >>./bin/<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl >>Bib authority heading linking report >>------------------------------------ >>Number of bibs checked: ÃÂ ÃÂ ÃÂ 14911 >>Number of bibs modified: ÃÂ ÃÂ Ã 14886 >>Number of bibs with errors: ÃÂ 0 >>but - the staff interface no longer correctly finds authorities !!! >>[1] Manually link (staff client) auth "Lubbock" to bib "Arctic Whalers" >>-- now linked, 35 other Lubbock bibs not linked. Ã Is this a question of >>zebra re-indexing? >>So: Ã KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib >>./bin/migration_tools/<http://rebuild_zebra.pl>rebuild_zebra.pl -a -r -v >>and Ã KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib >>./bin/migration_tools/<http://rebuild_zebra.pl>rebuild_zebra.pl -b -r -v -x >>koha@nelson:/usr/share/koha$ >>./bin/<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl >>... processed 100 records >>/ ... / >>... processed 14900 records >>Bib authority heading linking report >>------------------------------------ >>Number of bibs checked: ÃÂ ÃÂ ÃÂ 14911 >>Number of bibs modified: ÃÂ ÃÂ Ã 1 >>Number of bibs with errors: ÃÂ 0 >>koha@nelson:/usr/share/koha$ >>All this did was to *un-link" the bib/auth link that I had manually >>entered at [1] above. Could someone involved with signing off bug 5683 >>please explain why link==unlink? >>Help - please. >>Paul >>_______________________________________________ >>Koha-devel mailing list >><mailto:Koha-devel@lists.koha-community.org>Koha-devel(a)lists.koha-community.org >> >>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>website : <http://www.koha-community.org/>http://www.koha-community.org/ >>git : <http://git.koha-community.org/>http://git.koha-community.org/ >>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/ >> >> >>_______________________________________________ >>Koha-devel mailing list >><mailto:Koha-devel@lists.koha-community.org>Koha-devel(a)lists.koha-community.org >>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>website : <http://www.koha-community.org/>http://www.koha-community.org/ >>git : <http://git.koha-community.org/>http://git.koha-community.org/ >>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/ >> >> >> >> >>-- >>Ian Walls >>Lead Development Specialist >>ByWater Solutions >>ALA Midwinter Booth #2048 >>Phone # (888) 900-8944 >><http://bywatersolutions.com>http://bywatersolutions.com >><mailto:ian.walls@bywatersolutions.com>ian.walls(a)bywatersolutions.com >>Twitter: @sekjal >>_______________________________________________ >>Koha-devel mailing list >><mailto:Koha-devel@lists.koha-community.org>Koha-devel(a)lists.koha-community.org >>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>website : <http://www.koha-community.org/>http://www.koha-community.org/ >>git : <http://git.koha-community.org/>http://git.koha-community.org/ >>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/ > >--- >Archives and Collections (ACS) Society >205, Main Street, Picton, Ontario, K0K 2T0, Canada ><http://www.aandc.org/>http://www.AandC.org >Canadian Charitable Organization 88721 9921 RR0001 >Dedicated to maritime conservation and education. > >_______________________________________________ >Koha-devel mailing list >Koha-devel(a)lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/ --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. <http://UltraMarine.ca>, <http://AandC.org> and <http://MarDoc.ca>

4 7

help with nomenclature
by Paul 16 Feb '12

16 Feb '12

Does anyone have a "cheat sheet" covering the nomenclature of fields, strings, descriptions, etc used in the various components of Koha? (mysql, perl, inc, tt etc) For example, I would like to add to a "moredetails.pl" prequest a couple more editing possibilities (just like "Lost", "Damaged" and "Withdrawn") such as "Shelving location" (952$c) and 952$g that we use for Fair Market Value. As an example of trying to trace a variable (from additem.pl), the raw data gets transformed/added to during various loops from 952$c and tracing "shelving" from: <div id="subfield952c175402" class="subfield_line" style=""> <label> c - <span id="error13" title="Shelving location">Shelving location</span> </label> <select id="tag_952_subfield_c_166742" class="input_marceditor" size="1" tabindex="1" name="field_value"> <input type="hidden" value="952" name="tag"> <input type="hidden" value="c" name="subfield"> <input type="hidden" value="0" name="mandatory"> </div> which poses as many questions as it answers -- and I'd like to proceed as efficiently as possible. The various online manuals etc do not appear to have a nomenclature / logic flow chart but I am hoping that one of you may have something ready made along these lines I'd also appreciate pointers to the origins of subfield952c175402, error13, tag_952_subfield_c_166742, etc - I'm guessing I could cause a minor disaster in the sql. Thanks and best regards, Paul

1 0

Replacing dependency on Date::ICal
by Robin Sheat 13 Feb '12

13 Feb '12

Date::ICal is no longer really maintained, and is being dropped by Debian at least: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653301 I haven't looked to see how we use it, but we should probably consider replacing it with DateTime::Format::ICal. Oh, grep has finished: $ grep -ri 'Date::ical' * C4/Installer/PerlDependencies.pm: 'Date::ICal' => { INSTALL.fedora7:Date::ICal opac/opac-ics.pl:use Date::ICal; opac/opac-ics.pl: my $datestart = Date::ICal->new( opac/opac-ics.pl: my $dateend = Date::ICal->new( I've created http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7532 to change this, just want to know if anyone has any issues with doing this. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

2 1

IRC meeting decision about "discussion"
by Paul Poulain 12 Feb '12

12 Feb '12

Hi all, (Koha general mailing list reader: this mail is also for you, please read it's an opportunity to participate to Koha, even if you don't have technical skills !) Tomas was faster than me to start a discussion, I planned to send a mail about the results of our discussion during the last IRC session. The question was : "how to deal with patches that are 'in discussion'". I made a proposal, that has been a little bit amended during the discussion, here is the final result: * when a patch is rejected or is conflicting with another one (not a technical conflict, but a "strategic" conflict), the bug status can be set to "In Discussion" * Someone (I volunteered to do it, but feel free to do it yourself if you're concerned= - start a wiki page with describing the problem - send a mail to koha-devel and koha pointing to the wiki page, and calling for comments. * the delay for discussion is "until next IRC meeting, not less than 1 week" * at the next IRC meeting, - if the discussion (on the wiki or koha-devel) result in a general agreement, nothing specific need to be made (no vote needed) - if the discussion result in a balanced situation, organize a vote. About the vote (this has not been discussed on last IRC meeting, thinking of it and proposing it now): as some of us are always sleeping during IRC meetings, it's unfair to vote only on the IRC channel during the meeting. I propose that ppl could also express their preference/vote on the wiki itself (and have a specific section for that on the page). Those votes would be added to the IRC vote. I've started the general discussion wiki page: http://wiki.koha-community.org/wiki/Bug_and_Enhancement_Discussion -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

4 5

Re: [Koha-devel] [Discussion tech] database table naming
by Marc Balmer 10 Feb '12

10 Feb '12

Am 10.02.12 23:15, schrieb David Schuster: > Where you the one working on getting Koha to work with postgres SQL? Yes. > The School district I work for wants to see about making Koha work with > SequelSQL(Microsoft)... A bad choice. Consider PostgreSQL. > > What have you found in your journey that we should be aware of and > thinking about to make Koha database agnostic? Some shim layer in Perl will be needed to make Koha DB agnostic. I plan to work on this during the Marseille hackathon, > > We did work with Moodle to make it agnostic in regards to DB storage. > > On Fri, Feb 10, 2012 at 5:14 AM, Marc Balmer <marc(a)msys.ch > <mailto:marc@msys.ch>> wrote: > > Am 10.02.12 11:44, schrieb Paul Poulain: > > Continuing with open discussion for the future of Koha: > > > > On the page http://wiki.koha-community.org/wiki/DB_schema_bugs, we > have > > referenced some inconsistencies on the database. > > One of them is about table naming : acquisition related tables all > start > > by aq, other tables don't have any prefix. > > > > OTOH, Julian has submitted a bug > > http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5339 that > adds > > an "invoice" table, related to acquisitions, but don't has a aq > prefix. > > > > The possible options are on > > http://wiki.koha-community.org/wiki/Table_naming, let's start the > > discussion ! > > A better (imo) move than prefixes would be to use DB schemas. This way > related tables can be grouped in a schema, and default security can be > defined at the schema level (at least on PostgreSQL). > > afaik, both MySQL and PostgreSQL support schemas. > > _______________________________________________ > Koha-devel mailing list > Koha-devel(a)lists.koha-community.org > <mailto:Koha-devel@lists.koha-community.org> > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > > > > -- > David Schuster > Plano ISD > Library Technology Coordinator

1 0

Automatic detection of "does not apply" patch
by Paul Poulain 10 Feb '12

10 Feb '12

Hello, I wrote a small script that automatically retrieve all patches that "Needs signoff" and try to apply them on master. If it's not possible, the script can also update the status on the bug. I'll push the script on misc/devel/ directory (that will be created) I was wondering what to do with this script. The script can be run regularly, and automatically change the status of the bug, adding a comment saying "The script misc/devel/testbugzillapatches.pl has detected that this patch does not apply anymore:" with the git error message. I ran the script, with the update of bugzilla commented, and can already say the following patches don't apply: 3571, 4032, 5166, 5786, 5787, 5788, 5877, 5911, 6296, 6334, 6504, 7411, 7430 (13 patches on 77 waiting for being signed-off) My proposition is: * to push this patch into Koha, to let everybody know how it works * run it myself regularly, something like every friday. In some weeks/months, once i'm sure there are no hidden problems with the script, run it cronly, from jenkins PS: I also could easily check for signed off patches, just in case some of them don't apply anymore before being QAed PS2: the only problem i've found is when a patch depends on another one. My script won't be able to detect that and will mark "does not apply". I think it's an uncommon situation that we can deal with, but switching back to "need signoff" when the initial patch is pushed. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

2 2