November 2010 - Koha-devel - lists.koha-community.org

Re: [Koha-devel] MARC record size limit
by Thomas Dukleth 18 Nov '10

18 Nov '10

Reply inline: Original Subject: [Koha-devel] Search Engine Changes : let's get some solr On Mon, October 4, 2010 08:10, LAURENT Henri-Damien wrote: [...] > zebra is fast and embeds native z3950 server. But it has also some major > drawbacks we have to cope with on our everyday life making it quite > difficult to maintain. [...] > I think that every one agrees that we have to refactor C4::Search. > Indeed, query parser is not able to manage independantly all the > configuration options. And usage of usmarc as internal for biblio comes > with a serious limitation of 9999 bytes, which for big biblios with many > items, is not enough. How do MARC limitations on record size relate to Solr/Indexing or Zebra indexing which lacks Solr/Lucene support in the current version? How does BibLibre intend to fix the limitation on the size of bibliographic records as part of its work on record indexing and retrieval in Koha or in some parallel work.? > > BibLibre investigated in a catalogue based on solr. > A University in France contracted us for that development. > This University is in relation with all the community here in France and > solr will certainly be adopted by all the libraries France wide. > We are planning to release the code on our git early spring next year > and rebase on whatever Koha version will be released at that time 3.4 or > 3.6. > > > Why ? > > Solr indexes with data with HTTP. > It can provide fuzzy search, search on synonyms, suggestions > It can provide facet search, stemming. > utf8 support is embedded. > Community is really impressively reactive and numerous and efficient. > And documentation is very good and exhaustive. > > You can see the results on solr.biblibre.com and > catalogue.solr.biblibre.com > > http://catalogue.solr.biblibre.com/cgi-bin/koha/opac-search.pl?q=jean > http://solr.biblibre.com/cgi-bin/koha/admin/admin-home.pl > you can log there with demo/demo lgoin/password > > http://solr.biblibre.com/cgi-bin/koha/solr/indexes.pl > is the page where ppl can manage their indexes and links. > > a) Librarians can define their own indexes, and there is a plugin that > fetches data from rejected authorities and from authorised_values (that > could/should have been achieved with zebra but only with major work on > xslt). > > b) C4/Search.pm count lines of code could be shrinked ten times. > You can test from poc_solr branch on > git://git.biblibre.com/koha_biblibre.git > But you have to install solr. > > Any feedback/idea welcome. [...] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

4 7

Preliminary Release Notes for 3.2.1
by Chris Nighswonger 18 Nov '10

18 Nov '10

Hi all, Below are the preliminary release notes for the pending 3.2.1 release on 22/11/10. Please review them and reply with any corrections, amendments, additions, etc. Kind Regards, Chris Nighswonger Koha 3.2.x Release Maintainer ------------------------------------------------------------ RELEASE NOTES FOR KOHA 3.2.1 - 22 November 2010 ======================================================================== Koha is the first free and open source software library automation package (ILS). Development is sponsored by libraries of varying types and sizes, volunteers, and support companies from around the world. The website for the Koha project is http://koha-community.org/ Koha 3.2.1 can be downloaded from: http://download.koha-community.org/koha-3.02.01.tar.gz Installation instructions can be found at: http://wiki.koha-community.org/wiki/Installation_Documentation Koha 3.2.1 is a bugfix/maintenance release. Bugs fixed in 3.2.1 ====================== 2122 Grayed out Fields not always visible 2567 008 editor falsely populating Illustrations and Nature of Contents positions 3013 Value builder for 006 and 008 need choices for all format types 3211 cataloging value plugin forms should be valid XHTML 3271 Missing message when adding to a list with no items selected 3811 biblios record.abs indexes 008 language as numeric (marc21) 4074 The 'Subject(s)' link(s) are malformed resulting in no search results 4141 reconcile 3.0.x and HEAD database updates for 3.2.0 4254 adding new patron advanced notice doesn't save 4261 keyword mapping should show which framework the map is for 4305 Amazon book covers do not work with ISBN13 4359 noItemTypeImages not active in OPAC 4423 Staff Client XSLT is just a copy of the OPAC one 4472 img tags in xslt broken after automatic translation 4498 Acq always shows '1 suggestions waiting' 4515 Few very small errors in opac-search.pl 4520 facets "show more" doesn't work 4866 Optionally enable Change event for item plugins 4912 After editing private list, user should be redirect to private lists 4913 Budget planning pages should show currency name instead of symbol 4924 Public/Internal notes missing in staff normal view 4933 Link to subfield's edit tab from MARC subfield structure admin summary 4963 sys prefs need date hints and/or picker 4979 Acq: input fields for new record are too short 4980 Acq: pull down 'Restrict access to:' for funds not translatable 4986 move serials prefs from cataloging tab 4991 Overhaul Calendar interface 5003 Can not search for organisation by name 5008 "Remove" link missing when viewing Cart in expanded "More details" view 5019 funds link doesn't go to list 5037 If patron category is empty it shouldn't show 5050 Staff client's language preference should be grouped with I18N/L10N preferences 5056 Untranslatable strings in members.js 5059 Inconsistent use of ordering price and list price in vendor form 5066 Incorrect use of localtime function when calling _session_log 5075 Terms not highlighted w/ xslt on 5082 Not translatable name of default framework 'Default' in MARCdetail.tmpl 5110 NewItemsDefaultLocation should be under cataloging 5112 Organisation does not show links to professional patrons 5114 Can't edit basket in Internet Explorer 5117 Misspelled word: Orgnisztion 5118 Misspelled word: Currencey 5119 Misspelled word: correspounding 5121 Misspelled words: stripts biographyl Begininning 5122 Misspelled word: Transfered/transfered 5123 Misspelled words: Depdending Commited flutucations 5124 Duplicate and Misspelled words: periodicy outputing 5128 Define default 9xx fields for Unimarc setup in all languages 5130 Misspelled words: biblographic delimeter extention 5132 Misspelled words: Acquistion Succesfully professionnal 5133 Misspelled words: reservior notifiying deleete 5134 Misspelled words: exisiting anomolies genereated 5135 Authorized value input maxlength should match table column 5136 Replace embedded SQL query with call to GetAuthorisedValues 5137 Remove obsolete code for counting issues by item type in circulation 5142 Untranslatable strings in tag review template 5146 Patron Import Requires Header Row 5149 Link to noItemTypeImages pref on item types is wrong 5151 Saved Report Breadcrumb in bold 5152 confirm buttons different styles on lists 5162 patron attributes 'new' link should create blank value 5163 holds to pull is titled pending holds 5168 add holdings' should read 'add/edit items' 5171 edit items should be edit item when under 1 item 5175 The opac XSLTDetails view field Publisher: doesn't provide a hyperlink as the non XSLT view does. 5177 Descending sort search result defined by syspref doesn't work 5190 MARC and UNIMARC bibliographic fields documentation accesible when cataloging 5204 Unimarc xslt default templates are in french 5210 Function of back button on batch modification result page unclear 5214 undefined itype images cause broken image icon in request.pl 5219 Perltidy code 5221 Preselect tab containing itemtype/authorized value image in use 5223 related subjects' should read 'subjects' 5224 Replace Shopping Basket with basket 5235 checkout receipt should include patron name 5236 "hide my tags" link does nothing 5237 Special characters in patron barcodes cause renewals to fail. 5243 C4::Search::SimpleSearch call in AuthoritiesMarc.pm 5254 no need to scroll left to right on acq z search 5258 Order should read 'order line' in receive table 5301 In C4/XSLT.pm, itemcallnumber can contain special XML characters 5308 subscriptionroutinglist table is too lax 5309 Progress bar inaccurate 5311 Language dropdown in advanced search broken? 5315 Authority search Templates refer to unused variable 5318 rank weight error in ccl.properties 5322 pwgen not a dependency for the packages 5326 when ExtendedPatronAttributes off error links to old sys prefs 5327 Unit tests required for all C4 modules 5363 Remove dependency on Cache::Memcached::Fast 5368 Browse Shelf link appears even if there isn't an itemcallnumber 5370 Fix all the references to koha.org 5372 Editing a bilbio record, existing value can be replace by default value 5380 an end to copy-and-paste-itis 5381 Fines in notices prints always 0.00 5385 Correct POD errors as highlighted by podchecker 5389 Business::ISBN should be marked as required dependency 5392 reserve/renewscript logging lots of warnings 5393 test case for verifying XML/XSLT files are well-formed 5396 UseTablesortForCirc syspref to toggle table sorter on circ/circulation.pl 5400 add test case to detect merge conflict markers in the code 5412 Double quotes in publisher name, no date cause search results links to break Commits in 3.2.1 without a bug report: * Adding a simple test for Service.pm * create unit test files * Create Unit Test for ImportBatch * unit test stub for Z3950.pm * Test modules compile * Updated links in Main Page Help * remove extraneous semicolon * History updates * history updates - recent releases * Adding 3.2 Release Maintainer to Release Team List * Adding possibility to cleanup_database.pl to purge only older sessions * Display available error information during bulkmarcimport * add missing help file for merging records * (MT 2985) simplify CanBookBeReserved * fix use of outdated boilerplate * remove unused template include * Misspells: deleteing -> deleting * Misspell: Quanity -> Quantity System requirements ====================== Changes since 3.0: * The minimum version of Perl required is now 5.8.8. * There are a number of new Perl module dependencies. Run ./koha_perl_deps.pl -u -m to get a list of any new modules to install during upgrade. Upgrades ====================== The structure of the acquisitions tables have changed significantly from 3.0.x. In particular, the budget hierarchy is quite different. During an upgrade, a new database table is created called fundmapping that contains a record of how budgets were mapped. It is strongly recommended that users of Koha 3.0.x acquisitions carefully review the results of the upgrade before resuming ordering in Koha 3.2.x. Documentation ====================== As of Koha 3.2, the Koha manual is now maintained in DocBook. The home page for Koha documentation is http://koha-community.org/documentation/ As of the date of these release notes, several translations of the Koha manual are available: English: http://koha-community.org/documentation/3-2-manual/ Spanish: http://koha-community.org/documentation/3-2-manual-es/ French: http://koha-community.org/documentation/3-2-manual-fr/ The Git repository for the Koha manual can be found at http://git.koha-community.org/gitweb/?p=kohadocs.git;a=summary Translations ====================== Complete or near-complete translations of the OPAC and staff interface are available in this release for the following languages: * Chinese * Danish * English (New Zealand) * English (USA) * French (France) * French (Canada) * German * Greek * Hindi * Italian * Norwegian * Portuguese * Spanish * Turkish Translation related commits new to 3.2.1: * Staff interface .po file updates * Russian and Ukranian opac language updates * Ukranian and Russian syspref language updates * German and italian language updates Partial translations are available for various other languages. The Koha team welcomes additional translations; please see http://www.kohadocs.org/usersguide/apb.html for information about translating Koha, and join the koha-translate list to volunteer: http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-translate The most up-to-date translations can be found at: http://translate.koha.org/ Release Team ====================== The release team for Koha 3.2 is Release Manager: Galen Charlton <gmcharlt(a)gmail.com> Documentation Manager: Nicole Engard <nengard(a)gmail.com> Translation Manager: Chris Cormack <chris(a)bigallofwax.co.nz> Release Maintainer (3.0.x): Henri-Damien Laurent <henridamien.laurent(a)biblibre.com> Release Maintainer (3.2.x): Chris Nighswonger <cnighswonger(a)foundations.edu> Credits ====================== We thank the following individuals who contributed patches to Koha 3.2.1. Nahuel Angelinetti TomÃ¡s Cohen Arazi Jared Camins-Esakov Colin Campbell Galen Charlton Chris Cormack Nate Curulla FrÃ©dÃ©ric Demians Andrew Elwell Brian Engard Nicole Engard Magnus Enger Katrin Fischer Daniel Grobani Kyle M Hall Srdjan Jankovic Bernardo Gonzalez Kriegel Henri-Damien Laurent Owen Leonard Chris Nighswonger Paul Poulain MJ Ray Liz Rea Marcel de Rooy Robin Sheat ByWater Solutions Zeno Tajoli Ian Walls We regret any omissions. If a contributor has been inadvertantly missed, please send patch against these release notes to koha-patches(a)lists.koha-community.org. Revision control notes ====================== The Koha project uses Git for version control. The current development version of Koha can be retrieved by checking out the master branch of git://git.koha-community.org/koha.git The branch for Koha 3.2.x (i.e., this version of Koha and future bugfix releases) is 3.2.x. The next major feature release of Koha will be Koha 3.4.0. Bugs and feature requests ====================== Bug reports and feature requests can be filed at the Koha bug tracker at http://bugs.koha-community.org/ Naku te rourou, nau te rourou, ka ora ai te iwi.

1 0

Perl resources
by Colin Campbell 17 Nov '10

17 Nov '10

Hi. details of the talks at this years London Perl Workshop have been posted: http://conferences.yapceurope.org/lpw2010/talks One thing that may be of interest to some here is a new book on Perl written in French http://www.pearson.fr/livre/?GCOI=27440100979970 Cheers Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell(a)ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com

2 1

Koha - city field in patrons
by Linda Culberson 16 Nov '10

16 Nov '10

This is only a suggestion, and not a formal RFP. When showing Koha to some of our staff, I was asked why the state (region/province/provincial district in the case of some countries) isn't separated from the "borrowers.city" field. (Sorry, not enough coffee yet, this morning. ) In other words, why doesn't the state (region/province/provincial district) have it's own field? When asked, I didn't have an answer. I will admit that one of my borrower_attribute_types.codes is STATECODE, so the state information is actually duplicated in our database, but I need it for statistical reasons. I apologize if I'm posting this to the wrong list, but I plead caffeine deprivation. Thanks. -- Linda Culberson lculber(a)mdah.state.ms.us Archives and Records Services Division Ms. Dept. of Archives& History P. O. Box 571 Jackson, MS 39205-0571 Telephone: 601/576-6873 Facsimile: 601/576-6824

3 2

Time for reflexion
by Miguel Angel Calvo Lázaro 16 Nov '10

16 Nov '10

We want to take this opportunity to congratulate Biblibre for initiating the search engine change with which we are identified and in which we would like to collaborate, due to the important problem on the facets. We believe that it is time to examine some aspects of the project, with no intention of offending any member of the community in doing so. In Xercode, despite being a new company, some of our members have been working on the project since 2007, when we first came into contact with PERL. All of our present developers have been working on PHP and JAVA projects. Since then, we have been conscious that the structure of KOHA is unlike any other major project we have worked on previously. In our opinion, it has become necessary to upgrade technologies employed in the project making KOHA more competitive with similar software. We believe that is urgent to use a framework like DANCER, SOLR as an index engine (our main problem with clients), increase performance, ... We believe, however, that this will not completely resolve our present situation due to the fact that the project structure is not sufficiently defined. A good example is the propose of search engine change, a current point of conflict. We have traditionally followed an Object Oriented Metodology (working with interfaces and abstract classes). By doing this we would achieve an abstraction of the search engine and the result of using a method, for example “engineSearch”, would be the same with indifference to use SOLR or ZEBRA and the majority of problems that we are currently facing in the community (principally the integration of the diverse developments by each member) can be easily resolved. Applying this philosophy to .pl files would make them more concise and more easily understood. DRUPAL and OTRS (PERL) are good examples composed by a kernel and specific modules. Considering all of this, this is a perfect moment for reflexion due to the fact that these points may affect the scalability and evolution of KOHA. Regards, * Miguel Angel Calvo Lázaro* *Dirección de Soluciones* miguel.calvo(a)xercode.es Telf. 653 038 238 Rosalía de Castro, 53 4º C 15895 Milladoiro - Ames (A Coruña) Telf. 881 975 576 www.xercode.es info(a)xercode.es La información contenida en este mensaje y sus posibles documentos adjuntos es privada y confidencial y está dirigida únicamente a su destinatario/a. Si usted no es el/la destinatario/a original de este mensaje, por favor elimínelo. La distribución o copia de este mensaje no está autorizada.

8 11

Guidelines for Patch Acceptance/Rejection
by Ian Walls 15 Nov '10

15 Nov '10

Everyone, While there can be no guarantees as to whether a patch will be committed into the Koha codebase, I think in practice there are several requirements. This email is an attempt to identify a few of them, and hopefully start a discussion about whether they are truly requirements, and what others could possibly be added. 1. The patch must do what it claims to do, in all commonly-supported Koha environments 2. The patch must not break existing functionality 3. The patch must apply to the current HEAD of the master branch of the code 4. The patch must follow the Coding Style Guidelines 5. The patch must be MARC-flavour agnostic 6. The patch must contain appropriate copyright information 7. If a database update is require, the patch must handle the update both for new installs and upgrades 8. If a new feature is added, the patch must include appropriate Help documentation What do people think of these requirements? Are they reasonable? Should there be more? I understand that there may not be any set of requirements that's completely sufficient, but if we can identify as many as possible, it would make developers lives a bit easier, since we'd all have a better idea what is needed for our patches to be committable. Cheers, -Ian -- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls(a)bywatersolutions.com Twitter: @sekjal

8 12

Official Koha Newsletter: Volume 1, Issue 11: November 2010
by Nicole Engard 15 Nov '10

15 Nov '10

This issue has a lot of links and should be read online to see all of the external content: http://koha-community.org/koha-newsletter-volume-1issue-11-november-2010 Official Koha Newsletter (ISSN 2153-8328) Volume 1, Issue 11: November 2010 Table of Contents * KohaCon10 Summary o Summary of KohaCon10 in Blog Posts o KohaCon10 Presentations o KohaCon10 Videos o KohaCon10 Pictures KohaCon10 Summary of KohaCon10 in Blog Posts KohaCon10 took place in Wellington, New Zealand last month to honor 10 years of Koha development, use and community! It seems that your trusty documentation manager was unofficially names the official blogger of the conference, so excuse a newsletter full of links to posts written by me (and Ian Walls, my colleague who had cover the sessions I couldn’t). * Conference Proper o Keynote o Why a Librarian Loves Koha o Koha and Kete o Koha Adoption in Nigeria o Koha Governance o What’s Coming in Koha 3.4 o Fun with Library Data o Identifying eBooks o A cooperative view o Koha in Schools o Sharing is Good o History of Koha o Koha in Prison o Participation is Key o Koha in Malaysia o LAMP to Koha o Promoting Free Software in Libraries o Open Library and Koha o Koha: 10 years in 10 minutes * Hackfest o Intro to Git o Git and Koha o Template Toolkit o MySQL and Postgres Differences o Debian Packaging for Koha o Koha in the Cloud o An Argument for a project Coding Style o Ideas for Koha 4.0 o Koha Performance * Summaries o KohaCon10 is Finished o KohaCon10 Additional Posts KohaCon10 Presentations Some KohaCon10 presentations were posted on SlideShare, others posted on personal websites and others plotted on a map. If I’ve missed any please feel free to add them in the comments below. * KohaCon10 on SlideShare * KohaCon10 Presentation Map * Specific Presentations o Koha at ASHS by Mark Osborne o Fun with Library Data by David Friggens KohaCon10 Videos KohaCon10 videos are still making their way online, here are a few that are up so far. You can search for the rest as they’re added on blip.tv by using the keyword “kohacon10” * Keynote by Rosalie Blake * Why Librarians Love Koha by Lee Phillips o Video of Lee’s librarians * Kete and Koha by Walter McGinnis * Koha uptake in Nigeria by Olugbenga Adara * Library Data for Fun & Profit by David Friggens * Koha: 10 years in 10 minutes KohaCon10 Pictures Mostly KohaCon10 pictures were posted on Flickr and tagged “kohacon10“. My favorites were taken by Kristina, she knows how to take pictures!! Thank you to all who took pictures and shared them so that those back home could share in the festivities with us and so that we can all look back and remember our time together fondly. What we didn’t get was a group picture that I could post here, so you’ll have to go browse through the Flickr pictures and spot your friends and community members that way. Newsletter edited by Nicole C. Engard, Koha Documentation Manager. Please send future story ideas to me.

1 0

Externalizing PO files (Was Re: externalising translator and etc)
by Galen Charlton 15 Nov '10

15 Nov '10

Hi, [Splitting the thread, there are two distinct topics being discussed.] On Mon, Nov 15, 2010 at 4:16 AM, LAURENT Henri-Damien <laurenthdl(a)alinto.com> wrote: > Le 14/11/2010 22:51, Galen Charlton a écrit : > From my understanding of Scott Chacon (a git developer) book and git > talk, git stores the files in its database, since it is a DAG oriented > SCM. It may send the diffs, but actualy stores the files. A cursory reading of a Git book reveals that a statement that Git "actually stores the files" is simplistic, at particular as far as the implications for storage are concerned. Please take a look at: http://book.git-scm.com/7_how_git_stores_objects.html Git's object storage takes the form of compressed version of each object (commit, file, tag, etc.) a files and (and this is the important part) packfiles. Packfiles, roughly speaking, encode only the changes to objects, so a significant amount of space is saved. If one disabled automatic garbage collection in a git repo and never ran git gc, you might end up with something close to storing the (compressed) version of each changed file, but in practice that doesn't happen. > I don't think all the release tags would be broken. As it turns out, they would. I have actually tried running a git filter-branch to remove the PO directory. As I suspected, it does rewrite all of the commits (thereby changing the commit IDs), and it would be necessary to rewrite the tags. But doing so would invalidate the release tags that have a GPG signature. Rewriting history like this is a sure way to get months of complaints and calls for help on koha-devel with questions like "why does git rebase origin no longer work". > And it would allow to release localisation at a different pace... When > there is a need. There's is nothing stopping releasing localizations on a different schedule now. Packaging does not have any necessary connection with how the Git repository is laid out. I will grant that all other things being equal, it would have been a valid choice to have the PO files live in a separate repository. In fact, we perhaps can still do so. One thing that Chris Cormack mentioned to me yesterday is that Pootle has a Git plugin which can automatically commit changes to PO files to a repository. That would be a bit dangerous to allow for the main Koha repository, but safe enough for a separate translations file. Consequently, there is a possible compromise: [1] We seed a new repository for PO files. [2] We do a simple git rm misc/translator/po but *don't* rewrite history. This won't reduce the size of the main repository, of course, but achieves the separation. And, to be honest, it's mostly the possibility of integration with Pootle that would make me interested in this. However, since it is the translation manager who would be primarily affected by the split on a day-to-day basis, I would be interested in hearing from Frédéric as well as past TMs. Regards, Galen -- Galen Charlton gmcharlt(a)gmail.com

2 2

externalising translator and etc
by LAURENT Henri-Damien 15 Nov '10

15 Nov '10

Hi I propose to get etc and translator as external git repositories. Why ? It would allow to have po files in their own repository and limit the weight of Koha git archives. For etc, it would allow to change the installer in order to keep track of local changes for every person and therefore be able to synchronize with community version properly. HOW ? operation is simple : git filter-branch --subdirectory-filter misc/translator -- --all and then git push on an external repository. for instance etc.git and translator.git on git.koha-community This would allow to have more than one branch for etc and for instance create and etc/apache etc/nginx etc/plack and so on... WHEN ? when you think operation should be done, if you think it could be interesting. and then we could have git submodules in the koha.git repository in order to keep in synch and synchronise versions when we want. http://www.kernel.org/pub/software/scm/git/docs/git-submodule.html It is just a proposition. -- Henri-Damien LAURENT

7 8

Perl resource
by Colin Campbell 15 Nov '10

15 Nov '10

This may be of interest to some: http://www.onyxneon.com/books/modern_perl/index.html This new book is available as a free pdf. Cheers Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell(a)ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com

2 1