October 2010 - Koha-devel - lists.koha-community.org

koha-3.00.00 make fails
by Rick Welykochy 28 Dec '13

28 Dec '13

Hi all, I have downloaded http://download.koha.org/koha-3.00.00.tar.gz and began installing it on Debian/etch (in a Linux vserver environment). 0. FINE: cd koha-3.00.00 1. FINE: perl Makefile.PL 2. FAIL: make : : : Manifying blib/man3/C4::UploadedFile.3pm Manifying blib/man3/C4::BackgroundJob.3pm Manifying blib/man3/opac::opac-dictionary.3pm Manifying blib/man3/serials::member-search.3pm /usr/bin/perl "-Iblib/arch" "-Iblib/lib" fix-perl-path.PL blib /usr/bin/perl "-Iblib/arch" "-Iblib/lib" rewrite-config.PL blib/KOHA_CONF_DIR/koha-conf.xml Bad arg length for Socket::inet_ntoa, length is 0, should be 4 at rewrite-config.PL line 67. make: *** [blib/KOHA_CONF_DIR/koha-conf.xml] Error 255 Any ideas what went wrong and how to fix? Aside: there were a couple of other problems with this "out of the box" debian based install of Koha 3.0 which had to be fixed along the way, e.g. seven missing perl modules and libgd not found. I'll report on the entire process from unpacking to a running Koha/3.00.00 once I get to the end. cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor The manual said Windows XP/SP2 or better so I installed Linux.

4 5

Search Engine Changes : let's get some solr
by LAURENT Henri-Damien 07 Dec '10

07 Dec '10

Hi As you already read in Paul previous message about "BibLibre strategy for 3.4 and next version", we are growing, want be involved in the community as previously. Paul promised some POCs, here is one available. We also worked on Plack and support. We created a base of script to search for Memoryleaks. We'll demonstrate that later. zebra is fast and embeds native z3950 server. But it has also some major drawbacks we have to cope with on our everyday life making it quite difficult to maintain. 1. zebra config files are a nightmare. You can't drive the configuration file easily. namely : Can't edit indexs via HTTP or configuration. all is in files hardcoded on disk. ⇒ you can't list indexes you can't change indexes, you can't edit indexes, you can't say I want this index at OPAC, that in intranet. (Could be done with scraping ccl.properties, and then record.abs and bib1.att…. But what a HELL) So you cannot customize configuration defining the indexes you want easily. And ppl donot get a translation of the indexes since all the indexes are hardcoded in the ccl.properties and we donot have a translation process so that ccl attributes could be translated into different languages. 2. no real-time indexing : the use of a crontab is poor: when you add an authority while creating a biblio, you have to wait some some minutes to end your biblio (might be solved since zebra has some way to index biblios via z3950 extended services, but hard and should be tested and at the time community first tested that, a performance problem was raised on indexing.) 3. no way to access/process/delete data easily. If you have indexes in it or have some problems with your data, you have to reindex the whole stuff and indexing errors are quite difficult to detect. 4. during index process of a file, if you have a problem in your data, zebraidx just fails silently… And this is NOT secure. And you have no way to know WHICH biblio made the process crash. We had a LOT of trouble with Aix-Marseille universities that have some arabic translitterated biblios that makes zebra/icu completly crash ! We had to do some recursive script to find 14 biblios on 730 000 that makes zebra crash (even is properly stored & displayed) 5. facets are not working properly : they are on the result displayed because there are problems with diacritics & facets that can't be solved as of today. And noone can provide a solution (we spoke about that with indexdata and no clear solution was really provided. 6. zebra does not evolve anymore. There is no real community around it, it's just an opensource indexdata software. We sent many questions onlist and never got answers. We could pay for better support but the fee required is quite deterrent and benefit is still questionable. 7. icu & zebra are colleagues, not really friends : right truncation not working, fuzzy search not working and facets. 8. we use a deprecated way to define indexes for biblios (grs1) and the tool developped by indexdata to change to DOM has many flaws. we could manage and do with it. But is it worth the strive ? I think that every one agrees that we have to refactor C4::Search. Indeed, query parser is not able to manage independantly all the configuration options. And usage of usmarc as internal for biblio comes with a serious limitation of 9999 bytes, which for big biblios with many items, is not enough. BibLibre investigated in a catalogue based on solr. A University in France contracted us for that development. This University is in relation with all the community here in France and solr will certainly be adopted by all the libraries France wide. We are planning to release the code on our git early spring next year and rebase on whatever Koha version will be released at that time 3.4 or 3.6. Why ? Solr indexes with data with HTTP. It can provide fuzzy search, search on synonyms, suggestions It can provide facet search, stemming. utf8 support is embedded. Community is really impressively reactive and numerous and efficient. And documentation is very good and exhaustive. You can see the results on solr.biblibre.com and catalogue.solr.biblibre.com http://catalogue.solr.biblibre.com/cgi-bin/koha/opac-search.pl?q=jean http://solr.biblibre.com/cgi-bin/koha/admin/admin-home.pl you can log there with demo/demo lgoin/password http://solr.biblibre.com/cgi-bin/koha/solr/indexes.pl is the page where ppl can manage their indexes and links. a) Librarians can define their own indexes, and there is a plugin that fetches data from rejected authorities and from authorised_values (that could/should have been achieved with zebra but only with major work on xslt). b) C4/Search.pm count lines of code could be shrinked ten times. You can test from poc_solr branch on git://git.biblibre.com/koha_biblibre.git But you have to install solr. Any feedback/idea welcome. -- Henri-Damien LAURENT BibLibre

23 67

Facets performance
by Fridolyn SOMERS 26 Nov '10

26 Nov '10

Hie, I have posted a proposed patch for Bug 3154 : http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3154 It's about the fact that facets computation is limited to the records in search results page. I think I've found a good way to improve the facets extraction performance. Any comment or modification is welcome. Regards, -- Fridolyn SOMERS ICT engineer PROGILONE - Lyon - France fridolyn.somers(a)gmail.com

4 4

Re: [Koha-devel] MARC record size limit
by Thomas Dukleth 18 Nov '10

18 Nov '10

Reply inline: Original Subject: [Koha-devel] Search Engine Changes : let's get some solr On Mon, October 4, 2010 08:10, LAURENT Henri-Damien wrote: [...] > zebra is fast and embeds native z3950 server. But it has also some major > drawbacks we have to cope with on our everyday life making it quite > difficult to maintain. [...] > I think that every one agrees that we have to refactor C4::Search. > Indeed, query parser is not able to manage independantly all the > configuration options. And usage of usmarc as internal for biblio comes > with a serious limitation of 9999 bytes, which for big biblios with many > items, is not enough. How do MARC limitations on record size relate to Solr/Indexing or Zebra indexing which lacks Solr/Lucene support in the current version? How does BibLibre intend to fix the limitation on the size of bibliographic records as part of its work on record indexing and retrieval in Koha or in some parallel work.? > > BibLibre investigated in a catalogue based on solr. > A University in France contracted us for that development. > This University is in relation with all the community here in France and > solr will certainly be adopted by all the libraries France wide. > We are planning to release the code on our git early spring next year > and rebase on whatever Koha version will be released at that time 3.4 or > 3.6. > > > Why ? > > Solr indexes with data with HTTP. > It can provide fuzzy search, search on synonyms, suggestions > It can provide facet search, stemming. > utf8 support is embedded. > Community is really impressively reactive and numerous and efficient. > And documentation is very good and exhaustive. > > You can see the results on solr.biblibre.com and > catalogue.solr.biblibre.com > > http://catalogue.solr.biblibre.com/cgi-bin/koha/opac-search.pl?q=jean > http://solr.biblibre.com/cgi-bin/koha/admin/admin-home.pl > you can log there with demo/demo lgoin/password > > http://solr.biblibre.com/cgi-bin/koha/solr/indexes.pl > is the page where ppl can manage their indexes and links. > > a) Librarians can define their own indexes, and there is a plugin that > fetches data from rejected authorities and from authorised_values (that > could/should have been achieved with zebra but only with major work on > xslt). > > b) C4/Search.pm count lines of code could be shrinked ten times. > You can test from poc_solr branch on > git://git.biblibre.com/koha_biblibre.git > But you have to install solr. > > Any feedback/idea welcome. [...] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

4 7

November Newsletter
by Nicole Engard 13 Nov '10

13 Nov '10

Hello all, I'm thinking for the next Koha Newsletter we'll do a conference sum-up. So between the start of the conference and the 12th of November please send me links to posts you may have written about conference sessions, links to pictures from KohaCon, and anything else conference related. Thanks Nicole C. Engard

1 2

opac-search.pl
by Toni Rosa 09 Nov '10

09 Nov '10

Hello, We are using Koha 3.0.5 I noticed that the opac advanced search in it (opac-search.pl) when running in a nozebra installation doesn't honor "publication period" intervals (e.g entering the 1800-1900 interval the search doesn't work). I followed the code to the "C4/Search.pm" file, method "NZanalyse" and it seems that it actually doesn't even try to build the sql query taking the intervals into account. Did anyone find the same behavior? (I'd guess everybody should, as it seems a bug/omission to me!) Thanks, Toni

4 6

DBIx::Class vs current Koha's DBI performance
by Dobrica Pavlinusic 08 Nov '10

08 Nov '10

I just figured out today that kohacon10 is in progress, and one of topics seems to be DBIx::Class migration. Let me start by saying that I'm all for it, but probably not for same resons as everybody else. I was trying to profile koha search page. On first run under Devel::NYTProf it showed 2220 invocations of DBI::st::execute. That's more than 2000 calls which wait for database to return answer. This might explain extremly high usage of mysql query cache which I'm seeing in production: http://mjesec.ffzg.hr/munin/koha/koha-mysql_queries.html So, I decided to experiment a bit. and few patches later I was able to reduce number of queries issued on each page load down to 876 If you are interested in what I changed, my code is at: http://git.rot13.org/?p=koha.git;a=shortlog;h=refs/heads/nytprof-cache It's basically using either global hash to cache values per request of simple memoize around function. Which brings me back to DBIx::Class. If we can cache values using dbic, this will be huge WIN for Koha performance as opposed to loss :-) I never used dbic in that contex, but google seems to think it's possible, and I'm willing to take a try at it. -- Dobrica Pavlinusic 2share!2flame dpavlin(a)rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin

9 16

BibLibre branches for 3.4
by Paul Poulain 31 Oct '10

31 Oct '10

Hello, As we've said previously, we have a lot of patches to submit for 3.4 (637 exactly) We've rebased them against 3.2 official, and splitted them in "small" chunks. There are 10 different branches. You can pull them from http://git.biblibre.com/?p=koha;a=heads The branches 3.4/BibLibre-<module> are the one that are related to this stuff. I've added a page on the wiki to describe what each branch contains. Some smart ppl at KohaCon hackfest have started some testing & added their comment : http://wiki.koha-community.org/wiki/BibLibre_patches_to_be_integrated_for_3… We also have setup a "sandbox" install for each branch. It contains just all the default datas you can load from the webinstaller, marc21/english Note that all the tests have been made against a fresh installation, not from on an updatedatabase. Colin/Chris/Anyone, it's up to you ! feel free to ask questions, report problems,... Enjoy ! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

2 1

Adding events to the website
by Nicole Engard 31 Oct '10

31 Oct '10

A discussion today at Hackfest was about the Events calendar on the Koha site (http://koha-community.org) For those who don't know how to use the events calendar on our site I created a video tutorial for this for another group I was working with. http://www.youtube.com/watch?v=sxj2N92priE Hopefully that will help people add all Koha related events to the Koha website for us all to keep track of. The events calendar can then be subscribed to. Nicole

1 1

Re: [Koha-devel] November Newsletter
by MJ Ray 29 Oct '10

29 Oct '10

Nicole wrote: > Also remember to put your pics online and tag them kohacon10 and put > your slides in the Slideshare event: > http://www.slideshare.net/event/kohacon10 > > All these links will be put into the newsletter in 15 days or so. Slideshare stinks because it uses flash and requires registration before you can download, I think? So can we put slides on the conference site or a wiki page before linking them from our foss lms's newsletter please? Ta, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work http://www.software.coop/products/

1 0