- Koha-devel - lists.koha-community.org

Fwd: RE: rebuild_zebra and a limit at 700 000 biblios
by Paul Poulain 16 Jul '10

16 Jul '10

Scott, I already have : register: /home/koha/var/lib/zebradb/biblios/register:40G shadow: /home/koha/var/lib/zebradb/biblios/shadow:40G or I'm looking at the wrong file... double check ... unless i'm blind (or really too tired), I have 40G, so that should be enough :\ -------- Message original -------- Sujet: RE: [Koha-devel] rebuild_zebra and a limit at 700 000 biblios Date : Thu, 15 Jul 2010 17:17:16 -0400 De : Scott Kushner <skushner(a)mtpl.org> Pour : Paul Poulain <paul.poulain(a)biblibre.com> Paul, I think you have to increase your memory buffer in biblios.cfg to 8G or more. I had that problem. # Lock File Area lockDir: /root/koha-dev/var/lock/zebradb/biblios perm.anonymous:ar perm.root:rw passwd: /root/koha-dev/etc/zebradb/etc/passwd register: /root/koha-dev/var/lib/zebradb/biblios/register:8G shadow: /root/koha-dev/var/lib/zebradb/biblios/shadow:8G # Temp File area for result sets setTmpDir: /root/koha-dev/var/lib/zebradb/biblios/tmp # Temp File area for index program keyTmpDir: /root/koha-dev/var/lib/zebradb/biblios/key # Approx. Memory usage during indexing memMax: 150M rank:rank-1 truncmax: 1000000000 Scott Kushner Information Technologies Middletown Public Library -----Original Message----- From: koha-devel-bounces(a)lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Paul Poulain Sent: Thursday, July 15, 2010 5:11 PM To: koha-devel(a)lists.koha-community.org Subject: [Koha-devel] rebuild_zebra and a limit at 700 000 biblios Hello, We have a database of 726k biblios and 1.2M items. When we rebuild_zebra, everything is OK on the export. Then, the rebuild_zebra seems OK as well, except that only 702k biblios are indexed. We don't have any error, everything seems OK. Does anyone have seen this problem ? Logs : exporting biblio ==================== Records exported: 726210 <snip> 06:43:19-08/07 zebraidx(314) [log] Records: 700000 i/u/d 700000/0/0 06:43:29-08/07 zebraidx(314) [log] Records: 701000 i/u/d 701000/0/0 06:43:38-08/07 zebraidx(314) [log] Records: 702000 i/u/d 702000/0/0 07:42:38-08/07 zebraidx(314) [log] Merge 0% <snip> 07:00:52-08/07 zebraidx(314) [log] Merge 95.4% completed; 48 seconds remaining 07:01:02-08/07 zebraidx(314) [log] Merge 96.4% completed; 37 seconds remaining 07:01:12-08/07 zebraidx(314) [log] Merge 97.4% completed; 26 seconds remaining 07:01:22-08/07 zebraidx(314) [log] Merge 98.7% completed; 13 seconds remaining 07:01:32-08/07 zebraidx(314) [log] Merge 99.9% completed; 0 seconds remaining 07:01:38-08/07 zebraidx(314) [log] Iterations: isam/dict 301420236/21553295 07:01:38-08/07 zebraidx(314) [log] Dict: inserts/updates/deletions: 21553295/0/0 07:01:42-08/07 zebraidx(314) [log] Records: 702954 i/u/d 702954/0/0 07:01:42-08/07 zebraidx(314) [log] zebra_stop: 20557.86 6312.50 1447.43 Search works on every biblio, except ones after 702955. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha-devel mailing list Koha-devel(a)lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel

5 4

rebuild_zebra and a limit at 700 000 biblios
by Paul Poulain 15 Jul '10

15 Jul '10

Hello, We have a database of 726k biblios and 1.2M items. When we rebuild_zebra, everything is OK on the export. Then, the rebuild_zebra seems OK as well, except that only 702k biblios are indexed. We don't have any error, everything seems OK. Does anyone have seen this problem ? Logs : exporting biblio ==================== Records exported: 726210 <snip> 06:43:19-08/07 zebraidx(314) [log] Records: 700000 i/u/d 700000/0/0 06:43:29-08/07 zebraidx(314) [log] Records: 701000 i/u/d 701000/0/0 06:43:38-08/07 zebraidx(314) [log] Records: 702000 i/u/d 702000/0/0 07:42:38-08/07 zebraidx(314) [log] Merge 0% <snip> 07:00:52-08/07 zebraidx(314) [log] Merge 95.4% completed; 48 seconds remaining 07:01:02-08/07 zebraidx(314) [log] Merge 96.4% completed; 37 seconds remaining 07:01:12-08/07 zebraidx(314) [log] Merge 97.4% completed; 26 seconds remaining 07:01:22-08/07 zebraidx(314) [log] Merge 98.7% completed; 13 seconds remaining 07:01:32-08/07 zebraidx(314) [log] Merge 99.9% completed; 0 seconds remaining 07:01:38-08/07 zebraidx(314) [log] Iterations: isam/dict 301420236/21553295 07:01:38-08/07 zebraidx(314) [log] Dict: inserts/updates/deletions: 21553295/0/0 07:01:42-08/07 zebraidx(314) [log] Records: 702954 i/u/d 702954/0/0 07:01:42-08/07 zebraidx(314) [log] zebra_stop: 20557.86 6312.50 1447.43 Search works on every biblio, except ones after 702955. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

3 2

Image/Graphics Magick confusion
by Colin Campbell 15 Jul '10

15 Jul '10

Hi, just noticed that the debian package lists and scripts in install-misc install libimage-magick-perl but the koha dependency is for Graphics::Magick (libgraphics-magick-perl). Willing to generate a patch for that but wondered if there are any advantages in using Graphics Magick rather than Image Magick (as imagemagick has the advantage that the perl module can be installed from CPAN ) Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 208 366 1295 (phone) +44 (0) 7759 633626 (mobile) colin.campbell(a)ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com

3 4

Official Koha Newsletter : Volume 1, Issue 7: July 2010
by Nicole Engard 15 Jul '10

15 Jul '10

Read this newsletter online with live links: http://koha-community.org/koha-newsletter-volume-1issue-7-july-2010/ Official Koha Newsletter (ISSN 2153-8328) Volume 1, Issue 7: July 2010 Table of Contents * Tips & Tricks o Search in Koha o Amazon Content Not Appearing? * Koha in Libraries o Koha Users Forum launched in Nigeria o Archive & Access Project * Koha Events o Second Koha Open Day in London o KohaCon 2010 Program Available Tips & Tricks Search in Koha by Marilen Aretius Corciovei For people used to web searches and database oriented application Koha way of dealing with searches (since v.3) might seem a bit alien. This is a small set of notes related to some not very obvious questions which I had to answer in order to do some configurations which seemed trivial at the beginning. I am not saying this is wrong or ineffective, just different and different means time to learn. I’m writing this to help you reduce this time. [Read Part 1 & Part 2.] Amazon Content Not Working? by Nicole C. Engard Question: Why is Amazon’s enhanced content not appearing? Answer: Amazon uses your server time in its API requests and so if your system time is off you will not be able to use Amazon content. Update your server time and make sure all of your Amazon preferences are turned on, this should solve the problem. Koha in Libraries Koha Users Forum launched in Nigeria by Olugbenga Adara Projektlink Konsult Limited, the premier Koha support company in Nigeria hosted the first ever Koha Users Forum at its premises in Ibadan, Nigeria on Monday June 21, 2010. The event, which had 22 participants from 12 different institutions using Koha in Nigeria, featured tutorials, tips and tricks and discussions on challenges and solutions to problems encountered by the participants while running Koha. More details are available here. Archive & Access Project by Savitra Sirohi The Archive and Access project aims to set up a consortium of archives and libraries in India with the purpose of catalog sharing and eventual sharing of full text digitized resources. The project is supported by the Jamsetji Tata Trust and housed at the Centre or the Study of Culture and Society, Bangalore. This will be a union catalog, powered by Koha, of historical collections of libraries and archives from different parts of the India. This is an ongoing process, with the regular introduction of new institutions to the catalog. The project wishes to solicit contributions from libraries, organizations, historians and other scholars for this catalog. Those interested in participating can contact project members at: publicarchivesindia(a)gmail.com and more information can be found on the official page. Koha Events Second Koha Open Day in London by Andrea Chandler The Kings Fund and CAMLIS are pleased to announce that we will be holding another free open day for people interested in finding more about implementing or switching to Koha. All Koha users are also welcome. The event will be held in the afternoon of Friday 10th of September at CAMLIS, from 2pm-5pm. At the moment we don’t have a strict schedule but we hope to include some short presentations with discussion, followed by hands-on experience in small groups at the library computers. There will be some time for existing Koha users to chat with each other. If any user is interested in giving a 10 minute presentation or demonstration of some aspect of Koha, please contact one of us. Places for this event are limited so please book early by contacting Matthew Hale at the King’s Fund: m.hale(a)kingsfund.org.uk. If there is sufficient interest we will repeat it at a later date, so do register your interest even if you can’t necessarily make this day. KohaCon 2010 Program Available by Nicole C. Engard As we get closer to the next annual KohaCon (this year in NZ – where it all started) it’s time to check out the program for the event! We are still gathering speaker bios and program description updates, but the preliminary program is available on the conference site. Newsletter edited by Nicole C. Engard, Koha Documentation Manager. Please send future story ideas to me.

1 0

Moving system preference serialization into the database
by Chris Nighswonger 13 Jul '10

13 Jul '10

While doing some beginning work on a local syspref editor to fit in with the new system preference interface, I have run into the file permission "problem" of writing to files inside Koha's template directory structure. This is due to the fact that the system preferences are serialized in yaml format and stored in *.pref files within the template directory structure. While the problem of writing to files is one which needs to be addressed, it is not one that we have time to address before the 3.2 stable release. As regards the system preference issue in particular, the problem may be better resolved by moving the serialization into a table in the database. As it stands presently, part of the system preference data is kept in the database in the systempreferences table while the other part is kept in the *.pref files. Moving the serialization into the database will have the benefits of keeping the data in one location as well as overcoming the file writing issues. So, are there any serious objections to making this change? Kind Regards, Chris

7 16

#koha IRC meeting today about voting on software license
by Thomas Dukleth 13 Jul '10

13 Jul '10

Please remember that the #koha IRC meeting about voting on whether to upgrade the copyright license for Koha 3.4 will beheld today on the #koha IRC channel, 13 July 2010 at 19:00 UTC+0. Time Converter: http://tinyurl.com/2eaohr4 Agenda: http://wiki.koha-community.org/wiki/License_Upgrade_Vote_IRC_Meeting,_13_Ju… Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

1 0

ISBN search
by Fridolyn SOMERS 12 Jul '10

12 Jul '10

Hie, I noticed that when performing a search on ISBN index, search is made on any index. The query that appears on result page is "nb: test", but the query performed by Zebra server is : "@attrset Bib-1 test". It should be : "@attrset Bib-1 @att 1=7 test" It comes from C4/Search.pm : When index 'nb' detected, the flag $indexes_set is set to 1. That means that index mus't be added to query. Is is a normal behaviour ? -- Fridolyn SOMERS Information and Communication Technologies engineer fridolyn.somers(a)gmail.com

2 4

QueryWeightFields in Koha 3.2
by Fridolyn SOMERS 12 Jul '10

12 Jul '10

Hie, Are you sure about "wrdlist", isn't it "wrdl" instead ? Look for it in zebradb/ccl.properties. Could you post the zebra server query : "@attrset Bib-1 @att ..." ? Regards. -- Fridolyn SOMERS Information and Communication Technologies engineer fridolyn.somers(a)gmail.com Hi all, > > While testing Koha 3.01.00.142 (master in dev mode) with Zebra/MARC21, I > notice that searching words without indexes all have 0 hits, when I have > disabled QueryWeightFields. > Does anyone of you already know this bug? > I did not find a report in Bugzilla (only opposite behavior in Unimarc). > > If I use an index in the search expression like kw=vermeer then the results > are correct. > > Note that Koha adds kw, wrdlist: to the search expression when I did not > pass any index. The kw,wrdlist stuff apparently is passed to Zebra/YAZ and > results in no hits. > > Regards, > > Marcel

3 3

Re: [Koha-devel] QueryWeightFields in Koha 3.2
by Fridolyn SOMERS 12 Jul '10

12 Jul '10

Is "kw" present in ccl.properties ? trie direct ccl by editing url : .../opac-search.pl?q=ccl=kw,wrld=rembrandt Does it work ? On Mon, Jul 12, 2010 at 12:13 PM, Marcel de Rooy <M.de.Rooy(a)rijksmuseum.nl>wrote: > 12:08:43-12/07 zebrasrv(38) [request] Search biblios OK 0 1 1+0 RPN > @attrset Bib-1 @and @and kw wrdl: rembrandt > > > > *Van:* Fridolyn SOMERS [mailto:fridolyn.somers@gmail.com] > *Verzonden:* maandag 12 juli 2010 12:00 > *Aan:* Marcel de Rooy > *CC:* koha-devel(a)lists.koha-community.org > *Onderwerp:* Re: QueryWeightFields in Koha 3.2 > > > > Could you post the zebra server query : "@attrset Bib-1 @att ..." ? > > On Mon, Jul 12, 2010 at 9:55 AM, Marcel de Rooy <M.de.Rooy(a)rijksmuseum.nl> > wrote: > > Yes, I meant wrdl. Sorry for the confusion.. > > Searching with yaz-client works fine. > > > > *Van:* Fridolyn SOMERS [mailto:fridolyn.somers@gmail.com] > *Verzonden:* vrijdag 9 juli 2010 12:37 > *Aan:* koha-devel(a)lists.koha-community.org > *CC:* Marcel de Rooy > *Onderwerp:* QueryWeightFields in Koha 3.2 > > > > Hie, > > Are you sure about "wrdlist", isn't it "wrdl" instead ? > Look for it in zebradb/ccl.properties. > > Could you post the zebra server query : "@attrset Bib-1 @att ..." ? > > Regards. > > -- > Fridolyn SOMERS > Information and Communication Technologies engineer > fridolyn.somers(a)gmail.com > > Hi all, > > While testing Koha 3.01.00.142 (master in dev mode) with Zebra/MARC21, I > notice that searching words without indexes all have 0 hits, when I have > disabled QueryWeightFields. > Does anyone of you already know this bug? > I did not find a report in Bugzilla (only opposite behavior in Unimarc). > > If I use an index in the search expression like kw=vermeer then the results > are correct. > > Note that Koha adds kw, wrdlist: to the search expression when I did not > pass any index. The kw,wrdlist stuff apparently is passed to Zebra/YAZ and > results in no hits. > > Regards, > > Marcel > > > > > -- > Fridolyn SOMERS > Information and Communication Technologies engineer > +33(0)6.86.63.77.68 > fridolyn.somers(a)gmail.com > -- Fridolyn SOMERS Information and Communication Technologies engineer +33(0)6.86.63.77.68 fridolyn.somers(a)gmail.com

1 0

Biblionumber field auto fill
by Fridolyn SOMERS 09 Jul '10

09 Jul '10

Hie, I had a problem with a biblio framework where field corresponding to biblionumber was visible (090$9) : In this case, editing a record and saving it made a record with 2 fields 090. It comes from the fact that this field is always added by Biblio->TransformHtmlToMarc(), meaning it shouldn't be visible. It does work when this field is hidden in framework. I also noticed that this field is set again by Biblio->_koha_marc_update_bib_ids(). Is it a bug ? -- Fridolyn SOMERS Information and Communication Technologies engineer fridolyn.somers(a)gmail.com

1 0