Paul, I find that around 70,000 biblios starts to push th 4GB limit, so 700,000 may start to push the 40GB limit, but from what you've pasted, you're not in that situation (there'd be a warn). Do you have enough hard drive space total for the exported files, register and shadow? Maybe it's that you can't export all the files? This is a shot in the dark, really. -Ian 2010/7/15 Paul Poulain <paul.poulain@biblibre.com>
Scott,
I already have : register: /home/koha/var/lib/zebradb/biblios/register:40G shadow: /home/koha/var/lib/zebradb/biblios/shadow:40G
or I'm looking at the wrong file... double check ... unless i'm blind (or really too tired), I have 40G, so that should be enough :\
-------- Message original -------- Sujet: RE: [Koha-devel] rebuild_zebra and a limit at 700 000 biblios Date : Thu, 15 Jul 2010 17:17:16 -0400 De : Scott Kushner <skushner@mtpl.org> <skushner@mtpl.org> Pour : Paul Poulain <paul.poulain@biblibre.com> <paul.poulain@biblibre.com>
Paul, I think you have to increase your memory buffer in biblios.cfg to 8G or more. I had that problem.
# Lock File Area lockDir: /root/koha-dev/var/lock/zebradb/biblios perm.anonymous:ar perm.root:rw passwd: /root/koha-dev/etc/zebradb/etc/passwd register: /root/koha-dev/var/lib/zebradb/biblios/register:8G shadow: /root/koha-dev/var/lib/zebradb/biblios/shadow:8G
# Temp File area for result sets setTmpDir: /root/koha-dev/var/lib/zebradb/biblios/tmp
# Temp File area for index program keyTmpDir: /root/koha-dev/var/lib/zebradb/biblios/key
# Approx. Memory usage during indexing memMax: 150M rank:rank-1 truncmax: 1000000000
Scott Kushner Information Technologies Middletown Public Library
-----Original Message----- From: koha-devel-bounces@lists.koha-community.org
[mailto:koha-devel-bounces@lists.koha-community.org <koha-devel-bounces@lists.koha-community.org>] On Behalf Of Paul Poulain Sent: Thursday, July 15, 2010 5:11 PM To: koha-devel@lists.koha-community.org Subject: [Koha-devel] rebuild_zebra and a limit at 700 000 biblios
Hello,
We have a database of 726k biblios and 1.2M items. When we rebuild_zebra, everything is OK on the export. Then, the rebuild_zebra seems OK as well, except that only 702k biblios are indexed. We don't have any error, everything seems OK.
Does anyone have seen this problem ?
Logs : exporting biblio ==================== Records exported: 726210 <snip> 06:43:19-08/07 zebraidx(314) [log] Records: 700000 i/u/d 700000/0/0 06:43:29-08/07 zebraidx(314) [log] Records: 701000 i/u/d 701000/0/0 06:43:38-08/07 zebraidx(314) [log] Records: 702000 i/u/d 702000/0/0 07:42:38-08/07 zebraidx(314) [log] Merge 0% <snip> 07:00:52-08/07 zebraidx(314) [log] Merge 95.4% completed; 48 seconds remaining 07:01:02-08/07 zebraidx(314) [log] Merge 96.4% completed; 37 seconds remaining 07:01:12-08/07 zebraidx(314) [log] Merge 97.4% completed; 26 seconds remaining 07:01:22-08/07 zebraidx(314) [log] Merge 98.7% completed; 13 seconds remaining 07:01:32-08/07 zebraidx(314) [log] Merge 99.9% completed; 0 seconds remaining 07:01:38-08/07 zebraidx(314) [log] Iterations: isam/dict 301420236/21553295 07:01:38-08/07 zebraidx(314) [log] Dict: inserts/updates/deletions: 21553295/0/0 07:01:42-08/07 zebraidx(314) [log] Records: 702954 i/u/d 702954/0/0 07:01:42-08/07 zebraidx(314) [log] zebra_stop: 20557.86 6312.50 1447.43
Search works on every biblio, except ones after 702955.
-- Paul POULAINhttp://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
_______________________________________________ Koha-devel mailing listKoha-devel@lists.koha-community.orghttp://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal