[Koha-devel] Fwd: RE: rebuild_zebra and a limit at 700 000 biblios

[Ian Walls]

Paul,


I find that around 70,000 biblios starts to push th 4GB limit, so 700,000
may start to push the 40GB limit, but from what you've pasted, you're not in
that situation (there'd be a warn).

Do you have enough hard drive space total for the exported files, register
and shadow?  Maybe it's that you can't export all the files?  This is a shot
in the dark, really.


-Ian

2010/7/15 Paul Poulain <paul.poulain at biblibre.com>

>  Scott,
>
> I already have :
> register: /home/koha/var/lib/zebradb/biblios/register:40G
> shadow: /home/koha/var/lib/zebradb/biblios/shadow:40G
>
> or I'm looking at the wrong file...
> double check ...
> unless i'm blind (or really too tired), I have 40G, so that should be
> enough :\
>
> -------- Message original --------  Sujet: RE: [Koha-devel] rebuild_zebra
> and a limit at 700 000 biblios  Date : Thu, 15 Jul 2010 17:17:16 -0400  De :
> Scott Kushner <skushner at mtpl.org> <skushner at mtpl.org>  Pour : Paul Poulain
> <paul.poulain at biblibre.com> <paul.poulain at biblibre.com>
>
> Paul,
> I think you have to increase your memory buffer in biblios.cfg to 8G or
> more. I had that problem.
>
> # Lock File Area
> lockDir: /root/koha-dev/var/lock/zebradb/biblios
> perm.anonymous:ar
> perm.root:rw
> passwd: /root/koha-dev/etc/zebradb/etc/passwd
> register: /root/koha-dev/var/lib/zebradb/biblios/register:8G
> shadow: /root/koha-dev/var/lib/zebradb/biblios/shadow:8G
>
> # Temp File area for result sets
> setTmpDir: /root/koha-dev/var/lib/zebradb/biblios/tmp
>
> # Temp File area for index program
> keyTmpDir: /root/koha-dev/var/lib/zebradb/biblios/key
>
> # Approx. Memory usage during indexing
> memMax: 150M
> rank:rank-1
> truncmax: 1000000000
>
>
> Scott Kushner
> Information Technologies
> Middletown Public Library
>
> -----Original Message-----
> From: koha-devel-bounces at lists.koha-community.org
>
> [mailto:koha-devel-bounces at lists.koha-community.org <koha-devel-bounces at lists.koha-community.org>] On Behalf Of Paul
> Poulain
> Sent: Thursday, July 15, 2010 5:11 PM
> To: koha-devel at lists.koha-community.org
> Subject: [Koha-devel] rebuild_zebra and a limit at 700 000 biblios
>
> Hello,
>
> We have a database of 726k biblios and 1.2M items.
> When we rebuild_zebra, everything is OK on the export.
> Then, the rebuild_zebra seems OK as well, except that only 702k biblios
> are indexed.
> We don't have any error, everything seems OK.
>
> Does anyone have seen this problem ?
>
> Logs :
> exporting biblio
> ====================
> Records exported: 726210
> <snip>
> 06:43:19-08/07 zebraidx(314) [log] Records: 700000 i/u/d 700000/0/0
> 06:43:29-08/07 zebraidx(314) [log] Records: 701000 i/u/d 701000/0/0
> 06:43:38-08/07 zebraidx(314) [log] Records: 702000 i/u/d 702000/0/0
> 07:42:38-08/07 zebraidx(314) [log] Merge 0% <snip>
> 07:00:52-08/07 zebraidx(314) [log] Merge 95.4% completed; 48 seconds
> remaining
> 07:01:02-08/07 zebraidx(314) [log] Merge 96.4% completed; 37 seconds
> remaining
> 07:01:12-08/07 zebraidx(314) [log] Merge 97.4% completed; 26 seconds
> remaining
> 07:01:22-08/07 zebraidx(314) [log] Merge 98.7% completed; 13 seconds
> remaining
> 07:01:32-08/07 zebraidx(314) [log] Merge 99.9% completed; 0 seconds
> remaining
> 07:01:38-08/07 zebraidx(314) [log] Iterations: isam/dict
> 301420236/21553295
> 07:01:38-08/07 zebraidx(314) [log] Dict: inserts/updates/deletions:
> 21553295/0/0
> 07:01:42-08/07 zebraidx(314) [log] Records: 702954 i/u/d 702954/0/0
> 07:01:42-08/07 zebraidx(314) [log] zebra_stop: 20557.86 6312.50 1447.43
>
> Search works on every biblio, except ones after 702955.
>
>
> --
> Paul POULAINhttp://www.biblibre.com
> Expert en Logiciels Libres pour l'info-doc
> Tel : (33) 4 91 81 35 08
>
> _______________________________________________
> Koha-devel mailing listKoha-devel at lists.koha-community.orghttp://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>



-- 
Ian Walls
Lead Development Specialist
ByWater Solutions
Phone # (888) 900-8944
http://bywatersolutions.com
ian.walls at bywatersolutions.com
Twitter: @sekjal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20100715/e5bda2d1/attachment-0001.htm>