Happy Thanksgiving Day to Canadians and good evening to you all; Logging the zebra incrementals, Ubuntu 12.04 seems to be a tad touchy on this subject, so I'm now using user=koha crontab -e for this. I had problems with functionality of recommended envars and had highcoughs for a "user" cron if 'koha' appeared on the exec line: * * * * * koha $KOHA_PATH/bin/migration_tools/rebuild_zebra.pl / ^^^^ -a -b -z -v >> /tmp/cron_koha.log koha@nelson:/tmp$ cat cron_koha.log /bin/sh: 1: koha: not found) but the following gave results: koha@nelson:$ crontab -l KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib * * * * * /usr/share/koha/bin/migration_tools/rebuild_zebra.pl -a -b -z -v >> /tmp/cron_koha.log 2>&1 Question: to tidy up the [warn]s -- does anyone please have any pointers? Here's an extract: Zebra configuration information ================================ Zebra biblio directory = /var/lib/koha/zebradb/biblios Zebra authorities directory = /var/lib/koha/zebradb/authorities Koha directory = /usr/share/koha/intranet/cgi-bin BIBLIONUMBER in : 999$c BIBLIOITEMNUMBER in : 999$d ================================ ==================== exporting authority ==================== Records exported: 0 1. {what is this? not line wrap? Just the first of many lines?} Records exported: 4 ==================== REINDEXING zebra ==================== ==================== exporting biblio ==================== Records exported: 0 1.................. {same? {what is this? not line wrap? Just the first of many lines?}} Records exported: 38 ==================== REINDEXING zebra ==================== 18:13:02-08/10 zebraidx(5501) [warn] Missing attribute 'type' in attribute info 18:13:02-08/10 zebraidx(5501) [warn] Missing attribute 'type' in attribute info 18:13:02-08/10 zebraidx(5501) [warn] Unknown register type: ==================== CLEANING ==================== Thanks and best regards -- Paul
Paul schreef op ma 08-10-2012 om 19:18 [-0400]:
problems with functionality of recommended envars and had highcoughs for a "user" cron if 'koha' appeared on the exec line:
* * * * * koha $KOHA_PATH/bin/migration_tools/rebuild_zebra.pl / ^^^^ -a -b -z -v >> /tmp/cron_koha.log
This is because you don't specify a username in a user crontab (it doesn't make sense), you only do this in the global ones. See 'man 5 crontab' for details.
1.................. {same? {what is this? not line wrap? Just the
It's because you have '-v' for verbose in the command line, and so it's telling you its progress. You should only do that when you're running it interactively, otherwise you'll get piles of emails. I'd figure out what records are causing you problems. There are tools around that do this. This is one that could help: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7286 Note that I've not used it. There's another one that talks to zebra to find out what isn't there and tells you about them, but I can't find it with a quick search. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
At 01:42 PM 10/9/2012 +1300, Robin Sheat wrote:
Paul schreef op ma 08-10-2012 om 19:18 [-0400]: [snip] interactively, otherwise you'll get piles of emails.
Thanks Robin, you're quite right of course. In fact, I'm only using these rather non-standard methods as a temp measure for testing the sandbox 3.8.5 -- I even tried it with a double " -v " to increase it's verbosity and the file size is not too bad (and the sandbox has no email capability at this time). Sorry I wasn't more clear in my first email.
I'd figure out what records are causing you problems. There are tools around that do this.
This is one that could help: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7286 Note that I've not used it.
Thanks for that. I'll dig a bit deeper later on, as I must be missing a subtlety: it appears to run the "standard" rebuild-zebra in chunks, with output and logging separate. From the bug, it seems to be designed to locate failures within rebuild-zebra. My "standard" reindexing completes every time (full and incremental) with just the three "[warn]s" but not failures, and the output from this tool *apparently* doesn't record them as such -- grep -i "\[warn\]" gives nothing in either the record or the log files -- but maybe there's other messages that will help. koha@server:/usr/share/koha/bin/migration_tools$ ./rebuild_zebra_sliced.sh -o 0 -l 20000 -s 1000 -t 'authorities' [snip ...] CLEANING ==================== NOTHING cleaned : the export /tmp/rebuild/export has been kept. You can re-run this script with the -s parameter if you just want to rebuild zebra after changing the record.abs or another zebra config file Splitting file in chunks of 1000 records Indexing /tmp/rebuild/export/authority/exported_records_1000_0000000 [snip] Indexing /tmp/rebuild/export/authority/exported_records_1000_0000016
There's another one that talks to zebra to find out what isn't there and tells you about them, but I can't find it with a quick search.
If anyone remembers what that is, I would really appreciate. Again thanks and best regards - Paul
Greetings,
There's another one that talks to zebra to find out what isn't there and tells you about them, but I can't find it with a quick search.
If anyone remembers what that is, I would really appreciate.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6566 Assuming you have placed the script in /usr/share/koha/bin/migration_tools/checkNonIndexedBiblios.pl and that the user you (or the packages) created for indexing is called library-koha: $ sudo -i # su - library-koha $ echo $KOHA_CONF [this should be set to the proper value] $ echo $PERL5LIB [this should be set to the proper value] $ /usr/share/koha/bin/migration_tools/checkNonIndexedBiblios.pl -c -z In our migration from tarball to packages, we had indexing problems. A full reindex said there was a malformed tag, but I couldn't find it. I started this at night and went to bed: $ while true; do /usr/share/koha/bin/migration_tools/checkNonIndexedBiblios.pl -c -z; sleep 360; done In the morning everything was properly indexed (0 records added), so I just had to kill the process with a CTRL-C. It works knowing that the zebra indexing is triggered automatically every 5 minutes in packages. So, by sleeping 6 minutes, that let's at least some indexing happen before retrying. This, of course, will bloat the zebra queue table, but the packages have a script to automatically clean that up already installed. I tried the sliced reindexing and it just messed up our indexes again. So, this is the method I'm sticking with. Your mileage may vary. GPML, Mark Tompsett
participants (3)
-
Mark Tompsett -
Paul -
Robin Sheat