strange recent indexing problems on koha 3.0.x branch, anyone?
Hi All The basic issue seems to be that the latest koha 3.0.x branch has some strange indexing (and encoding?) errors (for me at least) i have spotted these 'Invalid XML encoding name' errors when running the zebraqueue_daemon.pl / koha-zebraqueue-ctl.sh scripts - after adding a new bib via... * a z39-pull * web-gui batch-import * cmdline bulkmarcimport.pl script i can replicate this on 2 separate debian-etch boxes rolling my repo back from the latest 3.0.x commit, to the 'v3.00.00' tag fixes the issue (which is good, but i havent clicked to what the issue actually is yet ;/ ) the best way to replicate the indexing error for me, was this... fetch, and rewind the repo $ git-fetch $ git-fetch --tags #needed too ?? $ git rebase origin/3.0.x $ git reset --hard v3.00.00 $ ./misc/migration_tools/bulkmarcimport.pl -d -v -fk -n 10 -file / tmp/sample.mrc deleting biblios .......... 10 MARC records done in 0.157172918319702 seconds $ ./misc/bin/zebraqueue_daemon.pl -f ~/etc/koha-conf.xml 22:52:52-05/01 zebrasrv(17) [session] Session - OK 17 unix:/home/ mason/koha/ashs/var/run/zebradb/bibliosocket 26837 22:52:52-05/01 zebrasrv(17) [request] Auth idPass kohauser - 22:52:52-05/01 zebrasrv(17) [request] Init OK - ID:81/81 Name:ZOOM-C/ YAZ Version:1.116/2.1.54 22:52:52-05/01 zebrasrv(17) [log] function: 1 22:52:52-05/01 zebrasrv(17) [log] Waitaction: 2 22:52:52-05/01 zebrasrv(17) [log] Received DB Update 22:52:52-05/01 zebrasrv(17) [log] action 22:52:52-05/01 zebrasrv(17) [log] specialUpdate 22:52:52-05/01 zebrasrv(17) [log] database: biblios 22:52:52-05/01 zebrasrv(17) [log] enabling shadow spec=/home/mason/ koha/ashs/var/lib/zebradb/biblios/shadow:4G 22:52:52-05/01 zebrasrv(17) [log] cache_fname = /home/mason/koha/ashs/ var/lib/zebradb/biblios/shadow/cache 22:52:52-05/01 zebrasrv(17) [log] record 0 type XML 22:52:52-05/01 zebrasrv(17) [log] 2095 bytes: <?xml version="1.0" encoding="UTF-8"?> <record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://w ... 22:52:52-05/01 zebrasrv(17) [log] update grs.xml <no file> 0 22:52:52-05/01 zebrasrv(17) [log] Iterations: isam/dict 8/2 22:52:52-05/01 zebrasrv(17) [log] Dict: inserts/updates/deletions: 1/0/1 22:52:52-05/01 zebrasrv(17) [log] Records: 1 i/u/d 0/1/0 22:52:52-05/01 zebrasrv(17) [request] Extended Service: Update (done) 22:52:52-05/01 zebrasrv(17) [log] function: 1 22:52:52-05/01 zebrasrv(17) [log] Waitaction: 2 22:52:52-05/01 zebrasrv(17) [log] adm request database biblios 22:52:52-05/01 zebrasrv(17) [log] adm-commit ... SUCCESS!!!!!!!!!!!!!!!!!!!! ---------------------------------------------------- roll the branch forward... $ git rebase origin/3.0.x $ ~/git/stable/misc/migration_tools/bulkmarcimport.pl -d -v -fk -n 10 -file /tmp/sample.mrc deleting biblios .......... 10 MARC records done in 0.157172918319702 seconds $ ./misc/bin/zebraqueue_daemon.pl -f ~/koha/ashs/etc/koha-conf.xml & $ sudo tail /var/log/syslog Jan 5 23:58:35 fugu Koha Zebraqueue [27085]: Starting Zebraqueue log at Mon Jan 5 23:58:35 2009 Jan 5 23:58:35 fugu Koha Zebraqueue [27085]: Mon Jan 5 23:58:35 2009 POE Session Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: 10 operations waiting to be run Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: Executing zebra operations Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 10 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 9 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 8 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 7 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 6 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 5 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 4 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 3 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 2 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 1 is malformed: Entity: line 1: parser error : Invalid XML encoding name FAIL !!!!!!!!!!!!!!!!!!!! any ideas, can anyone else replicate this, or confirm otherwise? Cheers, Mason
On 2009/01/6, at 12:30 AM, Mason James wrote:
Hi All
The basic issue seems to be that the latest koha 3.0.x branch has some strange indexing (and encoding?) errors (for me at least)
Jan 5 23:58:35 fugu Koha Zebraqueue [27085]: Starting Zebraqueue log at Mon Jan 5 23:58:35 2009 Jan 5 23:58:35 fugu Koha Zebraqueue [27085]: Mon Jan 5 23:58:35 2009 POE Session Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: 10 operations waiting to be run Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: Executing zebra operations Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 10 is malformed: Entity: line 1: parser error : Invalid XML encoding name Jan 5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 9 is malformed: Entity: line 1: parser error : Invalid XML encoding name
any ideas, can anyone else replicate this, or confirm otherwise?
Just a followup to my own previous post... I re-updated my problem box with the preferred debian packages, rather than the latest random CPAN builds and my 3.0.x HEAD indexing glitch disappeared :) $ sudo dpkg --set-selections < ./koha3.git/install_misc/debian.packages cheers to the debian-packages people for saving my ass again ;) Mason.
Hi, Paul were you having another crack at the MARC Framework editor as well at the moment? or someone else? Having just spent an hour or 2 working with it and giving up because it's just tooo much clicking to delete all the fields you don't need, we'll resort to a "programmer" method. However, it would be useful to be able to do 2 things 1/. Create a new framework based on either a "blank" or preferably, with only the fields that are mandatory for Koha to work, so that you just add in any fields you want, (usually about 20) rather than deleting all the ones you don't (about 900!) 2/. Be about to do a bulk delete on fields - ie click all the ones you want to delete and get rid of them in one go per page, rather than each delete taking 2-3 clicks. Cheers Rachel
Rachel Hamilton-Williams a écrit :
Hi,
Hi Rach,
Paul were you having another crack at the MARC Framework editor as well at the moment? or someone else?
not me (and in fact, if you look at gitstats, you'll see I don't commit a lot those months...)
Having just spent an hour or 2 working with it and giving up because it's just tooo much clicking to delete all the fields you don't need, we'll resort to a "programmer" method.
However, it would be useful to be able to do 2 things
1/. Create a new framework based on either a "blank" or preferably, with only the fields that are mandatory for Koha to work, so that you just add in any fields you want, (usually about 20) rather than deleting all the ones you don't (about 900!)
A suggestion here : DELETE FROM marc_subfield_structure WHERE tab=-1 AND kohafield='' AND frameworkcode='WHATEVER_YOU_WANT' should delete all unused subfields in a given framework. then DELETE FROM marc_tag_structure WHERE (SELECT count(*) FROM marc_subfield_structure WHERE marc_subfield_structure.tag=marc_tag_structure.field)=0 UPDATE marc_subfield_structure SET tab=-1 WHERE frameworkcode='WHATEVER' will "hide" every fields, you can then have the few you need reappear, and do the previous DELETE to clean things. Note : previous SQL untested, it's basically the idea. Once this is done, you can export marc_subfield_structure and marc_tag_structure, and get a SQL file quite interesting to have a simple cataloguing framework. In UNIMARC, when you install Koha, you have the choice between 2 UNIMARC flavours : "complete MARC frameworks" or "public library MARC framework" (with much less fields !) We did them with the previous commands. It was boring to do the 1st time, but now, it's done !!! -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
participants (3)
-
Mason James -
paul POULAIN -
Rachel Hamilton-Williams