[Koha-devel] strange recent indexing problems on koha 3.0.x branch, anyone?

[Mason James]

Hi All

The basic issue seems to be that the latest koha 3.0.x branch has  
some strange indexing (and encoding?) errors (for me at least)

i have spotted these 'Invalid XML encoding name' errors when running  
the  zebraqueue_daemon.pl / koha-zebraqueue-ctl.sh scripts
- after adding a new bib via...

* a z39-pull
* web-gui batch-import
* cmdline bulkmarcimport.pl script

i can replicate this on 2 separate debian-etch boxes

rolling my repo back from the latest 3.0.x commit, to the 'v3.00.00'  
tag fixes the issue (which is good, but i havent clicked to what the  
issue actually is yet ;/ )

the best way to replicate the indexing error for me, was this...

fetch, and rewind the repo
$ git-fetch
$ git-fetch --tags             #needed too ??
$ git rebase origin/3.0.x
$ git reset --hard v3.00.00

$ ./misc/migration_tools/bulkmarcimport.pl -d -v -fk -n 10   -file / 
tmp/sample.mrc
deleting biblios
..........
10 MARC records done in 0.157172918319702 seconds

$ ./misc/bin/zebraqueue_daemon.pl  -f ~/etc/koha-conf.xml

22:52:52-05/01 zebrasrv(17) [session] Session - OK 17 unix:/home/ 
mason/koha/ashs/var/run/zebradb/bibliosocket 26837
22:52:52-05/01 zebrasrv(17) [request] Auth idPass kohauser -
22:52:52-05/01 zebrasrv(17) [request] Init OK - ID:81/81 Name:ZOOM-C/ 
YAZ Version:1.116/2.1.54
22:52:52-05/01 zebrasrv(17) [log] function: 1
22:52:52-05/01 zebrasrv(17) [log] Waitaction: 2
22:52:52-05/01 zebrasrv(17) [log] Received DB Update
22:52:52-05/01 zebrasrv(17) [log] action
22:52:52-05/01 zebrasrv(17) [log] specialUpdate
22:52:52-05/01 zebrasrv(17) [log] database: biblios
22:52:52-05/01 zebrasrv(17) [log] enabling shadow spec=/home/mason/ 
koha/ashs/var/lib/zebradb/biblios/shadow:4G
22:52:52-05/01 zebrasrv(17) [log] cache_fname = /home/mason/koha/ashs/ 
var/lib/zebradb/biblios/shadow/cache
22:52:52-05/01 zebrasrv(17) [log] record 0 type XML
22:52:52-05/01 zebrasrv(17) [log] 2095 bytes:
<?xml version="1.0" encoding="UTF-8"?>
<record
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://w ...
22:52:52-05/01 zebrasrv(17) [log] update grs.xml <no file> 0
22:52:52-05/01 zebrasrv(17) [log] Iterations: isam/dict 8/2
22:52:52-05/01 zebrasrv(17) [log] Dict: inserts/updates/deletions: 1/0/1
22:52:52-05/01 zebrasrv(17) [log] Records: 1 i/u/d 0/1/0
22:52:52-05/01 zebrasrv(17) [request] Extended Service: Update (done)
22:52:52-05/01 zebrasrv(17) [log] function: 1
22:52:52-05/01 zebrasrv(17) [log] Waitaction: 2
22:52:52-05/01 zebrasrv(17) [log] adm request database biblios
22:52:52-05/01 zebrasrv(17) [log] adm-commit
...

SUCCESS!!!!!!!!!!!!!!!!!!!!

----------------------------------------------------

roll the branch forward...
$ git rebase origin/3.0.x

$ ~/git/stable/misc/migration_tools/bulkmarcimport.pl -d -v -fk -n  
10   -file /tmp/sample.mrc
deleting biblios
..........
10 MARC records done in 0.157172918319702 seconds

$ ./misc/bin/zebraqueue_daemon.pl  -f ~/koha/ashs/etc/koha-conf.xml &

$ sudo tail /var/log/syslog

Jan  5 23:58:35 fugu Koha Zebraqueue [27085]: Starting Zebraqueue log  
at Mon Jan  5 23:58:35 2009
Jan  5 23:58:35 fugu Koha Zebraqueue [27085]: Mon Jan  5 23:58:35  
2009 POE Session
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: 10 operations waiting  
to be run
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: Executing zebra operations
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 10  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 9  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 8  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 7  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 6  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 5  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 4  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 3  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 2  
is malformed: Entity: line 1: parser error : Invalid XML encoding name
Jan  5 23:58:36 fugu Koha Zebraqueue [27085]: biblioserver record 1  
is malformed: Entity: line 1: parser error : Invalid XML encoding name

FAIL !!!!!!!!!!!!!!!!!!!!


any ideas, can anyone else replicate this, or confirm otherwise?


Cheers, Mason