[Original subject: Re: [Koha-devel] Solr and Z3950 server some news.] 1. YAZ CONFIGURATION. SimpleServer, like all IndexData products is dependent upon YAZ. My thought on the bad MARC error from testing BibLibre work on SimpleServer as a Z39.50/SRU server using Solr/Lucene is that the record syntax/schema serialisation is not working correctly because no GFS configuration file has been specified for YAZ. A SimpleServer implementation without a configuration file for YAZ would lack CQL to PQF conversion, Explain support, etc. The built in defaults seem to be insufficient for proper record syntax/schema serialisation. SimpleServer can be started with the -f option to specify a GFS configuration file for YAZ, http://www.indexdata.com/yaz/doc/server.vhosts.html . YAZ retrieval facility documentation, http://www.indexdata.com/yaz/doc/tools.retrieval.html , is needed to help understand serialisation for the GFS configuration file. Z39.50 object identifiers (OIDs) are listed at http://www.loc.gov/z3950/agency/defns/oids.html . A YAZ GFS configuration example is in etc/yazgfs.xml in the YAZ source code. Other needed configuration files such as pqf.properties, cqlpass.properties, and maps.xml are linked from yazgfs.xml. In Koha, the YAZ GFS configuration file is etc/koha-conf.xml which links to other files, such as etc/zebradb/explain-biblios.xml, and differently named files, such as etc/zebradb/cql.properties. The Koha examples have several mistakes and omissions including the following. Generic XML might include MARCXML and Dublin Core along with any other XML schema , therefore, generic XML should not be conflated with MARCXML which have distinctive serialisations. UNIMARC and USMARC are distinctive and having UNIMARC use USMARC defaults causes confusion and may lead to bugs. There are other mistakes and omissions for Koha YAZ configuration but those seem most relevant to BibLibre's current work on SimpleServer. The ambiguity of the Z39.50 standard over whether records from the result set would need to be retrieved again from the server for the present command if they had already been retrieved as part of the response to the search command complicates my understanding of what may be happening on the server and client side when using the present command. YAZ behaviour is expected to parse MARCXML records in a distinctive MARC formatted manner also used to parse ISO 2709 records when the present command is issued. In the current state of work on SimpleServer, with the apparent absence of proper serialisation, the present command returns incompletely parsed MARC records. YAZ does not attempt parsing for generic XML. Saving MARCXML in raw format would avoid the MARC parsing from present. 2. INVARIANT RESULT SET. A more important problem remains that any SimpleServer query has been returning exactly the same result set whether or not there would be any legitimate matches. 1011 records had always been returned on when I tested on Friday. 1027 records were always being returned when I tested on Monday. 3. DIRECTION FOR NOW. I hope that my testing and direction to a possible solution has been helpful to people working on SimpleServer using Solr/Lucene at BibLibre. As much fun as more actively helping to fix the problems would be, I have to return to some non-library commitments presently. Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

Hi Just some more interesting news. Le 11/02/2011 20:01, Thomas Dukleth a écrit :

When briefly testing the BibLibre test configuration of SimpleServer Z39.50 server I have found same problem as BibLibre are reporting when using yaz-client. I have also noted some other issues.

1. BAD MARC ERROR.

When setting format to UNIMARC in yaz-client, the following error message is returned along with the record when displaying a record with the show command.

"[]Record type: Unimarc

bad MARC. Dumping as it is:"

When setting format to XML in yaz-client, no error message is returned when displaying a record with the show command.

"[]Record type: XML"

The issue seems to be consistent for all records in the result set.

YAZ is converting between ISO 2709 and MARCXML on the client side.

yaz-client does not report a "bad MARC" error when displaying UNIMARC MARCXML records returned from Zebra in ISO 2709 form.

Whatever the MARC error may be, it prevents YAZ from parsing the records properly in ISO 2709. Consequently, unlike yaz-client, my very well tested PHP YAZ based Z39.50 client obtains the result set but cannot even attempt to display the records because it relies upon YAZ for parsing them. This issue is still worked upon... Work in progress. I hope we can find a solution this week. We are diving into the SimpleServer code + yaz source code to find a way to get this right.

2. INVARIANT RESULT SET.

In the configuration which I tested, any query would return exactly the same 1011 record result set whether or not there would be any legitimate matches. This has been fixed. This was owed to the fact that the RegExp::Grammars installed was too old.

3. MISSING EXPLAIN RECORD.

Lack of an Explain record should not be fatal but might be helpful for obtaining more debugging information from YAZ.

I have added a call to a minimal zebra configuration file. And should be fixed now.

4. DEBUGGING.

I do not have time to try intensive tests myself.

First, fix the issue where the result set is an invariant 1011 records.

I suggest having yaz-client save the ISO 2709 records, then attempt to validate them with MARC::Lint keeping in mind that most errors would be from the MARC 21 checks in MARC::Lint.

I have not verified that MARCXML is working with SimpleServer using Solr/Lucene but I would be happy to test that if some SimpleServer target would be configured to return only MARCXML.

If all else fails, Index Data can help solve the problems for a very significant fee.

Hope that helps. -- Henri-Damien LAURENT