[Koha-devel] SimpleServer using Solr progress

[LAURENT Henri-Damien]

Le 16/02/2011 11:54, Thomas Dukleth a écrit :
> [Original subject: Re: [Koha-devel] Solr and Z3950 server some news.]
> 
> 
> 1.  YAZ CONFIGURATION.
> 
> SimpleServer, like all IndexData products is dependent upon YAZ.
> 
> My thought on the bad MARC error from testing BibLibre work on
> SimpleServer as a Z39.50/SRU server using Solr/Lucene is that the record
> syntax/schema serialisation is not working correctly because no GFS
> configuration file has been specified for YAZ.  A SimpleServer
> implementation without a configuration file for YAZ would lack CQL to PQF
> conversion, Explain support, etc.  The built in defaults seem to be
> insufficient for proper record syntax/schema serialisation.
> 
> SimpleServer can be started with the -f option to specify a GFS
> configuration file for YAZ,
> http://www.indexdata.com/yaz/doc/server.vhosts.html .  YAZ retrieval
> facility documentation,
> http://www.indexdata.com/yaz/doc/tools.retrieval.html , is needed to help
> understand serialisation for the GFS configuration file.  Z39.50 object
> identifiers (OIDs) are listed at
> http://www.loc.gov/z3950/agency/defns/oids.html .
> 
> A YAZ GFS configuration example is in etc/yazgfs.xml in the YAZ source
> code.  Other needed configuration files such as pqf.properties,
> cqlpass.properties, and maps.xml are linked from yazgfs.xml.  In Koha, the
> YAZ GFS configuration file is etc/koha-conf.xml which links to other
> files, such as etc/zebradb/explain-biblios.xml, and differently named
> files, such as etc/zebradb/cql.properties.
> 
> The Koha examples have several mistakes and omissions including the
> following.  Generic XML might include MARCXML and Dublin Core along with
> any other XML schema  , therefore, generic XML should not be conflated
> with MARCXML which have distinctive serialisations.  UNIMARC and USMARC
> are distinctive and having UNIMARC use USMARC  defaults causes confusion
> and may lead to bugs.  There are other mistakes and omissions for Koha YAZ
> configuration but those seem most relevant to BibLibre's current work on
> SimpleServer.
> 
> The ambiguity of the Z39.50 standard over whether records from the result
> set would need to be retrieved again from the server for the present
> command if they had already been retrieved as part of the response to the
> search command complicates my understanding of what may be happening on
> the server and client side when using the present command.  YAZ behaviour
> is expected to parse MARCXML records in a distinctive MARC formatted
> manner also used to parse ISO 2709 records when the present command is
> issued.  In the current state of work on SimpleServer, with the apparent
> absence of proper serialisation, the present command returns incompletely
> parsed MARC records.  YAZ does not attempt parsing for generic XML. 
> Saving MARCXML in raw format would avoid the MARC parsing from present.
> 

Many thanks for your time invested, your feedback and thoughtfull hints.
We will try and get things out of this.
And we will try adding a yaz xml file and make that file used in the
SimpleServer.


> 
> 2.  INVARIANT RESULT SET.
> 
> A more important problem remains that any SimpleServer query has been
> returning exactly the same result set whether or not there would be any
> legitimate matches.  1011 records had always been returned on when I
> tested on Friday.  1027 records were always being returned when I tested
> on Monday.
I think this problem comes from the fact that whatever the query is, if
there is no answer possible, we are exposing the whole lot.

> 
> 
> 3.  DIRECTION FOR NOW.
> 
> I hope that my testing and direction to a possible solution has been
> helpful to people working on SimpleServer using Solr/Lucene at BibLibre. 
> As much fun as more actively helping to fix the problems would be, I have
> to return to some non-library commitments presently.
> 
Thanks Thomas.

-- 
Henri-Damien LAURENT