[Koha-zebra] Koha & zebra (continuing to try to understand...)

[Sebastian Hammer]

Hi All,

I'll attempt to answer the questions that relate directly to Zebra..

At 15:51 08-08-2005 +0200, Paul POULAIN wrote:

>1st point :
>============
>It seems that "embeeding the zebra", page 6, 2.1.2.3 is wrong, as it says 
>"The Perl API was submitted by Peter Popovics. It is supported & 
>maintained by Index Data. The following is a synopsis of it's usage: ..."
>
>Joshua told me this API was outdated, no more maintained. I tried "perl 
>Makefile.PL && make && make test" on idzebra-1.3.28/perl but got a nasty :
>Failed 8/8 test scripts, 0.00% okay. 103/103 subtests failed, 0.00% okay.

Yes. Just recently, we have learned that Peter is no longer maintaining 
this module. It may have bugs even at present, and it won't be compatible 
with future releases of Zebra. We're just debating at the office whether we 
should drop the API from the distribution in future releases. The 
challenge, as usual, is that there are actually people using it today.

>someone will confirm that it's a non sense to continue investigating what 
>is in this directory.

I think that since the ZOOM API should meet our requirements, it would 
indeed make no sense to continue along this path.

>2nd point :
>===========
>If I understand correctly (& I think I do), the yaz/zoom API gives 
>facilities to search AND manage the zebra database.

Yes.

>It's confirmed by the mail from Sebastian (jul, 11) :
>>o YAZ/ZOOM supports certain extensions to Z39.50 which allows a Perl 
>>application to directly manage databases in a Zebra installation,  update 
>>records, etc. I believe that this may be the neatest and most
>efficient way to integrate an application like Koha with Zebra.. superior 
>to system() calling the Zebra command-line interface, and more robust than 
>the native Zebra Perl API, which was developed by a third party and which 
>may not be entirely reliable.
>
>Once again, if I understand correctly, this Perl/API is not 100% ready & 
>not released yet.

Correct. The YAZ/ZOOM API (C language) supports these extensions, but they 
are not supported by Net::Z3950, which doesn't use YAZ/ZOOM. Net::Z3950 
predates the ZOOM specification, and it is implemented on top of a 
lower-level API in the YAZ toolkit. The solution that we have discussed 
with Joshua involves building a new and better Perl API that more closely 
reflects the ZOOM standard, directly atop the YAZ/ZOOM API. It would thus 
benefit from a range of extended functionality in YAZ/ZOOM, such as 
normalizing MARC records to UTF-8/MARCXML, and the extended services for 
database updates.

>3rd point :
>===========
>(that troubles me a little bit...)
>when I read http://zoom.z3950.org/api/zoom-1.4.html, I don't see anything 
>to add or update or delete records.
>am I becoming mad ? I don't think so. Someone will explain me the trick : 
>is zoom-1.4.html an uncomplete doc or does yaz/zoom have an extension to 
>the zoom official API or something else ?

YAZ/ZOOM supports certain Index Data-specific extensions to the ZOOM API. 
The underlying protocol implementation follows the Z39.50 standard, but the 
ZOOM API does not (yet) support these functions.

Unfortunately (and this troubles me a little bit too, but for a different 
reason :-), these extensions to the YAZ/ZOOM C language API are not 
presently documented, although they have been used in production systems 
for more than a year. Part of the work we would do to support this 
functionality for Koha would be to:

o Document the YAZ/ZOOM C API extensions to allow database updates
o Implement (and document!) a new Perl API on top of YAZ/ZOOM

>anyway, now I consider we will soon have a Perl API to manage the zebra DB.

Yes.

>Except for point 1, I need to work with zebra.
>It seems I have some time to spend on this topic this week. Then 2 weeks 
>off, then, in september probably only a few time free (2 migrations to 
>end, 1 week in burkina faso at the end of the month, plus today still 
>unknown things ;-) )
>
>My idea, while waiting for Perl/Zoom package to be ready, would be to 
>begin the work with some perl system call to run zebra & update the database.
>Then, once the Perl package is ready, move to it.

That makes sense. The Zebra manual has simple examples of how to create a 
simple database.

>I imagine this last step very easy, just replacing a file writing (the 
>iso2709 record) & a system call (to run idzebra) by a "zebra connection 
>open" and "deals the MARC record".

That makes sense, too.

>You, indexdata guys, do you think i'm right there or wrong ? should I go 
>that way ?

I think it makes sense. I'd start by just running the admin interface a bit 
from the shell.. try loading up some marc records and see how it goes.

I *think* I would recommend that you start with the CVS version of YAZ and 
Zebra rather than using the current releases.

--Seb
--
Sebastian Hammer, Index Data, www.indexdata.com
Direct phone: (603) 209-6853 Fax: (603) 357-1813