FW: RDBMS like behaviour or merged result sets from ZEBRA-ZOOM
-----Original Message----- From: Mike Taylor [mailto:mike@miketaylor.org.uk] Sent: Tuesday, July 25, 2006 12:43 PM To: Tümer Garip Cc: support@indexdata.dk; 'Joshua Ferraro' Subject: RDBMS like behaviour or merged result sets from ZEBRA-ZOOM Tümer Garip writes:
We have come a long way in integrating ZEBRA and ZOOM into KOHA and first examples are already running in some libraries.
However we now have a problem that we could not solve and wonder > whether its us or the capabilities of ZEBRA or ZOOM (probably all) >
That's great news. thats preventing further development. >
We have two different types of MARC records. Namely bibliographic > and holdings. These records are related to each other with uniquie > ID numbers. A bibliographic record can have multiple holdings > record related to it. > > We are having problem searching these records in RDBMS manner. i.e > Searching for say title which exists in bibliographic record and > limiting that search to a specific branch which exists in holdings > record.
We tried putting all the records in the same database could not do > it. We than tried putting them in different databases (even > different servers) but could not find a way of telling zebra to > merge
Let me make sure I understand what you're trying to do here. It's a two-stage search, in which you first want to search the list of works for books with a certain title, then find the holdings records for nearby libraries that have the books of that title? In a relational database, it would be something like this: select all from works,holdings where works.title = 'Harry Potter' and holdings.work_id = works.id and holdings.library_id in (45,76,102); Correct? If so, you simply cannot do this with Zebra -- it's a relational query and Zebra is not a relational database. You can't even express such a query in the Z39.50 Type-1 or SRU CQL query languages. the results together. >
For a small resultset we can do a recursive search of ID's on the > other database and get a merged resultset but library operations > work on large numbers and that is impossible when the small set is > around 1800 records.
I think you mean code like this? (WARNING: UNTESTED) $rs = $works_conn->search('title="harry potter"); foreach my $i (1 .. $rs->size()) { $rec = $rs->record($i); $id = extract_identifier_from_record($rec); $rs2 = $holdings_conn->search("id=$id and library_id=45 or library_id=76 or library_id=102"); push @holdings_records, records_extracted_from($rs2); } That approach is the obvious one to take; but, no, it doesn't scale well when the initial result set is large.
Is there any way we can actually do this with the capabilities of > ZEBRA&ZOOM? Something like @attr 1=52 (@resultset1 @attr 1=52)? Or may be a new z:index called linkid which joins two databases or > different records within a database.
What you're talking about is certainly in the realm of the possible, but it would be a _significant_ research, design and implementation task -- one, I think, that would beyond the Koha consortium's ability to fund. I am sorry to disappoint, but I think you're going to have to work round this one.
Any idea is appreciated as the whole database design is currently > stuck with this problem.
Have you considered adding redundant data, such as author and title, to the holdings records? Then you could search them directly. Not pretty, but it may be pragmatic. _/|_ ___________________________________________________________________ /o ) \/ Mike Taylor <mike@miketaylor.org.uk> http://www.miketaylor.org.uk )_v__/\ The Unix command to kill processes is not "kill -9". The Unix command to make a tape archive is not "tar cv". Options are optional.
Hmmm It sounds like storing the holdings info in zebra is going to be a no go. I still think that storing information that changes regularly, like issue status, and other holdings info should be stored in something like a rdbms Would storing the holdings info in mysql (or other rdbms) be a bad thing? Chris On Tue, Jul 25, 2006 at 12:58:53PM +0300, T?mer Garip said:
-----Original Message----- From: Mike Taylor [mailto:mike@miketaylor.org.uk] Sent: Tuesday, July 25, 2006 12:43 PM To: T?mer Garip Cc: support@indexdata.dk; 'Joshua Ferraro' Subject: RDBMS like behaviour or merged result sets from ZEBRA-ZOOM
T?mer Garip writes:
We have come a long way in integrating ZEBRA and ZOOM into KOHA and first examples are already running in some libraries.
That's great news.
However we now have a problem that we could not solve and wonder > whether its us or the capabilities of ZEBRA or ZOOM (probably all) > thats preventing further development. > We have two different types of MARC records. Namely bibliographic > and holdings. These records are related to each other with uniquie > ID numbers. A bibliographic record can have multiple holdings > record related to it. > > We are having problem searching these records in RDBMS manner. i.e > Searching for say title which exists in bibliographic record and > limiting that search to a specific branch which exists in holdings > record.
Let me make sure I understand what you're trying to do here. It's a two-stage search, in which you first want to search the list of works for books with a certain title, then find the holdings records for nearby libraries that have the books of that title? In a relational database, it would be something like this:
select all from works,holdings where works.title = 'Harry Potter' and holdings.work_id = works.id and holdings.library_id in (45,76,102);
Correct?
If so, you simply cannot do this with Zebra -- it's a relational query and Zebra is not a relational database. You can't even express such a query in the Z39.50 Type-1 or SRU CQL query languages.
We tried putting all the records in the same database could not do > it. We than tried putting them in different databases (even > different servers) but could not find a way of telling zebra to > merge the results together. > For a small resultset we can do a recursive search of ID's on the > other database and get a merged resultset but library operations > work on large numbers and that is impossible when the small set is > around 1800 records.
I think you mean code like this? (WARNING: UNTESTED)
$rs = $works_conn->search('title="harry potter"); foreach my $i (1 .. $rs->size()) { $rec = $rs->record($i); $id = extract_identifier_from_record($rec); $rs2 = $holdings_conn->search("id=$id and library_id=45 or library_id=76 or library_id=102"); push @holdings_records, records_extracted_from($rs2); }
That approach is the obvious one to take; but, no, it doesn't scale well when the initial result set is large.
Is there any way we can actually do this with the capabilities of > ZEBRA&ZOOM? Something like @attr 1=52 (@resultset1 @attr 1=52)? Or may be a new z:index called linkid which joins two databases or > different records within a database.
What you're talking about is certainly in the realm of the possible, but it would be a _significant_ research, design and implementation task -- one, I think, that would beyond the Koha consortium's ability to fund.
I am sorry to disappoint, but I think you're going to have to work round this one.
Any idea is appreciated as the whole database design is currently > stuck with this problem.
Have you considered adding redundant data, such as author and title, to the holdings records? Then you could search them directly. Not pretty, but it may be pragmatic.
_/|_ ___________________________________________________________________ /o ) \/ Mike Taylor <mike@miketaylor.org.uk> http://www.miketaylor.org.uk )_v__/\ The Unix command to kill processes is not "kill -9". The Unix command to make a tape archive is not "tar cv". Options are optional.
_______________________________________________ Koha-devel mailing list Koha-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/koha-devel
-- Chris Cormack Programmer 027 4500 789 Katipo Communications Ltd chris@katipo.co.nz www.katipo.co.nz
Chris Cormack a écrit :
Hmmm
It sounds like storing the holdings info in zebra is going to be a no go.
I still think that storing information that changes regularly, like issue status, and other holdings info should be stored in something like a rdbms
Would storing the holdings info in mysql (or other rdbms) be a bad thing?
Storing everything in zebra will let user see the document status from z3950 server directly. The no-go is probably why no z3950 server does this : it's too much time consuming. another reason for holding in a separate DB is to have a complete MARC record for holdings. Holdings for UNIMARC exists, even if all the libraries I know uses "recommandation 995" that store holding in 995 field. -- Paul POULAIN et Henri Damien LAURENT Consultants indépendants en logiciels libres et bibliothéconomie (http://www.koha-fr.org) Tel : 04 91 31 45 19
Coming late to the conversation ... Storing ANYthing in an RDBMS is a bad idea (you can tell I'm not a fan :-) And Tumer is just plain wrong when he says what you want "is a relational query". It may be phrased in a relational manner, because that's what most people understand, but LIST HOLDINGS WHEN TITLE = 'Harry Potter' AND LIBRARY = 45 76 102 is a query, written in a database language called ENGLISH, which will do what you want (debugging excepted) and it's not a relational database. Not knowing Zebra, I can't tell you how to solve the problem in Zebra. But I *can* tell you it's got absolutely nothing to do with whether or not Zebra is a relational database. Cheers, Wol -----Original Message----- From: koha-devel-bounces+anthony.youngman=eca-international.com@nongnu.org [mailto:koha-devel-bounces+anthony.youngman=eca-international.com@nongnu.org] On Behalf Of Paul POULAIN Sent: 25 July 2006 11:41 To: Chris Cormack Cc: koha-devel@nongnu.org Subject: Re: [Koha-devel] FW: RDBMS like behaviour or merged result sets fromZEBRA-ZOOM Chris Cormack a écrit :
Hmmm
It sounds like storing the holdings info in zebra is going to be a no go.
I still think that storing information that changes regularly, like issue status, and other holdings info should be stored in something like a rdbms
Would storing the holdings info in mysql (or other rdbms) be a bad thing?
Storing everything in zebra will let user see the document status from z3950 server directly. The no-go is probably why no z3950 server does this : it's too much time consuming. another reason for holding in a separate DB is to have a complete MARC record for holdings. Holdings for UNIMARC exists, even if all the libraries I know uses "recommandation 995" that store holding in 995 field. -- Paul POULAIN et Henri Damien LAURENT Consultants indépendants en logiciels libres et bibliothéconomie (http://www.koha-fr.org) Tel : 04 91 31 45 19 _______________________________________________ Koha-devel mailing list Koha-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/koha-devel * ************************************************************************ * This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system. Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333. * ************************************************************************ *
participants (4)
-
Anthony Youngman -
Chris Cormack -
Paul POULAIN -
Tümer Garip