Using a Native XML Database with Koha
Developers, A recent post in the Koha discussion list about MODS support reminded me of something I'd be thinking of for a while. I've heard of talk of developing Koha to support other standards that MARC, such as DC, MODS or METS. It certainly would be nice to be able to use whichever schema best fit ones collection, and to be able to mix different types of standards together into a common interface and management system. I suppose one of the major difficulties would be finding a way to map all those different standards to a relational database, without exploding the number of tables and complexity. So, the thought occurred to me: since most of these formats are already in XML (or could be mapped to it easily enough), why not use a Native XML Database? This would save on mappings, since XQuery would be used to extract information directly from the XML. It would also be flexible enough to allow for home-grown Schemas to be mixed in with standards. There would also be one less translation in getting the information out of storage and into an AJAX interface. The one major concern I see (aside from all the re-coding) is speed. Could a Native XML Database, with the proper indexes, perform as quickly as Koha does now with MySQL and Zebra? And is there any way to answer that without doing the re-coding? I just thought I'd throw this out as food for thought, and hopefully some discussion. I don't know if this has been mentioned before (couldn't find it in the email archives), but it could prove to be worth looking at. Cheers, Ian Walls Systems Integration Librarian NYU Health Sciences Libraries (212) 263-8687 </PRE> <html> <body> ------------------------------------------------------------<br /> This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is proprietary, confidential, and exempt from disclosure under applicable law. Any unauthorized review, use, disclosure, or distribution is prohibited. If you have received this email in error please notify the sender by return email and delete the original message. Please note, the recipient should check this email and any attachments for the presence of viruses. The organization accepts no liability for any damage caused by any virus transmitted by this email.<br /> ================================= </body> </html> <PRE>
Funny you should mention this now. I was just thinking about removing MARC from Koha and dropping in DC. Switching to all XML is a much better idea. The reason I was thinking about this is because some people I work with want to use Koha for Asset Management and Inventory. I have a few librarians already doing it along side the library holdings. My thought was that DC would handle equipment better than MARC. So now I'm thinking of just dropping the binary marc column, and instead of frameworks have record formats for MARC, DC, etc. I'd also have to use NoZebra. I don't know if I could get zebra to index different formats like that. Anyway, thanks for the idea. 2009/6/2 Walls, Ian <Ian.Walls@med.nyu.edu>
Developers,
A recent post in the Koha discussion list about MODS support reminded me of something I’d be thinking of for a while. I’ve heard of talk of developing Koha to support other standards that MARC, such as DC, MODS or METS. It certainly would be nice to be able to use whichever schema best fit ones collection, and to be able to mix different types of standards together into a common interface and management system. I suppose one of the major difficulties would be finding a way to map all those different standards to a relational database, without exploding the number of tables and complexity.
So, the thought occurred to me: since most of these formats are already in XML (or could be mapped to it easily enough), why not use a Native XML Database? This would save on mappings, since XQuery would be used to extract information directly from the XML. It would also be flexible enough to allow for home-grown Schemas to be mixed in with standards. There would also be one less translation in getting the information out of storage and into an AJAX interface.
The one major concern I see (aside from all the re-coding) is speed. Could a Native XML Database, with the proper indexes, perform as quickly as Koha does now with MySQL and Zebra? And is there any way to answer that without doing the re-coding?
I just thought I’d throw this out as food for thought, and hopefully some discussion. I don’t know if this has been mentioned before (couldn’t find it in the email archives), but it could prove to be worth looking at.
Cheers,
Ian Walls
Systems Integration Librarian
NYU Health Sciences Libraries
(212) 263-8687
------------------------------------------------------------ This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is proprietary, confidential, and exempt from disclosure under applicable law. Any unauthorized review, use, disclosure, or distribution is prohibited. If you have received this email in error please notify the sender by return email and delete the original message. Please note, the recipient should check this email and any attachments for the presence of viruses. The organization accepts no liability for any damage caused by any virus transmitted by this email. =================================
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
Zebra knows how to index probably anything XML as long as you give it the correct XSLT to massage your data and put it in Zebra canonical format. Sébastien ________________________________ De : koha-devel-bounces@lists.koha.org [mailto:koha-devel-bounces@lists.koha.org] De la part de Mike Hafen Envoyé : 2 juin 2009 14:35 À : Walls, Ian; koha-devel@lists.koha.org Objet : Re: [Koha-devel] Using a Native XML Database with Koha ... So now I'm thinking of just dropping the binary marc column, and instead of frameworks have record formats for MARC, DC, etc. I'd also have to use NoZebra. I don't know if I could get zebra to index different formats like that. Anyway, thanks for the idea.
Mike Hafen a écrit :
Funny you should mention this now. I was just thinking about removing MARC from Koha and dropping in DC. Switching to all XML is a much better idea.
The reason I was thinking about this is because some people I work with want to use Koha for Asset Management and Inventory. I have a few librarians already doing it along side the library holdings. My thought was that DC would handle equipment better than MARC.
So now I'm thinking of just dropping the binary marc column, and instead of frameworks have record formats for MARC, DC, etc. I'd also have to use NoZebra. I don't know if I could get zebra to index different formats like that. The structure to support non MARC is at early stage, but is here since 2.0
Having datas twice (biblio & items and marcxml row), with the help of TransformMarcToKoha should be very helpfull. Dirty hack : - modify TransformMarcToKoha to deal with DC (or whatever you want) (and some other subs inside C4/Biblio.pm and C4/Items.pm) - rewrite MARC editor display DC (big job...) - do the XSLT to display the MARC record - update zebra config file to index DC That should be enough... All other things (circ,...) should work like a charm without any changes. Of course, that's the theory. And you know that "theory and practise are the same... in theory" -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Hi, 2009/6/2 Walls, Ian <Ian.Walls@med.nyu.edu>:
So, the thought occurred to me: since most of these formats are already in XML (or could be mapped to it easily enough), why not use a Native XML Database? This would save on mappings, since XQuery would be used to extract information directly from the XML. It would also be flexible enough to allow for home-grown Schemas to be mixed in with standards. There would also be one less translation in getting the information out of storage and into an AJAX interface.
I agree that having more XML functionality in the DBMS would be handy. Of course, only bibliographic and authority metadata currently require the use of XML, and there's a lot of transactional processing (circulation, acquisitions, etc.) that benefits from the traditional relational structure, so I wouldn't recommend moving the entire schema to a native XML database. MySQL 5.1 goes part of the way by adding the ExtractValue() and UpdateXML() functions to do queries and updates on an XML blob using a subset of XPath, but there's no schema support. However, as long as it doesn't perform too badly, adding multi-metadata support may be enough to justify upgrading to MySQL 5.1, although I notice that 5.1 is still only in Debian experimental. Regards, Galen -- Galen Charlton VP, Research & Development, LibLime galen.charlton@liblime.com p: 1-888-564-2457 x709 skype: gmcharlt
participants (5)
-
Galen Charlton -
Mike Hafen -
paul POULAIN -
Sébastien Nadeau -
Walls, Ian