[Koha-devel] Improving the API for record imports

Fri Sep 30 04:26:06 CEST 2016

That’s awesome to hear, Tomas. I suppose I’d like to brainstorm more about the schema changes that would be required to support a more useful API. 

I figure if we collect a number of use cases, we can create a general approach to cover them all. 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

Office: 02 9212 0899

Direct: 02 8005 0595

From: Tomas Cohen Arazi [mailto:tomascohen at gmail.com] 
Sent: Monday, 26 September 2016 10:56 PM
To: David Cook <dcook at prosentient.com.au>; Koha Devel <koha-devel at lists.koha-community.org>
Subject: Re: [Koha-devel] Improving the API for record imports

David, I agree with your ideas. Having a REST endpoint handling records would be awesome improvement. I'll hapilly sign or QA those patches.

Regards

El mié., 21 sept. 2016 a las 22:00, David Cook (<dcook at prosentient.com.au <mailto:dcook at prosentient.com.au> >) escribió:

Hi all,

I want to improve the API(s) for importing (bibliographic) records.

But I’m not 100% sure how to go about what criteria to use to gauge “improvement”. Here are some ideas:

*         I think the REST API offers up one obvious addition and that’s DELETE. It would be great to delete records using the API. I’ve found this mostly thanks to OAI-PMH where you get a notice when a record has been deleted upstream and needs to be deleted from Koha too.

o   The difficulty in deleting bibliographic records is items. If there are items, it blocks the delete. You could auto-delete items, but you can’t if there are holds or if it’s currently issued. At best, you could return a failed delete via the API, I think.

*         I keep thinking that “import profiles” could be a good way to go too. This might include a MARC Modification Template, it might be a XSLT, it might be a Koha::Filter. It would let you turns your incoming metadata from format A to format B.

o   This would also allow you to POST a minimum amount of data to the API. You’d have your metadata record and your import profile code.

*  The downside of this is the POST wouldn’t detail the options for the import, and those imports would be more obfuscated. Changes to a profile could also impact subsequent imports unintentionally. 

o   I’m not sure how batches would work with filtering. You might have 50/100 records filter correctly and import, but that leaves 50 which aren’t imported. Do you undo the whole import, or do you fix your filter, and then re-import the first 50 alongside the first time import of the last 50?

*         The original metadata would be stored regardless of metadata format (e.g. marcxml, mods, rdf, dc, etc.) for historical purposes or to allow re-filtering in the event that a filter fails to correctly transform a record to valid MARCXML (or whatever internal metadata format Koha uses).

*         The import source and the type of import would be stored (ie http://oai-server.com and “oai-pmh”, lx2.loc.gov <http://lx2.loc.gov>  and “z3950”, <insert relevant source> and “oclc connexion”, etc.). 

o   I think this would be helpful for managing/processing a certain type of import, perhaps also in terms of matching records. 

o   At the moment, with my work on #10662, I have a separate table “import_oai” which obviously is for records imported via oai-pmh only. I don’t actually track the origin due to how I currently source the imports, but I could do it in the future.

*         Storing some sort of identifier would be good

o   OAI-PMH records have unique identifiers for the records

o   Z39.50… that’s tougher since it’s not a specific metadata format like the OAI-PMH format. So I don’t know about this one.

Anyway, these are just some ideas I’ve been having. They’re not fully formed by any stretch of the imagination. 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

Office: 02 9212 0899

Direct: 02 8005 0595

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org <mailto:Koha-devel at lists.koha-community.org> 
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

-- 

Tomás Cohen Arazi

Theke Solutions (https://theke.io <http://theke.io/> )
✆ +54 9351 3513384
GPG: B2F3C15F

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20160930/c30041f9/attachment-0001.html>