<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-AU link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><a name="_MailEndCompose"><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>That’s awesome to hear, Tomas. I suppose I’d like to brainstorm more about the schema changes that would be required to support a more useful API. <o:p></o:p></span></a></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>I figure if we collect a number of use cases, we can create a general approach to cover them all. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>David Cook<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Systems Librarian<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Prosentient Systems<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>72/330 Wattle St<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Ultimo, NSW 2007<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Australia<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Office: 02 9212 0899<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Direct: 02 8005 0595<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'> Tomas Cohen Arazi [mailto:tomascohen@gmail.com] <br><b>Sent:</b> Monday, 26 September 2016 10:56 PM<br><b>To:</b> David Cook <dcook@prosentient.com.au>; Koha Devel <koha-devel@lists.koha-community.org><br><b>Subject:</b> Re: [Koha-devel] Improving the API for record imports<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>David, I agree with your ideas. Having a REST endpoint handling records would be awesome improvement. I'll hapilly sign or QA those patches.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Regards<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>El mié., 21 sept. 2016 a las 22:00, David Cook (<<a href="mailto:dcook@prosentient.com.au">dcook@prosentient.com.au</a>>) escribió:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi all,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I want to improve the API(s) for importing (bibliographic) records.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>But I’m not 100% sure how to go about what criteria to use to gauge “improvement”. Here are some ideas:<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p><span style='font-family:Symbol'>·</span><span style='font-size:7.0pt'> </span>I think the REST API offers up one obvious addition and that’s DELETE. It would be great to delete records using the API. I’ve found this mostly thanks to OAI-PMH where you get a notice when a record has been deleted upstream and needs to be deleted from Koha too.<o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>The difficulty in deleting bibliographic records is items. If there are items, it blocks the delete. You could auto-delete items, but you can’t if there are holds or if it’s currently issued. At best, you could return a failed delete via the API, I think.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p><span style='font-family:Symbol'>·</span><span style='font-size:7.0pt'> </span>I keep thinking that “import profiles” could be a good way to go too. This might include a MARC Modification Template, it might be a XSLT, it might be a Koha::Filter. It would let you turns your incoming metadata from format A to format B.<o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>This would also allow you to POST a minimum amount of data to the API. You’d have your metadata record and your import profile code.<o:p></o:p></p><p style='margin-left:108.0pt'><span style='font-family:Wingdings'>§</span><span style='font-size:7.0pt'> </span>The downside of this is the POST wouldn’t detail the options for the import, and those imports would be more obfuscated. Changes to a profile could also impact subsequent imports unintentionally. <o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>I’m not sure how batches would work with filtering. You might have 50/100 records filter correctly and import, but that leaves 50 which aren’t imported. Do you undo the whole import, or do you fix your filter, and then re-import the first 50 alongside the first time import of the last 50?<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p><span style='font-family:Symbol'>·</span><span style='font-size:7.0pt'> </span>The original metadata would be stored regardless of metadata format (e.g. marcxml, mods, rdf, dc, etc.) for historical purposes or to allow re-filtering in the event that a filter fails to correctly transform a record to valid MARCXML (or whatever internal metadata format Koha uses).<o:p></o:p></p><p> <o:p></o:p></p><p><span style='font-family:Symbol'>·</span><span style='font-size:7.0pt'> </span>The import source and the type of import would be stored (ie <a href="http://oai-server.com" target="_blank">http://oai-server.com</a> and “oai-pmh”, <a href="http://lx2.loc.gov" target="_blank">lx2.loc.gov</a> and “z3950”, <insert relevant source> and “oclc connexion”, etc.). <o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>I think this would be helpful for managing/processing a certain type of import, perhaps also in terms of matching records. <o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>At the moment, with my work on #10662, I have a separate table “import_oai” which obviously is for records imported via oai-pmh only. I don’t actually track the origin due to how I currently source the imports, but I could do it in the future.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p><span style='font-family:Symbol'>·</span><span style='font-size:7.0pt'> </span>Storing some sort of identifier would be good<o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>OAI-PMH records have unique identifiers for the records<o:p></o:p></p><p style='margin-left:72.0pt'><span style='font-family:"Courier New"'>o</span><span style='font-size:7.0pt'> </span>Z39.50… that’s tougher since it’s not a specific metadata format like the OAI-PMH format. So I don’t know about this one.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Anyway, these are just some ideas I’ve been having. They’re not fully formed by any stretch of the imagination. <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>David Cook<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Systems Librarian<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Prosentient Systems<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>72/330 Wattle St<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Ultimo, NSW 2007<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Australia<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Office: 02 9212 0899<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Direct: 02 8005 0595<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div><p class=MsoNormal>_______________________________________________<br>Koha-devel mailing list<br><a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br><a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><o:p></o:p></p></blockquote></div><div><p class=MsoNormal>-- <o:p></o:p></p></div><div><div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Tomás Cohen Arazi<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Theke Solutions (<a href="http://theke.io/">https://theke.io</a>)<br></span><span style='font-size:9.5pt;font-family:"Segoe UI Symbol",sans-serif;color:#757575'>✆</span><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'> +54 9351 3513384<br>GPG: B2F3C15F<o:p></o:p></span></p></div></div></div></div></div></body></html>