<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-AU link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><a name="_MailEndCompose"><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>Which one, Tomas?<o:p></o:p></span></a></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>I’m planning to post the code for what I have already in early March.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>David Cook<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Systems Librarian<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Prosentient Systems<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>72/330 Wattle St<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Ultimo, NSW 2007<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Australia<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Office: 02 9212 0899<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Direct: 02 8005 0595<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'> Tomas Cohen Arazi [mailto:tomascohen@gmail.com] <br><b>Sent:</b> Thursday, 23 February 2017 2:16 PM<br><b>To:</b> David Cook <dcook@prosentient.com.au>; Tajoli Zeno <z.tajoli@cineca.it>; koha-devel@lists.koha-community.org<br><b>Subject:</b> Re: [Koha-devel] Task schedulers and message queues for Koha<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p>Share it :-)<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>El mié., 22 de feb. de 2017 9:57 PM, David Cook <<a href="mailto:dcook@prosentient.com.au">dcook@prosentient.com.au</a>> escribió:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>Hi Zeno,<br><br>I have a number of concerns about Celery. One of those is that it would add<br>numerous external dependencies and complexity to Koha implementations.<br><br>Your suggestion of Celery + RabbitMQ + AnyEvent::RabbitMQ sounds ok,<br>although it would involve work too. While Celery clients exist for PHP and<br>Node.js, we'd need to create a Perl implementation of the Celery protocol<br>using AnyEvent::RabbitMQ (or Net::RabbitFoot). Not that I'm necessarily<br>opposed to that.<br><br>We'd also still need to write the tasks in Python (or use web hooks which<br>would have the overhead of HTTP plus you'd have to worry about your web<br>server being up). I'm not sure how keen the community at large is to support<br>more server-side languages. I like writing Python, so I don't mind porting<br>over my OAI-PMH code from Perl to Python. I've abandoned the HTTP::OAI<br>module anyway for a few reasons.<br><br>RabbitMQ is a pretty heavy duty product as well which comes with its own<br>requirements: <a href="https://www.rabbitmq.com/production-checklist.html" target="_blank">https://www.rabbitmq.com/production-checklist.html</a>. While we<br>currently help people with Apache, MySQL, Zebra, and ElasticSearch, we'd<br>also all need to become experts with RabbitMQ.<br><br>I've already put together a Perl-based scheduler using POE which forks its<br>own workers. And I've already put together a basic Perl-based message queue<br>which sends events to pre-existing workers (like Celery). Celery with<br>RabbitMQ is more mature and complex, but my Perl programs do the trick.<br><br>Looking at DSpace's OAI-PMH harvester, it works very much like my first<br>design. It's a Java scheduler which uses threads rather than child processes<br>to do its work.<br><br>Due to the lack of engagement overall, I think I'll probably just keep my<br>existing design, since it works and works quite well.<br><br>David Cook<br>Systems Librarian<br>Prosentient Systems<br>72/330 Wattle St<br>Ultimo, NSW 2007<br>Australia<br><br>Office: 02 9212 0899<br>Direct: 02 8005 0595<br><br><br>> -----Original Message-----<br>> From: Tajoli Zeno [mailto:<a href="mailto:z.tajoli@cineca.it" target="_blank">z.tajoli@cineca.it</a>]<br>> Sent: Wednesday, 22 February 2017 7:49 PM<br>> To: David Cook <<a href="mailto:dcook@prosentient.com.au" target="_blank">dcook@prosentient.com.au</a>>; <a href="mailto:koha-devel@lists.koha-">koha-devel@lists.koha-</a><br>> <a href="http://community.org" target="_blank">community.org</a><br>> Subject: Re: [Koha-devel] Task schedulers and message queues for Koha<br>><br>> Hi David and all,<br>><br>> Il 21/02/2017 23:29, David Cook ha scritto:<br>> >. Two, they wanted to<br>> > execute OAI-PMH requests every 2-3 seconds and cron has 1 minute as<br>> >its finest granularity. Three, even if you setup a cronjob to run<br>> >every minute, long running tasks could get duplicated (although you<br>> >could mitigate that with locks which would be a pain). Plus, you want<br>> >to run tasks in parallel, so you're going to want to use multiple<br>> >processes, which cron isn't really set up to achieve.<br>><br>> Ok, if you need those features cron isn't enough.<br>> But why do you drop the option Celery + RabbitMQ + AnyEvent::RabbitMQ<br>><br>> They have official debiano packages:<br>> <a href="https://packages.debian.org/jessie/python-celery" target="_blank">https://packages.debian.org/jessie/python-celery</a><br>> <a href="https://packages.debian.org/jessie/rabbitmq-server" target="_blank">https://packages.debian.org/jessie/rabbitmq-server</a><br>> <a href="https://packages.debian.org/jessie/libanyevent-rabbitmq-perl" target="_blank">https://packages.debian.org/jessie/libanyevent-rabbitmq-perl</a><br>><br>> We still use one of their dpendencies for similar tasks (libanyevent-perl<br>"event<br>> loop framework with multiple implementations").<br>><br>> Python is already present in our Debian/Ubuntu system, it is a prereq of<br>the<br>> distributions.<br>><br>> Redone a so complex stack in perl i think is very complex.<br>><br>> Bye<br>> Zeno Tajoli<br>><br>><br>><br>> --<br>> Zeno Tajoli<br>> /SVILUPPO PRODOTTI CINECA/ - Automazione Biblioteche<br>> Email: <a href="mailto:z.tajoli@cineca.it" target="_blank">z.tajoli@cineca.it</a> Fax: 051/6132198<br>> *CINECA* Consorzio Interuniversitario - Sede operativa di Segrate (MI)<br><br><br>_______________________________________________<br>Koha-devel mailing list<br><a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br><a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><o:p></o:p></p></blockquote></div><div><p class=MsoNormal>-- <o:p></o:p></p></div><div><div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Tomás Cohen Arazi<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Theke Solutions (<a href="http://theke.io/">https://theke.io</a>)<br></span><span style='font-size:9.5pt;font-family:"Segoe UI Symbol",sans-serif;color:#757575'>✆</span><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'> +54 9351 3513384<br>GPG: B2F3C15F<o:p></o:p></span></p></div></div></div></div></div></body></html>