[Koha-devel] Bug 1993 - Task Scheduler Needs Re-write

Mon Jun 18 10:11:22 CEST 2018

I recently wrote to Nick about this as well: http://lists.koha-community.org/pipermail/koha-devel/2018-June/044605.html. I wrote my own job server for Bug 10662 but figured the Koha community would never accept it, so I scrapped it and made a purpose-built scheduler just for OAI-PMH harvesting, since the sponsor had particular requirements. 

I’m also interested to see the work Jonathan does on this. I know I’m pretty time poor at the moment (for many reasons), so I can’t offer much except feedback.

When I was working on Bug 10662, I looked at Minion… and I think the main reasons I didn’t use it were because Koha hadn’t really adopted Mojolicious yet and I couldn’t see a way to tell a worker to cancel in-progress tasks/jobs and I couldn’t remove queued jobs that were already enqueued and it used a database whereas I wanted to use an in-memory queue to keep it separate from Koha.  But if Koha does further embrace Mojolicious and we’re not needing to interact with jobs/tasks once they’ve been queued and we don’t mind baking the scheduler into the Koha database schema… I would be in favour of using Minion. 

Standing on the shoulders of giants sounds like a plan to me.

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

Office: 02 9212 0899

Direct: 02 8005 0595

From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel-bounces at lists.koha-community.org] On Behalf Of Renvoize, Martin
Sent: Monday, 18 June 2018 5:47 PM
To: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Cc: Koha-devel <koha-devel at lists.koha-community.org>
Subject: Re: [Koha-devel] Bug 1993 - Task Scheduler Needs Re-write

Interested to see what you've started Jonathan,

In rebus:list we used Minion, the job queue spin-off project from the same guys that wrote Mojolicious. Granted, it works best with a PostgreSQL backend, but I believe there are MySQL backends too.. I'd probably start there... the more I worked with them the more complicated I came to realise queues are.. lean on the shoulders of giants and all that ;)

Martin

Martin Renvoize

Development Manager

T: +44 (0) 1483 378728

F: +44 (0) 800 756 6384

E:  <mailto:martin.renvoize at ptfs-europe.com> martin.renvoize at ptfs-europe.com

 <https://www.ptfs-europe.com> www.ptfs-europe.com

 <https://www.ptfs-europe.com> 

Registered in the United Kingdom No. 06416372   VAT Reg No. 925 7211 30

The information contained in this email message may be privileged, confidential and protected from disclosure. If you are not the intended recipient, any dissemination, distribution or copying is strictly prohibited. If you think that you have received this email message in error, please email the sender at info at ptfs-europe.com <mailto:info at ptfs-europe.com> 

On 14 June 2018 at 17:16, Jonathan Druart <jonathan.druart at bugs.koha-community.org <mailto:jonathan.druart at bugs.koha-community.org> > wrote:

Few links:

http://lists.koha-community.org/pipermail/koha-devel/2017-February/043489.html

 <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662> Bug 10662 - Build OAI-PMH Harvesting Client

 <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032> Bug 15032 - [Plack] Scripts that fork (like stage-marc-import.pl <http://stage-marc-import.pl> ) don't work as expected

Basically we need a daemon watching a job queue (DB table) to answer all these needs at once.

I have started something already, I will get back to it when I will have time (and motivation).

Cheers,

Jonathan

On Thu, 14 Jun 2018 at 12:59 Barton Chittenden <barton at bywatersolutions.com <mailto:barton at bywatersolutions.com> > wrote:

This is an old issue that's never really been addressed. As I understand it, at one time, we had a tool that used the unix 'at' scheduler to run requested reports. This was a huge security hole, and we disabled it.... and it's never worked since.

As such, those who host servers must add calls to runreport.pl <http://runreport.pl>  by hand.

Bug 1993 was opened to address this. It was marked as 'In Discussion' in 2013. I would like to see this move forward -- server admins are dying from a thousand paper cuts, and I think that users would benefit from having more control over when their reports are scheduled.

What still needs to be discussed here?

Thanks,

--Barton

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org <mailto:Koha-devel at lists.koha-community.org> 
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org <mailto:Koha-devel at lists.koha-community.org> 
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20180618/20f1b9cc/attachment-0001.html>