October 2018 - Koha-bugs - lists.koha-community.org

[Bug 13098] Item lost multiple times by the same patron will create only be charged once
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13098 Alex Buckley <alexbuckley(a)catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #80862|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 13098] Item lost multiple times by the same patron will create only be charged once
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13098 Alex Buckley <alexbuckley(a)catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #80863|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 13098] Item lost multiple times by the same patron will create only be charged once
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13098 Alex Buckley <alexbuckley(a)catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alexbuckley(a)catalyst.net.nz Attachment #80864|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15032] [Plack] Scripts that fork (like stage-marc-import.pl) don't work as expected
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032 --- Comment #33 from David Cook <dcook(a)prosentient.com.au> --- (In reply to Jonathan Druart from comment #26) > I do not think your Koha::Daemon will work for the background jobs. > > To me, the best way to fix this would be to provide a Koha daemon, instead. > But it would watch a DB table, which would allow us to build a view on top > of it to manage the different jobs (and history). > > We could also have different daemons for different needs (a param of the > daemon matching a column in the DB table). > > The question behind that is security, we will have to list the different > operations the daemon is allowed to process (and so we will loose > flexibility). > > I have spent a *lot* of time trying to fix this issue by forking, etc. And > my conclusion is: it's not feasible (see discussions and tries on related > bug reports) > > See also bug 1993. I just realized that I misread your comment, Jonathan... Take a look at line 106 of "Harvester.pm" at https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=106…. Using POE::Component::JobQueue, I check a DB table called "oai_harvester_import_queue" for "new" import jobs. You could do the same thing. In my case, the job in "oai_harvester_import_queue" actually has a field containing a JSON array of record identifiers, and the import task fetches those records from the file system and then works on them as a batch. Going 1 by 1 for each record would be way too slow (both in terms of retrieving tasks and importing records). But yeah the web interface could upload the records, store their location, put that in a job, and then store that data in the database. The import daemon would poll the database (which isn't super efficient but efficient enough for Koha's purposes I imagine), and then work on batches as it could. -- (One thing to keep in mind here is import matching rules that use Zebra... if you have high performance importing, Zebra won't be able to keep up, and it means that you could wind up with duplicate records if someone tries to import the same record too quickly... ) -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15032] [Plack] Scripts that fork (like stage-marc-import.pl) don't work as expected
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032 --- Comment #32 from David Cook <dcook(a)prosentient.com.au> --- We could look at something like http://gearman.org/, http://www.celeryproject.org/, or one of the other many projects... but they don't offer as much nuanced control as we might like. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15032] [Plack] Scripts that fork (like stage-marc-import.pl) don't work as expected
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032 --- Comment #31 from David Cook <dcook(a)prosentient.com.au> --- Actually, when working on #10662, I thought about creating a standalone "import daemon", as importing records into Koha is the hardest part of OAI-PMH harvesting. (I already have an extremely fast download worker, but the bottleneck is importing into Koha *sadface*.) For #15032, "Stage MARC records for import" could simply be a script that allows a use to upload MARC records from their browser client to the Koha server, and to pass on some configuration options to accompany those records. We could store all of that on the file system, in the database, or wherever makes the most sense. If we wanted to have fun, we could leave them in-memory and let an import daemon suck them out of memory instead of a disk which would require way more I/O ops. Once the records are uploaded to the Koha server and stored <wherever> with the job details, the PSGI/CGI script alerts the "import daemon" that it would like to start the job. Now the "import daemon" should probably have a master listener process which does very little work, which makes it highly responsive to queries from the Koha web client. That master listener process can talk to worker processes that do the actual work. In terms of imports, it probably would be good if the master process can query the workers for progress updates (and probably also allow cancellations of imports - which might be useful for very large imports). (Oh another thing... this import daemon wouldn't just be usable from a web interface. It could also be used by command line tools. Thus we have 1 point for getting records into Koha, and that daemon can act as gatekeeper.) We default the "import daemon" workers to maybe 1 worker, but allow it to be configurable, so that large Koha instances can commit more resources to it. (If we were smart, we'd probably have the import daemon use TCP for communications and allow workers to be distributed. Although to do that we'd need to package Koha libraries for working with records and distribute those too. Plus we'd need database configuration information distributed. I mean we could have an API to handle all the actual imports, but that would be slow...) Just some ideas ^_^. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15032] [Plack] Scripts that fork (like stage-marc-import.pl) don't work as expected
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032 --- Comment #30 from David Cook <dcook(a)prosentient.com.au> --- (In reply to Jonathan Druart from comment #26) > I do not think your Koha::Daemon will work for the background jobs. > > To me, the best way to fix this would be to provide a Koha daemon, instead. > But it would watch a DB table, which would allow us to build a view on top > of it to manage the different jobs (and history). > > We could also have different daemons for different needs (a param of the > daemon matching a column in the DB table). > > The question behind that is security, we will have to list the different > operations the daemon is allowed to process (and so we will loose > flexibility). > > I have spent a *lot* of time trying to fix this issue by forking, etc. And > my conclusion is: it's not feasible (see discussions and tries on related > bug reports) > > See also bug 1993. My Koha::Daemon is just a module for making a daemon process; it's similar to Proc::Daemon on CPAN. But yeah I could see my Koha::OAI::Harvester::Listener and Koha::OAI::Harvester not working for all background jobs. They work well in my case but I'd be happy to actually replace (elements of) them with a more general Koha solution, if it worked for my use case too. Yeah I think that different daemons for different needs makes sense too. Like in the case of #10662, I originally wrote a generic job server, but then realized it might not suit everyone, so I re-wrote it just for that specific task of harvesting OAI-PMH records. Yeah, I don't think forking makes sense in a Plack/persistent environment. Actually, I don't think it makes sense in the CGI environment either, but it "worked". Setting up separate daemons is easy. I don't know why we don't just do that. The issue then I suppose is security, as you've said, and deciding on communication protocols. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15032] [Plack] Scripts that fork (like stage-marc-import.pl) don't work as expected
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032 --- Comment #29 from David Cook <dcook(a)prosentient.com.au> --- (In reply to Jonathan Druart from comment #27) > My idea would be "something that works", I have already tons of things that > do not work. I am not sure letting other developer retrying again what did > not work is a good idea :) > Let's discuss and agree on an idea, then I can implement it. That works for me too! So long as we have some sort of timeline/deadline I think, so that we don't let the discussion go on forever? -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 14048] Change RefundLostItemFeeOnReturn to be branch specific
by bugzilla-daemon＠bugs.koha-community.org 25 Oct '18

25 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14048 Jonathan Druart <jonathan.druart(a)bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |21654 --- Comment #98 from Jonathan Druart <jonathan.druart(a)bugs.koha-community.org> --- (In reply to Jonathan Druart from comment #97) > There is one occurrence left: > C4/Installer.pm: push @fnames, C4::Context->config('intranetdir') . > "/installer/data/mysql/mandatory/refund_lost_item_fee_rules.sql"; > > Please fix! See bug 21654, thanks! Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21654 [Bug 21654] Installer is loading a non-existing file -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15486] Restrict number of holds placed by day
by bugzilla-daemon＠bugs.koha-community.org 24 Oct '18

24 Oct '18

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15486 --- Comment #90 from Jonathan Druart <jonathan.druart(a)bugs.koha-community.org> --- Created attachment 81087 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81087&action=ed… Bug 15486: Fix failing test in GetHardDueDate.t -- You are receiving this mail because: You are watching all bug changes.

1 0