- Koha-devel - lists.koha-community.org

Cache::Memory must be removed (21955)
by Jonathan Druart 21 Dec '18

21 Dec '18

Hi devs, I am still recovering from my holidays and I think I caught a quite big fish. After an interesting track game I will explain why I am suggesting to remove Cache::Memory that is currently used as fallback for the L2 cache. What I tried to fix: Jenkins is complaining about selenium tests (regressions.t) failing on 18.05, it is a succession of events and bugs that were not linked at first glance. Here are the different steps I went though: - On bug 21426 we noticed that USE_MEMCACHED was not taken into account. If set to "no" the default memcached config was defined in the koha-config.xml file anyway - A new regression selenium test was added on bug 21777 to catch the presence of an audio alert on the circulation page - Investigating the failing tests I noticed that koha-testing-docker was not setting the memcached config on the 18.05 branches (I guess the image has not been rebuilt yet) search_utf8.t output "Warning: script running in daemon mode, without recommended caching system (memcached)." - I also find that [% Koha.Preference('AudioAlerts') %] did not return the value set by the tests, but the value that the DB has before the tests were launched => It is a cache issue! - ...but only when memcached is not set... - Reading Koha::Cache->new we can notice that Cache::Memory is used for the L2 cache when memcached is not defined in the config And so we have the problem: If a value is set in the cache by a Plack worker, it will not be available from another one, as the L2 cache is not shared (!) To recreate easily the problem you can: - remove the memcached config - edit intranet-bottom.inc and add ===[% Koha.Preference('AudioAlerts') %]=== - restart plack - Modify the value of AudioAlerts (using the UI) - Reload the page (reload several times if the value is still correct, it will depend on which worker will serve the request) Solution: I am considering removing Cache::Memory unless somebody else has a better idea Bug 21955 - Cache::Memory should not be used as L2 cache Note that it should not affect a lot of people as everybody is supposed to have memcached configured and working correctly! Cheers, Jonathan

10 15

Re: [Koha-devel] [Koha] 18.05.07 Released
by Renvoize, Martin 21 Dec '18

21 Dec '18

Congratulations on your first release guys, that first one is always the hardest :) *Martin Renvoize* <https://www.ptfs-europe.com> Development Team Manager *Phone:* +44 (0) 1483 378728 *Mobile:* +44 (0) 7725 985 636 *Email:* martin.renvoize(a)ptfs-europe.com *Fax:* +44 (0) 800 756 6384 www.ptfs-europe.com Registered in the United Kingdom No. 06416372 VAT Reg No. 925 7211 30 The information contained in this email message may be privileged, confidential and protected from disclosure. If you are not the intended recipient, any dissemination, distribution or copying is strictly prohibited. If you think that you have received this email message in error, please email the sender at info(a)ptfs-europe.com On Thu, 20 Dec 2018 at 20:37, Jesse Maseto <jesse(a)bywatersolutions.com> wrote: > The Koha community is proud to announce the release of Koha 18.05.07. > > Please note, this release removes support for using Cache::Memory as your > caching strategy due to unresolvable bugs with its implementation which > cause issues in plack environments. memcached has been the recommended > caching module for some time now, so anyone still relying on Cache::Memory > should consider migrating to Memcached. > > This is a maintenance release that includes 2 enhancements, 28 bugfixes. > > The full release notes are available at https://koha-community.org/ > koha-18-05-07-released/ > <https://koha-community.org/koha-18-05-07-released/> > > Debian packages will be updated within a few days. > > Best Regards, > Jesse and Lucas > > -------------------- > Jesse Maseto > Dev/Ops Support > ByWater Solutions > Support & Consulting for OSS > Office - Stratford,CT > T/F 888.900.8944 > http://bywatersolutions.com > Jesse(a)bywatersolutions.com > _______________________________________________ > Koha mailing list http://koha-community.org > Koha(a)lists.katipo.co.nz > https://lists.katipo.co.nz/mailman/listinfo/koha >

4 3

Koha 17.11.13 release
by Fridolin SOMERS 21 Dec '18

21 Dec '18

The Koha community is proud to announce the release of Koha 17.11.13. This is a maintenance release. The full release notes are available at https://koha-community.org/koha-17-11-13-release/ Debian packages will be upgraded in a few days. Pay attention to Bug 21955 integration : Cache::Memory should not be used as L2 cache. This means Memcached is more than recommanded on production servers. Have a nice end of the year and merry christmas ;) Hohoho, -- Fridolin SOMERS <fridolin.somers(a)biblibre.com> BibLibre, France - software and system maintainer

1 0

Koha 18.11.01 release
by Renvoize, Martin 20 Dec '18

20 Dec '18

The Koha community is proud to announce the release of Koha 18.11.01. Please note, this release removes support for using Cache::Memory as your caching strategy due to unresolvable bugs with its implementation which cause issues in plack environments. memcached has been the recommended caching module for some time now, so anyone still relying on Cache::Memory should consider migrating to Memcached. This is a maintenance release containing 35 bugfixes The full release notes are available at *https://koha-community.org/koha-18-11-01-release/ <https://koha-community.org/koha-18-11-01-release/>* Debian packages will be updated within a few days That just leaves me to wish everyone a Happy Christmas and express my thanks to everyone involved in this release, *Martin Renvoize* <https://www.ptfs-europe.com> Development Team Manager *Phone:* +44 (0) 1483 378728 *Mobile:* +44 (0) 7725 985 636 *Email:* martin.renvoize(a)ptfs-europe.com *Fax:* +44 (0) 800 756 6384 www.ptfs-europe.com Registered in the United Kingdom No. 06416372 VAT Reg No. 925 7211 30 The information contained in this email message may be privileged, confidential and protected from disclosure. If you are not the intended recipient, any dissemination, distribution or copying is strictly prohibited. If you think that you have received this email message in error, please email the sender at info(a)ptfs-europe.com

1 0

Possible memory leak when using ZOOM libraries with Zebra
by David Cook 19 Dec '18

19 Dec '18

Hi all, I've been playing with the ZOOM libraries and Zebra, and it seems like there's a memory leak when using C4::Context->Zconn with the environmental variable GATEWAY_INTERFACE. I had a script that was creating a new connection for each request, and C4::Context->Zconn did look like it was calling the destroy method on the connection before creating a new one, and it was clear that each request was using a new connection, but I noticed the memory usage on the script slowly started going up. For a long-running script, the memory usage got over a gigabyte. If I re-use the connection (ie don't have GATEWAY_INTERFACE turned on), there's no memory leak. It shouldn't matter for CGI Koha, but I'm curious how this might affect Plack Koha. I haven't looked yet, but I figured it was worth sharing my observation. I suppose Plack recycles workers, so it's not dangerous, but I wonder if this is affecting the memory usage of Plack workers. Food for thought at least. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595

2 1

Re: [Koha-devel] Low Zebra performance for persistent/re-used connections
by David Cook 19 Dec '18

19 Dec '18

And now I try again today with re-using the connection and the performance is amazing and the CPU usage is much higher than 3% but nowhere near 100%. Looks like I must've just been misusing the ZOOM library perhaps.. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: David Cook [mailto:dcook@prosentient.com.au] Sent: Monday, 17 December 2018 6:07 PM To: 'koha-devel(a)lists.koha-community.org' <koha-devel(a)lists.koha-community.org> Subject: Low Zebra performance for persistent/re-used connections Hi all, I'm writing a custom script that uses C4::Context->Zconn to create a Zebra connection and then I'm sending a bunch of queries. If I don't have the GATEWAY_INTERFACE environmental variable defined, that method returns the same connection. (Note that I'm working with Unix sockets and not TCP sockets.) When re-using the same connection, my script uses 100% CPU. When using new connections for every request, my script uses 3% CPU. When re-using the same connection, Zebra seems pretty fast for the first 1000-2000 queries (I didn't capture exact numbers), but performance degrades rapidly. Around 3000 queries, Zebra is only processing about 4-6 queries per second. When using new connections, Zebra is handling about 93 queries per second. Zebra is forking a process for every connection, so it seems like the overhead should be greater creating a new process for every request, but performance is exponentially better when forcing new connections for each request than when re-using the connection. It looks like Zebra doesn't expect a single connection/worker process to handle too many requests. Or maybe it's an issue with the ZOOM library. I don't know. I don't know enough about the internals of either of them. So this could affect any command line tool that handles high volumes of Zebra queries. While re-using connections makes sense in theory, it seems to actually cause problems when talking to Zebra. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595

1 0

Low Zebra performance for persistent/re-used connections
by David Cook 17 Dec '18

17 Dec '18

Hi all, I'm writing a custom script that uses C4::Context->Zconn to create a Zebra connection and then I'm sending a bunch of queries. If I don't have the GATEWAY_INTERFACE environmental variable defined, that method returns the same connection. (Note that I'm working with Unix sockets and not TCP sockets.) When re-using the same connection, my script uses 100% CPU. When using new connections for every request, my script uses 3% CPU. When re-using the same connection, Zebra seems pretty fast for the first 1000-2000 queries (I didn't capture exact numbers), but performance degrades rapidly. Around 3000 queries, Zebra is only processing about 4-6 queries per second. When using new connections, Zebra is handling about 93 queries per second. Zebra is forking a process for every connection, so it seems like the overhead should be greater creating a new process for every request, but performance is exponentially better when forcing new connections for each request than when re-using the connection. It looks like Zebra doesn't expect a single connection/worker process to handle too many requests. Or maybe it's an issue with the ZOOM library. I don't know. I don't know enough about the internals of either of them. So this could affect any command line tool that handles high volumes of Zebra queries. While re-using connections makes sense in theory, it seems to actually cause problems when talking to Zebra. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595

1 0

Call for News: December 2018 Newsletter
by kohanews 14 Dec '18

14 Dec '18

I'm collecting news for the December newsletter. Send anything noteworthy to: k o h a news AT gmail dot com News criteria: --------------------------- ** For events **: - Please include dates for past events. If I can't find dates I may not add it. - Announcements for future events with dates T.B.A. are fine ...Eg., Kohacon - For past events , **** one month back is the cut-off ****. * News items can be of any length. * Images are fine * Anything and everything Koha. * Submit by the 26th of the month. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. Thank you! -- Chad Roseburg Editor, Koha Community Newsletter

1 0

Missing FK on serial tables (?)
by Jonathan Druart 13 Dec '18

13 Dec '18

Hi devs, A quick question, in case I am missing something obvious. Reported on bug 21901: indices are missing on the serial and subscription tables. Worst is, in the serial table: `biblionumber` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '', `subscriptionid` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '', So I think we should: Add FK (with on delete cascade) on - serial.biblionumber - serial.subscritionid - subscription.biblionumber Would it make sense? Cheers, Jonathan

2 1

Cover image 'thumbnail' size is bigger than 'imagefile' size
by Andreas Roussos 12 Dec '18

12 Dec '18

Dear Developers, We use local cover images in our setup, and also resize the covers we scan to a width of 160px before uploading and attaching them to a Koha bibliographic record (we upload one cover per biblio). While using phpMyAdmin to view the contents of the 'biblioimages' table (relevant screenshot here: https://imgur.com/a/cSEVLBI) we noticed that the size of the BLOBs for the 'thumbnail' column was in some cases twice as big as that of the 'imagefile' column. In fact, this happens for more than 50% of uploaded covers as you can see from the output of the SQL queries below: mysql> SELECT COUNT( * ) AS count FROM biblioimages WHERE LENGTH( thumbnail ) > ( LENGTH( imagefile ) * 2 ) ; +-------+ | count | +-------+ | 1356 | +-------+ mysql> SELECT COUNT( * ) AS count, SUM( LENGTH( imagefile ) ) AS images_size, SUM( LENGTH( thumbnail ) ) AS thumbnails_size FROM biblioimages ; +-------+-------------+-----------------+ | count | images_size | thumbnails_size | +-------+-------------+-----------------+ | 2347 | 68323933 | 115839686 | +-------+-------------+-----------------+ It would appear that for each 160px-wide JPG with 24-bit depth that we have uploaded, the 'imagefile' column has been populated with an PNG of 8 bit depth with the same dimensions as the uploaded file, whereas the 'thumbnail' column contains a 24-bit PNG image with a width reduced to 140 pixels. I've tracked down the creation of resized 24-bit PNGs to this code: https://github.com/Koha-Community/Koha/blob/master/C4/Images.pm#L182-L183 Do thumbnails for covers need to be true colour (24-bit) images? Also, why is it that _scale_image() returns an image of 8-bit depth if the source image's dimensions are less than 600x800? (i.e. when no resizing is performed) Thank you in advance for your time. Kind regards, Andreas

3 3