December 2020 - Koha-devel - lists.koha-community.org

Mail list footer
by dcook＠prosentient.com.au 21 Dec '20

21 Dec '20

Hi all, I just noticed that a lot of the links on the "Koha-devel mailing list" footer are in HTTP instead of HTTPS. It's not a big drama since they all redirect to HTTPS, but we may as well update those to HTTPS and skip the extra web server request. Not 100% sure who manages this list. Maybe BibLibre? David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

2 1

Load balancing Zebra
by dcook＠prosentient.com.au 20 Dec '20

20 Dec '20

Hi Mengu, Do you use the YAZ Proxy (https://www.indexdata.com/resources/software/yazproxy/) at all for load balancing Zebra? How do you distribute updates to your different Zebra servers? I have a vague memory that you index one of them and then copy the Zebra database over periodically? David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

3 2

Misconfigured lists?
by Tomas Cohen Arazi 18 Dec '20

18 Dec '20

Hi all, is there a reason for the lists to have the 'Reply-to' set to the original sender? This makes it very easy to inadvertently go private on answering. The option is to choose to 'Reply to all', but this is really sub optimal, as people might receive the message several times, and the lists are configured to reject messages sent to more than X senders (I'm not sure about X, I think it is set to 3). I would propose to set the lists reply-to to the list itself. -- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F

3 2

Re: [Koha-devel] Zebra ICU issues
by dcook＠prosentient.com.au 18 Dec '20

18 Dec '20

Hi all, Slowly making progress at https://github.com/indexdata/idzebra/issues/24. It looks like some of the issues I've found so far are due to us using tokenization "l" instead of "s". However, it looks to me like even the latest Zebra is buggy and doesn't properly index phrases in the "p" register. Hoping Indexdata can fix things up based on my investigation (or point me on the right path). Note that my tests are being done in koha-testing-docker and I've used Zebra 2.0.59 and Zebra 2.2.2. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces(a)lists.koha-community.org> On Behalf Of dcook(a)prosentient.com.au Sent: Thursday, 17 December 2020 1:31 PM To: 'Fridolin SOMERS' <fridolin.somers(a)biblibre.com>; koha-devel(a)lists.koha-community.org Subject: Re: [Koha-devel] LinkBibHeadingsToAuthorities not working as expected Thanks, Fridolin! I've reported it to Indexdata but they say they can't reproduce the problem... David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces(a)lists.koha-community.org> On Behalf Of Fridolin SOMERS Sent: Wednesday, 16 December 2020 7:54 PM To: koha-devel(a)lists.koha-community.org Subject: Re: [Koha-devel] LinkBibHeadingsToAuthorities not working as expected Thanks a lot for your tests. I will try on our Bionic env with Zebra 2.2.1 Le 11/12/2020 à 06:56, dcook(a)prosentient.com.au a écrit : > This one is driving me a bit crazy, so I’ve logged an issue with > Indexdata and I’m hoping for the best: > https://github.com/indexdata/idzebra/issues/24 > <https://github.com/indexdata/idzebra/issues/24>. > > David Cook > > Software Engineer > > Prosentient Systems > > Suite 7.03 > > 6a Glen St > > Milsons Point NSW 2061 > > Australia > > Office: 02 9212 0899 > > Online: 02 8005 0595 > > *From:*Koha-devel <koha-devel-bounces(a)lists.koha-community.org> *On > Behalf Of *dcook(a)prosentient.com.au > *Sent:* Friday, 11 December 2020 4:31 PM > *To:* 'Koha Devel' <koha-devel(a)lists.koha-community.org> > *Subject:* Re: [Koha-devel] LinkBibHeadingsToAuthorities not working > as expected > > Using koha-testing-docker set up for ICU and with my fix from > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198 > <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198>: > > querytype ccl2rpn > > set_cclfile /etc/koha/zebradb/ccl.properties > > format xml > > elements zebra::snippet > > Z> f Match-heading,phr,ext,do-not-truncate="the Q" > > Sent searchRequest. > > Received SearchResponse. > > Search was a success. > > Number of hits: 34, setno 24 > > SearchResult-1: term=the cnt=34 > > records returned: 0 > > Elapsed: 0.000315 > > Z> show > > Sent presentRequest (1+1). > > Records: 1 > > Record type: XML > > <record xmlns="http://www.indexdata.com/zebra/ > <http://www.indexdata.com/zebra/>"> > > <snippet name="Match-heading" type="p" > fields="Match">Xenophon<s>the</s>Historian</snippet> > > </record>nextResultSetPosition = 2 > > Elapsed: 0.003864 > > So that’s special… admittedly that’s Zebra 2.0.59… > > On Zebra 2.1.4 with ICU: > > Z> f Match-heading,phr,ext,do-not-truncate="the Q" > > Sent searchRequest. > > Received SearchResponse. > > Search was a success. > > Number of hits: 0, setno 2 > > SearchResult-1: term=the cnt=5383, term=Q cnt=10 > > records returned: 0 > > Elapsed: 0.009691 > > Z> f Match-heading,phr,ext,do-not-truncate="the" > > Sent searchRequest. > > Received SearchResponse. > > Search was a success. > > Number of hits: 85, setno 3 > > SearchResult-1: term=the cnt=85 > > records returned: 0 > > Elapsed: 0.002209 > > However, 85 results is still too many. It should be 0 results. > > I can’t add the diagnostics from > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198 > <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198> > right now though… so will have to get back to this one another day probably. > > But maybe someone else using Zebra with ICU can look into this problem too. > > It’s leading to lots of duplicate authorities it seems… > > David Cook > > Software Engineer > > Prosentient Systems > > Suite 7.03 > > 6a Glen St > > Milsons Point NSW 2061 > > Australia > > Office: 02 9212 0899 > > Online: 02 8005 0595 > > *From:*Koha-devel <koha-devel-bounces(a)lists.koha-community.org > <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of > *dcook(a)prosentient.com.au <mailto:dcook@prosentient.com.au> > *Sent:* Friday, 11 December 2020 4:22 PM > *To:* 'Koha Devel' <koha-devel(a)lists.koha-community.org > <mailto:koha-devel@lists.koha-community.org>> > *Subject:* Re: [Koha-devel] LinkBibHeadingsToAuthorities not working > as expected > > Ugh yeah no… I’m reproducing it with koha-testing-docker too. > > Looks like yet another ICU bug… > > David Cook > > Software Engineer > > Prosentient Systems > > Suite 7.03 > > 6a Glen St > > Milsons Point NSW 2061 > > Australia > > Office: 02 9212 0899 > > Online: 02 8005 0595 > > *From:*Koha-devel <koha-devel-bounces(a)lists.koha-community.org > <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of > *dcook(a)prosentient.com.au <mailto:dcook@prosentient.com.au> > *Sent:* Friday, 11 December 2020 3:50 PM > *To:* 'Koha Devel' <koha-devel(a)lists.koha-community.org > <mailto:koha-devel@lists.koha-community.org>> > *Subject:* Re: [Koha-devel] LinkBibHeadingsToAuthorities not working > as expected > > Then again… I can’t reproduce this problem on koha-testing-docker, but > I can on a prod Koha running Zebra 2.1.4 with… a very strange > /etc/koha/zebradb/etc/default.idx file… > > David Cook > > Software Engineer > > Prosentient Systems > > Suite 7.03 > > 6a Glen St > > Milsons Point NSW 2061 > > Australia > > Office: 02 9212 0899 > > Online: 02 8005 0595 > > *From:*Koha-devel <koha-devel-bounces(a)lists.koha-community.org > <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of > *dcook(a)prosentient.com.au <mailto:dcook@prosentient.com.au> > *Sent:* Friday, 11 December 2020 3:32 PM > *To:* 'Koha Devel' <koha-devel(a)lists.koha-community.org > <mailto:koha-devel@lists.koha-community.org>> > *Subject:* [Koha-devel] LinkBibHeadingsToAuthorities not working as > expected > > Hi all, > > I’m still investigating, but it seems to me that the > C4::Linker::Default in C4::Biblio::LinkBibHeadingsToAuthorities isn’t > searching accurately using C4::Heading::authorities. > > I’m looking at C4::AuthoritiesMarc::SearchAuthorities and at a glance > it looks OK, but in practice I think that my search queries are > getting way too many results. > > By hand, if I try the following query: > > Match-heading,phr,ext,do-not-truncate="e" > > I get a huge number of results, which is odd, since that should be an > “exact” match. > > I’ve opened > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198 > <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27198> > because I need to improve the diagnostics available for a Zebra > authorities database, but yeah… not good. Hopefully I’ll know more soon. > > David Cook > > Software Engineer > > Prosentient Systems > > Suite 7.03 > > 6a Glen St > > Milsons Point NSW 2061 > > Australia > > Office: 02 9212 0899 > > Online: 02 8005 0595 > > > _______________________________________________ > Koha-devel mailing list > Koha-devel(a)lists.koha-community.org > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ git : > http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ > -- Fridolin SOMERS <fridolin.somers(a)biblibre.com> Software and system maintainer 🦄 BibLibre, France _______________________________________________ Koha-devel mailing list Koha-devel(a)lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ _______________________________________________ Koha-devel mailing list Koha-devel(a)lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/

1 0

Koha Command and Control
by dcook＠prosentient.com.au 18 Dec '20

18 Dec '20

Hi all, For a while, I've been thinking that it might be good to have an open source "Koha Command and Control" for managing Koha instances. While managing 1 Koha is not too much work, it's considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc. I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)? I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don't think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise. I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we'd configure Koha to push events to the "Koha Command and Control" (which would be configured by the Koha system administrator - not librarian level). If Zebra logs a 'ZOOM error 10014 "CCL parsing error"', we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they're better placed to raise Bugzilla reports/write patches. Anyway, it's just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don't think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful. I have to run but I'll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

5 7

Memory usage by Koha daemons (rebuild_zebra.pl, SIPServer.pm, etc)
by dcook＠prosentient.com.au 18 Dec '20

18 Dec '20

Hi all, I was thinking about memory usage by Koha daemons in light of a koha listserv post about Koha resource requirements. I notice that rebuild_zebra.pl uses about 170MB when idle. While that's not staggering on its own, that does add up. That's 1.7GB for 10 processes, 3.4GB for 20 processes, 5.1GB for 30 processes, and so on. You get the idea. That reminds me that in early 2018 on our openSUSE Koha server, I actually wrote a Koha indexer daemon that only uses 12MB of memory when idle. It periodically checks the databases for every Koha instance on that server and launches rebuild_zebra.pl as necessary (in non-daemon mode) and uses advisory file locking to avoid running duplicate index jobs for an index. The memory footprint is tiny. I can't recall the exact number of instances that covered but a conservative estimate would be 60. That's 1 process using 12MB vs 60 processes collectively using 10.2GB of RAM. I've opened https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27267 to explore this further. In fact, as I was writing comments there, I realized that an alternative would be to keep 1 process per Koha instance, but to make that a very lightweight process which just checks if there is Zebra indexing to do, and to launch rebuild_zebra.pl if there is. I suppose there pros and cons to both centralized and decentralized models. We already use decentralized so may as well stay that way and just optimize it. I think that I could actually use some paid work time to do this too, so if people are happy with this last approach, I'd go ahead and do it. -- I notice that SIPServer.pm also uses about 165MB of RAM when it's idling. That also adds up. However, I don't know how much of that is due to Net::Server::PreFork and how much of it is our Koha code. I'd have to explore that more, and since I don't run tonnes of SIP servers, it's not a priority for me. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Why does #wrap have padding?
by dcook＠prosentient.com.au 17 Dec '20

17 Dec '20

Hi all, Does anyone know why #wrap in koha-tmpl/opac-tmpl/bootstrap/css/src/_common.scss has padding-left: 40px and padding-right: 40px? It makes adding full-width headers/footers impossible without tricks and compensations for changes to the padding with the responsive media queries. You end up writing pages of CSS just for 1 banner. Or am I missing something? David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

3 3

Strange characters
by Fridolin SOMERS 17 Dec '20

17 Dec '20

Hi, I found some strange characters in sources : https://git.koha-community.org/Koha-community/Koha/src/branch/master/tools/… It se a <?> : Casta�eda, Carlos Sebastian Do you see that ? Is this non-UTF8 ? Can we build a command to find them all ? I've tried with 'grep -P' but impossible. Best regards, -- Fridolin SOMERS <fridolin.somers(a)biblibre.com> Software and system maintainer 🦄 BibLibre, France

3 2

Re: [Koha-devel] [Koha] Red Hat 8, installing dependencies
by dcook＠prosentient.com.au 14 Dec '20

14 Dec '20

Hi Tasha, It's great to hear that you're trying to install Koha! However, if I were you, I wouldn't bother with that old wiki page, as it's probably going to confuse more than help. Typically, the Koha community supports Koha installations on Debian and Ubuntu operating systems, but there are some of us who run Koha on openSUSE, Red Hat, Fedora, etc. We are rare though. Wherever possible, it's best to use Debian/Ubuntu as you'll get the best, supported experience that way, but I know in a corporate environment that you're often limited to only using Red Hat Enterprise Linux. Part of the reason your email stood out to me is that I am in the process of migrating a non-Koha Perl application from RHEL 6 to RHEL 8, so I'm already in this headspace. In all honesty, if I were you, I'd try to find a vendor that is willing - for a fee - to install Koha on your local RHEL 8 system. You can find more information at https://koha-community.org/support/paid-support/. We have installed Koha on RHEL for libraries with restricted corporate environments like you describe. I am sure there are vendors in Canada and the USA who have done it too who could help you out. Feel free to email me separately about that if you want to discuss that further. -- If you are committed to doing it yourself without paid help, then it's important to think about Koha as a multi-component system: Linux OS, Apache httpd web server, MySQL database, Perl application, Zebra indexing engine. You've already got your Linux OS (RHEL 8), and you'll be able to use the Apache httpd web server and MySQL database from the Red Hat repositories. The difficulty will be with the Perl application (Koha proper) and Zebra. Looking at https://www.indexdata.com/resources/software/zebra/, there is a "Redhat" download option, but it does not look like the RHEL packages have been maintained. You could contact Indexdata to get them to publish new packages for RHEL 8. Alternatively, it looks like they do provide some for CentOS 8, which is *currently* compatible with RHEL 8: http://ftp.indexdata.dk/pub/zebra/redhat/centos/8/README. I haven't used that but that would be my first step *before* trying to build Zebra from source. As for the Perl application (Koha proper), it's easier than ever to install dependencies thanks to the "cpanfile" provided with Koha. It provides a list of ~180 Perl dependencies, which you can install using a tool called "cpanminus" (https://metacpan.org/pod/App::cpanminus) Unfortunately for you, these dependencies need to be fetched over the Internet from the "CPAN" (Comprehensive Perl Archive Network). However, perhaps you'd be able to talk to your IT about setting up a local offline mirror of CPAN. You can use the "minicpan" tool for that: https://metacpan.org/pod/distribution/CPAN-Mini/bin/minicpan. I have used that tool before in restricted corporate environments for Perl applications. Now that sounds easier than it actually is. You mentioned that you had some trouble installing some packages like "ImageMagick-devel". You need "*-devel" packages when you're building code from course (like you'd be doing using "cpanminus" and the "cpanfile" via "minicpan"), as they provide interfaces to lower level software tools (e.g. "ImageMagic" without the -devel). Typically, you discover which "-devel" packages you need when particular Perl dependencies fail to install. If you're successful with all of this, then there is the task of setting up a Zebra "service" and Koha "cronjobs". This also all assumes that you're not use SELinux security on RHEL 8 as well. That is a layer of complexity that is too difficult to explain in this space. -- Alternatively, you could try running Docker containers - since Docker should be available in RHEL 8, or Virtual Machines on your RHEL 8 machine, and have those containers or virtual machines run Koha on Debian/Ubuntu. But that adds a different layer of complexity. -- Hopefully that was helpfully comprehensive rather than terrifyingly comprehensive ; ). For years, I have been thinking of putting together RPM packages that could be installed on Red Hat or openSUSE, but it would require more resources than I have available, so I've been in the gradual process of switching to only using Debian/Ubuntu for Koha wherever possible. But I appreciate that you might not have that option. Due to your limitations, I think your best bet is to get a paid support vendor to help you, but you can also try what I've suggested and try working through it here. You probably won't be able to receive much more than general assistance, since the majority of people use Debian/Ubuntu, but you can always try. Bon courage! David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- Date: Sat, 12 Dec 2020 21:37:08 +0000 From: "Bales (US), Tasha R" <tasha.r.bales(a)boeing.com> To: "koha(a)lists.katipo.co.nz" <koha(a)lists.katipo.co.nz> Subject: [Koha] Red Hat 8, installing dependencies Message-ID: <6d2acf16030d48a4b6f3885b47ae8b17(a)boeing.com> Content-Type: text/plain; charset="us-ascii" Good afternoon, I'm trying to install the latest version of Koha on Red Hat 8.1, using the Koha Wiki instructions<https://wiki.koha-community.org/wiki/Koha_on_Redhat_Enterprise_6> for Red Hat 6. Perhaps you can help with a couple questions. First, the Wiki contains a list of about 70 dependencies that should be installed before Koha. Since the Wiki article is dated, is there a method for determining whether this list is still accurate for Red Hat 8 and current Koha? Or, would one just need to be on the lookout for inevitable errors and complaints about missing dependencies? Next, I'm limited to working with a corporate repository-I cannot connect to the internet--but I was able to install all but 11 of the recommended packages, see below. I don't necessarily know that all of these are critical requirements, but some of them seem significant when I Googled them for more info. They are as follows: * cloog-ppl * ghostscript-devel (I have the non-devel package) * gnome-utils-libs (I don't think we need this) * ImageMagick * ImageMagick-devel * Imake (May not need this) * jasper-devel (We have jasper-libs) * lcms-devel * libXdmcp-devel * libxml2-python * ppl If you can advise where I ought to look to gain an understanding of whether these missing packages are 1) true requirements for Koha 20.x, given I obtained them from an older Wiki article; 2) show-stoppers that will prevent using Koha, or certain features of Koha, if they are not present. I appreciate and thank you in advance for any advice you may have. I am doing a good amount of research on the web to try to answer my own questions, but I'm a bit out of my league. Thanks for your time and consideration, Tasha R. Bales

1 0

Call for news - Newsletter December 2020
by Koha Newsletter 13 Dec '20

13 Dec '20

Hi I'm collecting news for the December 2020 Koha Community Newsletter. Please send anything noteworthy to: kohanews (at) gmail (dot) com News criteria: * News items can be of any length. * Images are fine. * Anything and everything Koha. * Submit by the 26th of the month. For events: * Please include dates for past events. If I can't find dates I may not add it. * Announcements for future events with dates to be announced are fine, e. g. Kohacon. * For past events, one month back is the cut-off. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. Thank you! Michael Kuhn Editor, Koha Community Newsletter

1 0