Send Koha-devel mailing list submissions to
    koha-devel@lists.koha-community.org

To subscribe or unsubscribe via the World Wide Web, visit
    https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
or, via email, send a message with subject or body 'help' to
    koha-devel-request@lists.koha-community.org

You can reach the person managing the list at
    koha-devel-owner@lists.koha-community.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Koha-devel digest..."


Today's Topics:

1. Re: Packaging idea (for decoupling systems, Docker, etc)
(Renvoize, Martin)
2. Re: Optimizing Starman startup (dcook@prosentient.com.au)


----------------------------------------------------------------------

Message: 1
Date: Wed, 21 Apr 2021 07:48:49 +0100
From: "Renvoize, Martin" <martin.renvoize@ptfs-europe.com>
To: David Cook <dcook@prosentient.com.au>
Cc: Mason James <mtj@kohaaloha.com>, Koha Devel
    <koha-devel@lists.koha-community.org>, Tomas Cohen Arazi
    <tomascohen@theke.io>
Subject: Re: [Koha-devel] Packaging idea (for decoupling systems,
    Docker, etc)
Message-ID:
    <CAB7SL8D3TWH+yh7E7eQ=m8W2YdNy623hQ-ZCu6Y-x3rzCr7v9w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

This is another area where I think our thoughts align David. I've also
wanted to split his into even smaller units for a while.. a clear example
is our move to background jobs. I see those as prime targets for horizontal
scaling.. but as you've highlighted, at the moment to get the library of
core code you may want to write a job you need to install the whole app..
koha-libs would certainly help with that

On Tue, 20 Apr 2021, 2:12 am , <dcook@prosentient.com.au> wrote:

> Hi Mason,
>
> [FYI this started as a response and turned into a
> stream-of-consciousness.][Also really keen to hear from Tomas and Agustin
> about their work with Docker/Kubernetes/Helm/etc.]
>
> The idea with "koha-core" was to separate out the third-party systems away
> from the core application. In this case, "koha-libs" would be about
> separating the core libraries away from the core application.
>
> Consider that the "koha-core" package contains the full Koha application
> (including code, templates, images, other assets, etc) and its application
> dependencies (e.g. at, cron, daemon, starman, sudo, unzip, xmlstarlet, yaz,
> mysql-client). My latest koha-core package alone was 38MB. A "koha-libs"
> package would probably be measured in terms of KB.
>
> I'm thinking about the absolute minimum required to run things like the
> SIP server or some other service/microservice (e.g. z3950-responder, REST
> service, etc). The koha-libs package would need to contain the Koha .pm
> files and koha-libs would depend on koha-perldeps.
>
> In theory, over time, some of the libraries and dependencies would
> probably move out of koha-libs and koha-perldeps to live in the individual
> services, but I don't have a full vision for that yet.
>
> You could think about a koha-libs package as a Perl API rather than a HTTP
> API, which could be used by Koha's existing background services, and it
> could be used by new separate services.
>
> I have a vision for creating a service that depends on koha-libs, points
> at a koha-conf.xml file, and then implements its own functionality separate
> from the main Koha codebase.
>
> It could allow for a lot of power and flexibility.
>
> --
>
> That said, maybe I'm thinking too small and too Debian-focused. Maybe a
> CPAN distribution of Koha/ modules makes more sense (or both koha-libs and
> a CPAN distribution). Apparently we could use the cpanfile to specify the
> dependencies for the CPAN distribution too. Then you could install it on
> any suitable OS/container. In a container environment, you could always
> install the Koha/ modules and dependencies on a volume, and then share that
> volume with your containers. That would have some pros and cons.
>
> I'm mostly just brainstorming. Thinking about how we can re-organize
> things to take advantage of modern technology. But then to what end?
>
> One advantage of Docker is to run different versions of the same software
> on the same virtual machine...
>
> While I was thinking about 1+ containers per Koha library, maybe we do
> just go with a 20.05 Koha container, a 20.11 Koha container, a 21.05 Koha
> container. I suppose each of those versions could have multiple containers
> though. For example, a SIP server container, a Z39.50 responder container,
> a Zebra indexer container, a Zebra server container, etc. Technically
> speaking, if we planned the volumes right, the Koha "etc" config and
> "share" and "var" volumes could all be shared across those containers. Of
> course, that means that all the containers have to be running on the same
> host. At the scale of most Koha libraries, that would probably be fine. I
> suppose the main change here would be to the debian scripts and koha-common
> init script...
>
> --
>
> Overall, I suppose though most vendors would win from improved
> multi-tenancy. Having 100 Starman instances for 50 Koha libraries where
> only 3 libraries might have requests at a time is massive overhead.
>
> I imagine improved multi-tenancy coupled with multiple versions per host
> via Docker... that would be a nicer setup.
>
> Preloading Koha::REST::V1 and getting its work done in the master starman
> process rather than the child processes would be better too...
>
> Ok, that's enough ranting for me for now heh. Too many ideas and never
> enough time.
>
> David Cook
> Software Engineer
> Prosentient Systems
> Suite 7.03
> 6a Glen St
> Milsons Point NSW 2061
> Australia
>
> Office: 02 9212 0899
> Online: 02 8005 0595
>
> -----Original Message-----
> From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf
> Of Mason James
> Sent: Friday, 16 April 2021 3:50 PM
> To: koha-devel@lists.koha-community.org
> Subject: Re: [Koha-devel] Packaging idea (for decoupling systems, Docker,
> etc)
>
> hi David
>
> i think we may already have this functionality with the 'koha-core'
> package?
>
>
> On 15/04/21 3:32 pm, dcook@prosentient.com.au wrote:
> >
> > Hey all,
> >
> > What do people think about creating a “koha-libs” package which just
> contains Koha’s libraries (ie C4/, Koha/, etc)?
> >
> > I’m not as familiar with DEB packaging as I am with RPM packaging, but I
> recently did it with a RPM-based project I manage. It allowed me to easily
> create other services with the same dependencies. I created these other
> services in other Docker containers, and I was able to use my core
> application libraries without having to install the entire application in
> every container. I share the application configuration file between them
> and that’s it.
> >
> > It’s not a perfect solution. Really what I want is app-service1,
> app-service2, app-service3, and app-common with app-common containing the
> shared libraries. But I like it as an intermediate step.
> >
> > And I would be lying if it didn’t have pros and cons. One con is
> updating a library in app-libs, when I need a change in app-service1, and
> then making sure that all my deployments have the right updated library.
> More overhead than just having a monolithic application. On the other hand,
> if I want to make a change just to app-service1, I can just update it
> without having to affect any other parts of the application. That added
> overhead also gives incentive for writing cleaner testable code which does
> the right thing in the first place.
> >
> > Anyway, just food for thought.
> >
> > David Cook
> >
> > Software Engineer
> >
> > Prosentient Systems
> >
> > Suite 7.03
> >
> > 6a Glen St
> >
> > Milsons Point NSW 2061
> >
> > Australia
> >
> > Office: 02 9212 0899
> >
> > Online: 02 8005 0595
> >
> >
> > _______________________________________________
> > Koha-devel mailing list
> > Koha-devel@lists.koha-community.org
> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> > website : https://www.koha-community.org/ git :
> > https://git.koha-community.org/ bugs :
> > https://bugs.koha-community.org/
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/ git :
> https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20210421/be7867df/attachment-0001.htm>

------------------------------

Message: 2
Date: Wed, 21 Apr 2021 17:26:44 +1000
From: <dcook@prosentient.com.au>
To: "'Renvoize, Martin'" <martin.renvoize@ptfs-europe.com>
Cc: "'Koha Devel'" <koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup
Message-ID: <024201d7367f$ae746890$0b5d39b0$@prosentient.com.au>
Content-Type: text/plain; charset="utf-8"

He did answer back fairly quickly but not a very helpful way. Maybe I just wasn’t asking the right questions.



Agreed. I wish that we kept up better with Mojolicious.



I was actually wondering why we don’t use Hypnotoad for the REST API. Is it just because of the extra process management? I recall someone (Julian or Kyle?) did some work to make Koha into a Mojolicious application and I have some pending work that uses Mojolicious more, so it would be nice to further embrace it and to use Hypnotoad.



David Cook

Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia



Office: 02 9212 0899

Online: 02 8005 0595



From: Renvoize, Martin <martin.renvoize@ptfs-europe.com>
Sent: Wednesday, 21 April 2021 4:40 PM
To: David Cook <dcook@prosentient.com.au>
Cc: Koha Devel <koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup



I've always found the OpenAPI Dev finally and responsive.



However, it's worth remembering were really not using the module as it's intended. Mojolicious D the ecosystem of modules around it are generally fast moving and we don't keep up at all.. we also deliver into the guts frequently and do things the authors never expected and finally we not using any of the recommended deployment options. Hypnotoad is by far the most optimal way of running mojo and this module.



We have so many layers of obfuscation in our code it's hard to comprehend.. various layers virtualizing other layers etc.



On Wed, 21 Apr 2021, 6:23 am , <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote:

It looks like newer version of the plugin might have cached spec/schema functionality, but I'm not finding the plugin author very helpful so far. My guess is that maybe... maybe... Mojolicious::Plugin::OpenAPI 3.41+ with JSON::Validator 4.10+ might perform better. But it's just a guess at this point.

David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>
Sent: Wednesday, 21 April 2021 1:21 PM
To: 'Ere Maijala' <ere.maijala@helsinki.fi <mailto:ere.maijala@helsinki.fi> >; koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup

Looking at newer versions of Mojolicious::Plugin::OpenAPI and the coding choices are... interesting.

I opened a ticket against JSON::Validator (https://github.com/jhthorsen/json-validator/issues/246), but maybe I should open it against Mojolicious::Plugin::OpenAPI. We'll see how responsive the author is on Github.

Of course, we're using such ancient versions of the Mojolicious modules that even convincing the author to cache validation or skip validation won't really help us in the short- to medium- term.

At the moment, I think that I might just have to resign myself to Koha being slow to start up, as we're beholden to this third-party module.

That said, we could fork Mojolicious::Plugin::OpenAPI.

I suppose I could just do that locally too. Hmmmm.

David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>
Sent: Wednesday, 21 April 2021 12:12 PM
To: 'Ere Maijala' <ere.maijala@helsinki.fi <mailto:ere.maijala@helsinki.fi> >; koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup

That was an interesting experience.

When you run koha-plack --start, it spins through a long list of instances quickly, because it returns immediately after starting the Starman master, which doesn't really do much, but fork off worker processes, which then consume a huge amount of resources concurrently.

However, when you preload the app using --preload-app, the resource consumption is in the Starman master, and the command doesn't return until after the Starman master has loaded the app. So you synchronously move through the list 1 by 1.

This has pros and cons.

Pro: You can start 60 Koha instances with very little resource usage, because you're only starting 1 Koha at a time. Yay!
Con: "service koha-common restart" will stop all Plack instances and then start them all. This is suboptimal, as it took me about 3.5 minutes to start 60 Koha instances. That's a lot of downtime for the last Koha instance in that list.

Of course, we could change "koha-common" init service to actually restart instead of stop/start on a "restart" action.

But the other gotcha is Debian package installs/upgrades (hello continuous deployment). According to https://manpages.debian.org/testing/debhelper/dh_installinit.1.en.html, the init script is "stopped" in the prerm and "started" in the postinst. The documentation is not clear if it's at the start or end of postinst...) Based on my review of syslog and /var/log/apt/history.log, it looks like it's actually started at the beginning of postinst, which is before koha-schema-upgrade actually runs. I suppose that makes sense as that should put Koha into maintenance mode until the schema upgrade completes...

Operations is so not fun heh.

Really at the end of the day I guess it comes back to trying to optimize that OpenAPI plugin to reduce the resource consumption and resulting downtime.

David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>
Sent: Wednesday, 21 April 2021 10:46 AM
To: 'Ere Maijala' <ere.maijala@helsinki.fi <mailto:ere.maijala@helsinki.fi> >; koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup

Thanks for that optimization. My restart time has gone from 2 minutes 45 seconds to 1 minute 30 seconds. Much better.

Now I'm going to try the preload and see if I can get it down to 45 seconds or better...

David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>
Sent: Wednesday, 21 April 2021 10:28 AM
To: 'Ere Maijala' <ere.maijala@helsinki.fi <mailto:ere.maijala@helsinki.fi> >; koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup

Hi Ere,

Thanks for your reply. 24700 looks much better. I'll look at backporting it locally.

Although I'm looking at JSON::Validator::OpenAPI::Mojolicious at https://metacpan.org/pod/release/JHTHORSEN/Mojolicious-Plugin-OpenAPI-2.19/lib/JSON/Validator/OpenAPI/Mojolicious.pm and it says "Do not use this module directly. Use Mojolicious::Plugin::OpenAPI instead." I notice that you're using the "bundle" method. Do we really need that there? Why don't we just load the JSON using the JSON module, merge with the plugin spec files, and then pass it to the OpenAPI plugin? Shouldn't the plugin take care of the $ref replacement?

Hmm... I didn't realize until now that the OpenAPI plugin was doing a validate behind the scenes. That's tricky.

At a glance, we might be able to pre-load the app into the Starman master process pre-fork. There are warnings about doing that with open database connections, so we'd need to review plack.psgi, but a quick glance suggests it might be OK. (Alternatively, I have wondered about running the REST API as a separate process apart from Starman using hypnotoad. According to https://docs.mojolicious.org/Mojolicious/Guides/Cookbook, Mojo::Server::Prefork preloads the application in the manager/master process, and Hypnotoad is based off that, so that would help.)

It does seem like changes to the OpenAPI plugin would be needed for caching.

I'm going to try backporting your change and try pre-loading and see how far that gets me.

David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of Ere Maijala
Sent: Tuesday, 20 April 2021 4:48 PM
To: koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org>
Subject: Re: [Koha-devel] Optimizing Starman startup

Hi,

I did some work on improving it here:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700

That shaved a good bit of time from it, but it's still a heavy operation, and it would make sense to

1.) avoid doing it too often

2.) cache the results and avoid doing it if results are cached

If you could address the first one, that'd go a long way. I'm afraid the second one would require changes to the OpenAPI plugin to support caching.

--Ere

dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> kirjoitti 20.4.2021 klo 6.15:
> Hi all,
>
> Do you despair when you see the following periodically in “top” when a
> starman worker is recreated ?
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
> COMMAND
>
> 9529 my-koha 20 0 460108 197212 17172 R 100.0 0.4 0:03.41
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> Or the following in top when you install koha-common package or
> restart the koha-common service?
>
> 11101 1-koha 20 0 447232 193320 16076 R 10.6 0.4 0:09.09
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11168 1-koha 20 0 447240 193264 16056 R 10.6 0.4 0:08.72
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11306 2-koha 20 0 447220 193148 16000 R 10.6 0.4 0:08.07
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11543 2-koha 20 0 447232 193036 15828 R 10.6 0.4 0:07.07
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11784 3-koha 20 0 441536 189664 16172 R 10.6 0.4 0:06.04
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11830 3-koha 20 0 439548 187212 15748 R 10.6 0.4 0:05.82
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11831 4-koha 20 0 438620 186344 15748 R 10.6 0.4 0:05.81
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> 11853 4-koha 20 0 437680 185672 16000 R 10.6 0.4 0:05.79
> /usr/share/koha/api/v1/app.pl <http://app.pl>
>
> Well, I still have a lot of investigation left to do, but I notice 1
> place that a lot of time taken is here (per worker):
>
> my $validator = JSON::Validator::OpenAPI::Mojolicious->new;
>
> $validator->load_and_validate_schema(
>
> $self->home->rel_file("api/v1/swagger/swagger.json"),
>
> {
>
> allow_invalid_ref => 1,
>
> }
>
> );
>
> push @{$self->routes->namespaces}, 'Koha::Plugin';
>
> my $spec = $validator->schema->data;
>
> $self->plugin(
>
> 'Koha::REST::Plugin::PluginRoutes' => {
>
> spec => $spec,
>
> validator => $validator
>
> }
>
> );
>
> $self->plugin(
>
> OpenAPI => {
>
> spec => $spec,
>
> route =>
> $self->routes->under('/api/v1')->to('Auth#under'),
>
> allow_invalid_ref =>
>
> 1, # required by our spec because $ref directly
> under
>
> # Paths-, Parameters-, Definitions- & Info-object
>
> # is not allowed by the OpenAPI specification.
>
> }
>
> );
>
> Anyone have ideas for improving this? Do we have to validate the
> schema every time? Can we move the schema validation into a different
> module and preload it into Starman using the -M flag so that it’s done
> 1 time per Starman master instance rather than 1 time per Starman worker instance?
>
> I find “/usr/share/koha/api/v1/app.pl <http://app.pl> ” to be the bane of deployments,
> as it puts a massive load on a server, when you have multiple Koha
> instances on the server.
>
> David Cook
>
> Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/ git :
> https://git.koha-community.org/ bugs :
> https://bugs.koha-community.org/
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland _______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20210421/023f1149/attachment.htm>

------------------------------

Subject: Digest Footer

_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/


------------------------------

End of Koha-devel Digest, Vol 185, Issue 18
*******************************************