Minimal docker images for Koha
Hi all, I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results. My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha and the Dockerfiles are here: https://github.com/jajm/koha-docker A few things worth mentioning: * I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base. * Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl. * I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies. * It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC. * Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else. * There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch. * Zebra is not installed * Images weigh ~1.15GB uncompressed (koha sources included) If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md -- Julian Maurice BibLibre
Hi Julian, Awesome to see you playing with Docker. I love Docker and I've been thinking about assembling similar Docker images for Koha, but haven't had the time, so hurray for you doing it! I couldn't get your first URL to work as I think that's an authorised user link. Is it safe to say that this is the public link? https://hub.docker.com/r/julianmaurice/koha Did you try the "debian:buster-slim" image? I keep meaning to look to see what makes these Koha images so large. With an interpreted language like Perl and Koha's immense functionality, I think it would be challenging to get a small image size. RedHat's registry.access.redhat.com/ubi8/perl-526 image is 658MB and that's without anything application specific even installed. That being said, I think I see some places where you could improve your Dockerfile. Have you checked out multi-stage builds at https://docs.docker.com/develop/develop-images/multistage-build/? In your final image, you shouldn't need packages like gcc, git, *-dev, make, cpanm, etc. You can install those in a "builder" image and do the Koha build there. Then you can copy your Perl libraries and built Koha into the final image. That should also help you cut down on the number of layers you have. Nice work identifying those system perl dependencies. I keep thinking how we should minimize those. It should be reasonably easy to fix those. Actually, if Koha and its Perl dependencies are localized, you could override those with a mounted volume... which makes me wonder why you couldn't use a more minimal Linux... I've actually been meaning to make some OpenSUSE Koha images which would help test out the concept of a minimal Koha running on something other than Debian as well, although one may as well use Debian, since that's the supported OS for Koha... David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 4:59 AM To: koha-devel <koha-devel@lists.koha-community.org> Subject: [Koha-devel] Minimal docker images for Koha Hi all, I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results. My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha and the Dockerfiles are here: https://github.com/jajm/koha-docker A few things worth mentioning: * I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base. * Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl. * I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies. * It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC. * Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else. * There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch. * Zebra is not installed * Images weigh ~1.15GB uncompressed (koha sources included) If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md -- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
The correct URL is indeed https://hub.docker.com/r/julianmaurice/koha Sorry for that :) I just tried buster-slim. The resulting image is 1.06GB. That's ~100MB less than with buster, not bad. But the debian page on dockerhub also states that "These tags are an experiment [...] and are definitely subject to change" so I prefer to stay on the non-slim variant for now. I also tried multi-stage builds, but I didn't like the fact that it leaves an untagged image after each build. Also I figured we might need the build tools once in the container, to install a missing Perl module for instance. And to be honest, after the "failures" of alpine and perl-slim, I decided it was not worth it to try to build the smallest image possible. But you're right, using multi-stage builds we can further reduce the final image size. And if you really really want the smallest image, I think you should use alpine and recompile perl with the correct flags to avoid the MARC::Charset problem. I estimated that you could have an image of 800MB or less. About mounted volumes, from what I understand volumes are for mutable data, and it's considered bad practice to use them for anything else. I would really like to see a non-Debian based image of Koha. It should be relatively easy to do so, by replacing the apt-get command with the corresponding one for OpenSUSE. Le 18/02/2020 à 01:25, dcook@prosentient.com.au a écrit :
Hi Julian,
Awesome to see you playing with Docker. I love Docker and I've been thinking about assembling similar Docker images for Koha, but haven't had the time, so hurray for you doing it!
I couldn't get your first URL to work as I think that's an authorised user link. Is it safe to say that this is the public link? https://hub.docker.com/r/julianmaurice/koha
Did you try the "debian:buster-slim" image? I keep meaning to look to see what makes these Koha images so large. With an interpreted language like Perl and Koha's immense functionality, I think it would be challenging to get a small image size. RedHat's registry.access.redhat.com/ubi8/perl-526 image is 658MB and that's without anything application specific even installed.
That being said, I think I see some places where you could improve your Dockerfile. Have you checked out multi-stage builds at https://docs.docker.com/develop/develop-images/multistage-build/? In your final image, you shouldn't need packages like gcc, git, *-dev, make, cpanm, etc. You can install those in a "builder" image and do the Koha build there. Then you can copy your Perl libraries and built Koha into the final image. That should also help you cut down on the number of layers you have.
Nice work identifying those system perl dependencies. I keep thinking how we should minimize those. It should be reasonably easy to fix those.
Actually, if Koha and its Perl dependencies are localized, you could override those with a mounted volume... which makes me wonder why you couldn't use a more minimal Linux...
I've actually been meaning to make some OpenSUSE Koha images which would help test out the concept of a minimal Koha running on something other than Debian as well, although one may as well use Debian, since that's the supported OS for Koha...
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 4:59 AM To: koha-devel <koha-devel@lists.koha-community.org> Subject: [Koha-devel] Minimal docker images for Koha
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
That sounds reasonable regarding buster-slim. I was thinking a minimal Koha in terms of something that would never really be entered, but rather re-generated if there were any problems. But I agree about it not being worth trying to get the smallest image possible. I think that's small image size is overly fetishized. (That said, I love my "from scratch" images that only have a single binary in them, which are absolutely tiny hehe.) That's a good point about the volumes. I suppose I was thinking about it from a testing perspective, but that defeats the purpose. I was actually wondering earlier how many packaged dependencies Koha really has. Off the top of my head, I think the main ones are Apache httpd, Zebra/YAZ, and Mysql client libraries. It looks like "unzip" is needed as well. A number of the other dependencies like xmlstarlet, tinymce, libjs-jquery, mysql, etc are just for the Debian packages. (Actually, I think there is a mysql client binary use in one of the unit tests which seems suboptimal.) Oh, and due to your use of that static file middleware, Apache httpd isn't even needed. A person could probably run the Zebra server in a different container, share a mounted volume for the data, and then just have Zebra/yaz client software in the Koha container. Or if you went the Elasticsearch route, you wouldn't need Zebra at all, right? So in theory you could maybe get away just with "unzip" as a dependency that needs installing via a package manager? (Technically, we could use pure Perl unzip modules as well...) David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Julian Maurice <julian.maurice@biblibre.com> Sent: Tuesday, 18 February 2020 5:09 PM To: dcook@prosentient.com.au; 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha The correct URL is indeed https://hub.docker.com/r/julianmaurice/koha Sorry for that :) I just tried buster-slim. The resulting image is 1.06GB. That's ~100MB less than with buster, not bad. But the debian page on dockerhub also states that "These tags are an experiment [...] and are definitely subject to change" so I prefer to stay on the non-slim variant for now. I also tried multi-stage builds, but I didn't like the fact that it leaves an untagged image after each build. Also I figured we might need the build tools once in the container, to install a missing Perl module for instance. And to be honest, after the "failures" of alpine and perl-slim, I decided it was not worth it to try to build the smallest image possible. But you're right, using multi-stage builds we can further reduce the final image size. And if you really really want the smallest image, I think you should use alpine and recompile perl with the correct flags to avoid the MARC::Charset problem. I estimated that you could have an image of 800MB or less. About mounted volumes, from what I understand volumes are for mutable data, and it's considered bad practice to use them for anything else. I would really like to see a non-Debian based image of Koha. It should be relatively easy to do so, by replacing the apt-get command with the corresponding one for OpenSUSE. Le 18/02/2020 à 01:25, dcook@prosentient.com.au a écrit :
Hi Julian,
Awesome to see you playing with Docker. I love Docker and I've been thinking about assembling similar Docker images for Koha, but haven't had the time, so hurray for you doing it!
I couldn't get your first URL to work as I think that's an authorised user link. Is it safe to say that this is the public link? https://hub.docker.com/r/julianmaurice/koha
Did you try the "debian:buster-slim" image? I keep meaning to look to see what makes these Koha images so large. With an interpreted language like Perl and Koha's immense functionality, I think it would be challenging to get a small image size. RedHat's registry.access.redhat.com/ubi8/perl-526 image is 658MB and that's without anything application specific even installed.
That being said, I think I see some places where you could improve your Dockerfile. Have you checked out multi-stage builds at https://docs.docker.com/develop/develop-images/multistage-build/? In your final image, you shouldn't need packages like gcc, git, *-dev, make, cpanm, etc. You can install those in a "builder" image and do the Koha build there. Then you can copy your Perl libraries and built Koha into the final image. That should also help you cut down on the number of layers you have.
Nice work identifying those system perl dependencies. I keep thinking how we should minimize those. It should be reasonably easy to fix those.
Actually, if Koha and its Perl dependencies are localized, you could override those with a mounted volume... which makes me wonder why you couldn't use a more minimal Linux...
I've actually been meaning to make some OpenSUSE Koha images which would help test out the concept of a minimal Koha running on something other than Debian as well, although one may as well use Debian, since that's the supported OS for Koha...
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 4:59 AM To: koha-devel <koha-devel@lists.koha-community.org> Subject: [Koha-devel] Minimal docker images for Koha
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
Le Tue, 18 Feb 2020 17:29:42 +1100, <dcook@prosentient.com.au> a écrit :
I was actually wondering earlier how many packaged dependencies Koha really has. Off the top of my head, I think the main ones are Apache httpd, Zebra/YAZ, and Mysql client libraries. It looks like "unzip" is needed as well. A number of the other dependencies like xmlstarlet, tinymce, libjs-jquery, mysql, etc are just for the Debian packages. (Actually, I think there is a mysql client binary use in one of the unit tests which seems suboptimal.) Oh, and due to your use of that static file middleware, Apache httpd isn't even needed. A person could probably run the Zebra server in a different container, share a mounted volume for the data, and then just have Zebra/yaz client software in the Koha container. Or if you went the Elasticsearch route, you wouldn't need Zebra at all, right? So in theory you could maybe get away just with "unzip" as a dependency that needs installing via a package manager? (Technically, we could use pure Perl unzip modules as well...)
What is unzip needed for ? I didn't think of volumes for Zebra. That's a great idea. I might try that later.
"unzip" is used for C4/ImportExportFramework.pm, tools/picture-upload.pl, and tools/upload-cover-image.pl. Ohhhh ./plugins/plugins-upload.pl uses Archive::Extract! Very interesting. There might be merit to not using a pure Perl implementation for pictures, as they may not be as performant as the "unzip" binary. C4/ImportExportFramework.pm probably could use Archive::Extract though. (Archive::Extract also has an option for using "unzip" as a backend I think, so that could be a togglable thing...) Cool! Let us know how you go! It's something I've thought about but hadn't gotten around to trying. I've been thinking about how we could separate out the Zebra server code from the Koha code and package it separately as well. It could help ease the transition to Elasticsearch if Koha just had Zebra client code. (Now that I think about it, there's no reason the Zebra indexing client code even needs to be on the same server as Koha. It just needs database and Zebra access. The only thing Koha really needs is Zebra query client code and configuration.) David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Julian Maurice <julian.maurice@biblibre.com> Sent: Tuesday, 18 February 2020 6:29 PM To: dcook@prosentient.com.au Cc: 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha Le Tue, 18 Feb 2020 17:29:42 +1100, <dcook@prosentient.com.au> a écrit :
I was actually wondering earlier how many packaged dependencies Koha really has. Off the top of my head, I think the main ones are Apache httpd, Zebra/YAZ, and Mysql client libraries. It looks like "unzip" is needed as well. A number of the other dependencies like xmlstarlet, tinymce, libjs-jquery, mysql, etc are just for the Debian packages. (Actually, I think there is a mysql client binary use in one of the unit tests which seems suboptimal.) Oh, and due to your use of that static file middleware, Apache httpd isn't even needed. A person could probably run the Zebra server in a different container, share a mounted volume for the data, and then just have Zebra/yaz client software in the Koha container. Or if you went the Elasticsearch route, you wouldn't need Zebra at all, right? So in theory you could maybe get away just with "unzip" as a dependency that needs installing via a package manager? (Technically, we could use pure Perl unzip modules as well...)
What is unzip needed for ? I didn't think of volumes for Zebra. That's a great idea. I might try that later.
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available! Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
One other question, how does this handle scripts that Koha specifically does not execute using plack? Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice < julian.maurice@biblibre.com> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ? I did nothing specific, so they are probably broken :) Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032. One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ? I did nothing specific, so they are probably broken :) Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations. Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au> wrote:
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032.
One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417.
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Are you talking about these files
https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ?
I did nothing specific, so they are probably broken :)
Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…) Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Kyle Hall <kyle.m.hall@gmail.com> Sent: Thursday, 20 February 2020 12:04 AM To: David Cook <dcook@prosentient.com.au> Cc: Julian Maurice <julian.maurice@biblibre.com>; koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations. Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032. One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ? I did nothing specific, so they are probably broken :) Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> <mailto:kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> >> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> >> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl> <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> <mailto:Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I am planning to get back to bug 22417 in the days. Le jeu. 20 févr. 2020 à 04:59, <dcook@prosentient.com.au> a écrit :
Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…)
Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project…
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
From: Kyle Hall <kyle.m.hall@gmail.com> Sent: Thursday, 20 February 2020 12:04 AM To: David Cook <dcook@prosentient.com.au> Cc: Julian Maurice <julian.maurice@biblibre.com>; koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations.
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au> wrote:
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032.
One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417.
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ?
I did nothing specific, so they are probably broken :)
Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
in the *next* days ;) Le jeu. 20 févr. 2020 à 08:58, Jonathan Druart <jonathan.druart@bugs.koha-community.org> a écrit :
I am planning to get back to bug 22417 in the days.
Le jeu. 20 févr. 2020 à 04:59, <dcook@prosentient.com.au> a écrit :
Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…)
Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project…
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
From: Kyle Hall <kyle.m.hall@gmail.com> Sent: Thursday, 20 February 2020 12:04 AM To: David Cook <dcook@prosentient.com.au> Cc: Julian Maurice <julian.maurice@biblibre.com>; koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations.
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au> wrote:
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032.
One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417.
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ?
I did nothing specific, so they are probably broken :)
Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Actually, now that I think about it, I assume an Apache-less Koha would also lose support for web-facing OPAC plugins (like the EBSCO EDS plugin)? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of dcook@prosentient.com.au Sent: Thursday, 20 February 2020 3:00 PM To: 'Kyle Hall' <kyle.m.hall@gmail.com> Cc: 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…) Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Sent: Thursday, 20 February 2020 12:04 AM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> >; koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations. Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032. One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ? I did nothing specific, so they are probably broken :) Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> <mailto:kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> >> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> <mailto:julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> >> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl> <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> <mailto:Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Plugin code is loaded into Plack processes. Does the EDS plugin contain cgi scripts that wouldn't run under plack? Getting all scripts to run under plack is definitely a goal we should have, but a short term kludge would be to run two containers, one plack and one cgi, and to direct traffic from a proxy to the correct container based on the url. Not elegant, but functional. --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Thu, Feb 20, 2020 at 7:30 PM <dcook@prosentient.com.au> wrote:
Actually, now that I think about it, I assume an Apache-less Koha would also lose support for web-facing OPAC plugins (like the EBSCO EDS plugin)?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Koha-devel <koha-devel-bounces@lists.koha-community.org> *On Behalf Of *dcook@prosentient.com.au *Sent:* Thursday, 20 February 2020 3:00 PM *To:* 'Kyle Hall' <kyle.m.hall@gmail.com> *Cc:* 'koha-devel' <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…)
Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project…
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Kyle Hall <kyle.m.hall@gmail.com> *Sent:* Thursday, 20 February 2020 12:04 AM *To:* David Cook <dcook@prosentient.com.au> *Cc:* Julian Maurice <julian.maurice@biblibre.com>; koha-devel < koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations.
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au> wrote:
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032.
One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417.
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Are you talking about these files
https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ?
I did nothing specific, so they are probably broken :)
Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
It could just be that I misunderstand how the plugins work with Plack, but when I use Koha via CGI, this is the path that is being used: /plugin/Koha/Plugin/EDS/opac/eds-search.pl?q=Search?query-1=AND,:{test}&default=1. I don’t think that would work using https://github.com/jajm/koha-docker/blob/master/master/app.psgi? I think just adding another mount would do the trick though? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Kyle Hall <kyle.m.hall@gmail.com> Sent: Friday, 21 February 2020 11:27 PM To: David Cook <dcook@prosentient.com.au> Cc: Julian Maurice <julian.maurice@biblibre.com>; koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha Plugin code is loaded into Plack processes. Does the EDS plugin contain cgi scripts that wouldn't run under plack? Getting all scripts to run under plack is definitely a goal we should have, but a short term kludge would be to run two containers, one plack and one cgi, and to direct traffic from a proxy to the correct container based on the url. Not elegant, but functional. --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Thu, Feb 20, 2020 at 7:30 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Actually, now that I think about it, I assume an Apache-less Koha would also lose support for web-facing OPAC plugins (like the EBSCO EDS plugin)? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> Sent: Thursday, 20 February 2020 3:00 PM To: 'Kyle Hall' <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Cc: 'koha-devel' <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…) Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Sent: Thursday, 20 February 2020 12:04 AM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> >; koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations. Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032. One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Minimal docker images for Koha Are you talking about these files https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ? I did nothing specific, so they are probably broken :) Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> <mailto:kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com> >> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> <mailto:julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> >> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl> <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> <mailto:Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I could be mistaken as well. I was under the impression that Plack loaded the plugin code into itself when the plugin was first run. I'm basing this on the fact that it used to be that updating a plugin did not reflect the updated info in the plugins table in the admin section for plugins. That particular issue has been resolved. I suppose the easiest way to test would be to load a plugin into a plack enabled Koha instance, make a minor change to the plugin's perl code ( like a warn statement or something ) and see if the change shows itself without restarting plack. If it doesn't, then plugins are probably running under plack. --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Sun, Feb 23, 2020 at 6:01 PM <dcook@prosentient.com.au> wrote:
It could just be that I misunderstand how the plugins work with Plack, but when I use Koha via CGI, this is the path that is being used: /plugin/Koha/Plugin/EDS/opac/ eds-search.pl?q=Search?query-1=AND,:{test}&default=1 <http://eds-search.pl?q=Search?query-1=AND,:%7Btest%7D&default=1>. I don’t think that would work using https://github.com/jajm/koha-docker/blob/master/master/app.psgi? I think just adding another mount would do the trick though?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Kyle Hall <kyle.m.hall@gmail.com> *Sent:* Friday, 21 February 2020 11:27 PM *To:* David Cook <dcook@prosentient.com.au> *Cc:* Julian Maurice <julian.maurice@biblibre.com>; koha-devel < koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
Plugin code is loaded into Plack processes. Does the EDS plugin contain cgi scripts that wouldn't run under plack?
Getting all scripts to run under plack is definitely a goal we should have, but a short term kludge would be to run two containers, one plack and one cgi, and to direct traffic from a proxy to the correct container based on the url. Not elegant, but functional.
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Thu, Feb 20, 2020 at 7:30 PM <dcook@prosentient.com.au> wrote:
Actually, now that I think about it, I assume an Apache-less Koha would also lose support for web-facing OPAC plugins (like the EBSCO EDS plugin)?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Koha-devel <koha-devel-bounces@lists.koha-community.org> *On Behalf Of *dcook@prosentient.com.au *Sent:* Thursday, 20 February 2020 3:00 PM *To:* 'Kyle Hall' <kyle.m.hall@gmail.com> *Cc:* 'koha-devel' <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
Exactly! I would love to see a task queue based record importer. (I keep thinking about a Koha plugin to provide a web ui for scheduling OAI-PMH harvesting, and then third-party applications that can consume from the task queue, stage the records… probably via the REST API… and then enqueue tasks to a high performance record importer worker…)
Are you going to the Hackfest in Marseille? (I wanted to go this year but life is just too hectic this year, although planning to go to Kohacon in October.) A task queue could be a great Hackfest project…
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Kyle Hall <kyle.m.hall@gmail.com> *Sent:* Thursday, 20 February 2020 12:04 AM *To:* David Cook <dcook@prosentient.com.au> *Cc:* Julian Maurice <julian.maurice@biblibre.com>; koha-devel < koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
Agreed! I think a task queue would be a huge benefit to Koha, we have so many long running batch operations.
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 7:21 PM <dcook@prosentient.com.au> wrote:
Funny I was just looking at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15032.
One of these days we should push forward more on queues: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417.
David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Direct: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Julian Maurice Sent: Tuesday, 18 February 2020 11:11 PM To: Kyle Hall <kyle.m.hall@gmail.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha
Are you talking about these files
https://github.com/Koha-Community/Koha/blob/master/debian/templates/apache-s... ?
I did nothing specific, so they are probably broken :)
Le 18/02/2020 à 12:45, Kyle Hall a écrit :
One other question, how does this handle scripts that Koha specifically does not execute using plack?
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Tue, Feb 18, 2020 at 6:43 AM Kyle Hall <kyle.m.hall@gmail.com <mailto:kyle.m.hall@gmail.com>> wrote:
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org>
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I just tried minideb, the resulting image weigh 1.1GB, right between debian:buster-slim and debian:buster I did not found a free plan for quay.io (only a 30-day free trial). Does it have one ? Le 18/02/2020 à 12:43, Kyle Hall a écrit :
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
They don’t make it easy to understand, all you need to do is create your account and when you go to add a repository you can choose a Public GitHub repository and you will not need any kind of paid account. On Tue, Feb 18, 2020 at 7:46 AM Julian Maurice <julian.maurice@biblibre.com> wrote:
I just tried minideb, the resulting image weigh 1.1GB, right between debian:buster-slim and debian:buster
I did not found a free plan for quay.io (only a 30-day free trial). Does it have one ?
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
--- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or
Le 18/02/2020 à 12:43, Kyle Hall a écrit : perl
slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This
is
because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as
possible.
Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two
ports,
one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by
reading
https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
-- --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
Mmm that’s a good point. The smaller attack surface is something I harp on about a lot when it comes to making minimal images. That’s actually led me down some very fun rabbit holes about operating systems and Linux in particular. For instance, here’s the Dockerfile for ubuntu:latest. It’s actually quite minimal with the majority of the work being done by “ADD ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /”, which can be found at https://partner-images.canonical.com/core/bionic/current/ubuntu-bionic-core-.... When you open that up, it’s just a small Ubuntu root file system. Now what does that get us? First I’ll backtrack. When the host boots, GRUB 2 finds the desired Linux kernel, loads the kernel and the initramfs, and then transfers control to the kernel, which runs the initramfs’s /init script (which typically invokes systemd these days). That /init script finds the “real” root file system, mounts it, and then executes systemd on the real root file system, which acts as the init system and becomes our old faithful PID 1. Obviously that process doesn’t correspond to a container’s lifecycle. When a container is started, the kernel is already running and the root file system is already mounted. There’s already kernel mode and user mode code running to manage the computer. Docker gives us isolation using Linux kernel features like cgroups and namespaces, and takes care of special file system cases like /dev, /proc/, and /sys for us. So a person doesn’t need a whole OS file system just to run a single program in Docker. However, in our case, it gets complicated quickly, since Koha needs MySQL client libraries, Zebra client libraries, and whatever other libraries and files our Perl modules need (DateTime leverages OS-level datetime files I think, there’s libxml, probably GD, etc.). If we were really thorough, we probably could get Koha running in a very minimal container, but it would take some work. It could be fun though. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Kyle Hall Sent: Tuesday, 18 February 2020 10:43 PM To: Julian Maurice <julian.maurice@biblibre.com> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available! Kyle --- http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com> > wrote: Hi all, I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results. My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha and the Dockerfiles are here: https://github.com/jajm/koha-docker A few things worth mentioning: * I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base. * Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl> . * I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies. * It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC. * Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else. * There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch. * Zebra is not installed * Images weigh ~1.15GB uncompressed (koha sources included) If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md -- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I gave another try at multi-stage builds. It turns out you can tag the intermediate image by building them first with `docker build --target <stage> ...` so my problem with multi-stage builds is gone :) The result is an image of ~875MB. I pushed it on https://hub.docker.com/r/julianmaurice/koha with the tag master-slim Le 19/02/2020 à 01:17, dcook@prosentient.com.au a écrit :
Mmm that’s a good point. The smaller attack surface is something I harp on about a lot when it comes to making minimal images. That’s actually led me down some very fun rabbit holes about operating systems and Linux in particular.
For instance, here’s the Dockerfile for ubuntu:latest. It’s actually quite minimal with the majority of the work being done by “ADD ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /”, which can be found at https://partner-images.canonical.com/core/bionic/current/ubuntu-bionic-core-.... When you open that up, it’s just a small Ubuntu root file system. Now what does that get us? First I’ll backtrack.
When the host boots, GRUB 2 finds the desired Linux kernel, loads the kernel and the initramfs, and then transfers control to the kernel, which runs the initramfs’s /init script (which typically invokes systemd these days). That /init script finds the “real” root file system, mounts it, and then executes systemd on the real root file system, which acts as the init system and becomes our old faithful PID 1.
Obviously that process doesn’t correspond to a container’s lifecycle. When a container is started, the kernel is already running and the root file system is already mounted. There’s already kernel mode and user mode code running to manage the computer. Docker gives us isolation using Linux kernel features like cgroups and namespaces, and takes care of special file system cases like /dev, /proc/, and /sys for us.
So a person doesn’t need a whole OS file system just to run a single program in Docker.
However, in our case, it gets complicated quickly, since Koha needs MySQL client libraries, Zebra client libraries, and whatever other libraries and files our Perl modules need (DateTime leverages OS-level datetime files I think, there’s libxml, probably GD, etc.). If we were really thorough, we probably could get Koha running in a very minimal container, but it would take some work. It could be fun though.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:*Koha-devel <koha-devel-bounces@lists.koha-community.org> *On Behalf Of *Kyle Hall *Sent:* Tuesday, 18 February 2020 10:43 PM *To:* Julian Maurice <julian.maurice@biblibre.com> *Cc:* koha-devel <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
Cool! Nice one, Julian! David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Julian Maurice <julian.maurice@biblibre.com> Sent: Thursday, 20 February 2020 7:27 PM To: dcook@prosentient.com.au; 'Kyle Hall' <kyle.m.hall@gmail.com> Cc: 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Minimal docker images for Koha I gave another try at multi-stage builds. It turns out you can tag the intermediate image by building them first with `docker build --target <stage> ...` so my problem with multi-stage builds is gone :) The result is an image of ~875MB. I pushed it on https://hub.docker.com/r/julianmaurice/koha with the tag master-slim Le 19/02/2020 à 01:17, dcook@prosentient.com.au a écrit :
Mmm that’s a good point. The smaller attack surface is something I harp on about a lot when it comes to making minimal images. That’s actually led me down some very fun rabbit holes about operating systems and Linux in particular.
For instance, here’s the Dockerfile for ubuntu:latest. It’s actually quite minimal with the majority of the work being done by “ADD ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /”, which can be found at https://partner-images.canonical.com/core/bionic/current/ubuntu-bionic-core-.... When you open that up, it’s just a small Ubuntu root file system. Now what does that get us? First I’ll backtrack.
When the host boots, GRUB 2 finds the desired Linux kernel, loads the kernel and the initramfs, and then transfers control to the kernel, which runs the initramfs’s /init script (which typically invokes systemd these days). That /init script finds the “real” root file system, mounts it, and then executes systemd on the real root file system, which acts as the init system and becomes our old faithful PID 1.
Obviously that process doesn’t correspond to a container’s lifecycle. When a container is started, the kernel is already running and the root file system is already mounted. There’s already kernel mode and user mode code running to manage the computer. Docker gives us isolation using Linux kernel features like cgroups and namespaces, and takes care of special file system cases like /dev, /proc/, and /sys for us.
So a person doesn’t need a whole OS file system just to run a single program in Docker.
However, in our case, it gets complicated quickly, since Koha needs MySQL client libraries, Zebra client libraries, and whatever other libraries and files our Perl modules need (DateTime leverages OS-level datetime files I think, there’s libxml, probably GD, etc.). If we were really thorough, we probably could get Koha running in a very minimal container, but it would take some work. It could be fun though.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:*Koha-devel <koha-devel-bounces@lists.koha-community.org> *On Behalf Of *Kyle Hall *Sent:* Tuesday, 18 February 2020 10:43 PM *To:* Julian Maurice <julian.maurice@biblibre.com> *Cc:* koha-devel <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Minimal docker images for Koha
This is fantastic Julian! The only thing I can contribute that hasn't already been said by you or David is to suggest taking a look at MiniDeb as a base image ( https://github.com/bitnami/minideb ). I would also suggest using quay.io <http://quay.io> to build and host your Docker images, as it has built in security scanning. I prefer minimal install images not for size reduction ( though it is nice ), but for the smaller attack surface they provide. Fewer things installed means fewer exploits available!
Kyle
---
http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org )
On Mon, Feb 17, 2020 at 12:59 PM Julian Maurice <julian.maurice@biblibre.com <mailto:julian.maurice@biblibre.com>> wrote:
Hi all,
I've been playing with docker lately, and I tried to build a minimal docker image for Koha. Here are the results.
My goals were: * Install only required "things" to get Koha up and running, and nothing else (no testing or dev tools), * No external dependencies except CPAN * Follow Docker best practices as much as possible
The resulting images are here: https://hub.docker.com/repository/docker/julianmaurice/koha
and the Dockerfiles are here: https://github.com/jajm/koha-docker
A few things worth mentioning:
* I tried to build the smallest image possible by using alpine or perl slim images at first but it was not that great, because the perl version shipped with those images is missing some libs, which cause MARC::Charset to build a database of several hundreds MBs (which is only 5MBs with a standard perl version). So I chose a more standard image (debian:buster) as base.
* Koha doesn't work well when running with a perl version different than the system perl installed in /usr/bin/perl. For example, the updatedatabase doesn't work when called from the web installer. This is because Perl scripts are called directly as executable files, and shebangs contain '/usr/bin/perl'. Same problem from misc/translator/translate which calls tmpl_process3.pl <http://tmpl_process3.pl>.
* I tried to make the Koha installation as self-contained as possible. Almost everything is installed as a non-root user in /home/koha, including Perl dependencies.
* It doesn't need a reverse proxy such as apache or nginx. The necessary URL rewriting is handled in PSGI file. The container expose two ports, one for intranet, the other one for OPAC.
* Each Perl dependency is installed in its latest version, so expect things to break. I can only confirm that the webinstaller, basic cataloguing and search/indexation work. I did not test anything else.
* There are docker-compose.yml files in the github repository to get Koha running quickly with mariadb, memcached and elasticsearch.
* Zebra is not installed
* Images weigh ~1.15GB uncompressed (koha sources included)
If you made it this far, thanks for reading :) And if you want to use these docker images, you should start by reading https://github.com/jajm/koha-docker/blob/master/README.md
-- Julian Maurice BibLibre _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Julian Maurice BibLibre
participants (4)
-
dcook＠prosentient.com.au -
Jonathan Druart -
Julian Maurice -
Kyle Hall