Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I've put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The "build.sh" script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into "input" (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your "output" directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that's because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the "build dependencies", although I think we could tweak the debian/control file so that we don't have to install everything just to build the packages. Also I couldn't get the tests to complete successfully on the first go so I've turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I'd share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595
David, try this https://hub.docker.com/r/koha/koha-dpkg/ I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results. El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au> escribió: Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages. Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I’d share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595
Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository? I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient. I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: Building Koha Debian packages in Docker David, try this https://hub.docker.com/r/koha/koha-dpkg/ I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results. El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió: Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages. Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I’d share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595
More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged) Chris On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?
I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.
I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: Building Koha Debian packages in Docker
David, try this
https://hub.docker.com/r/koha/koha-dpkg/
I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.
El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió:
Hi folks,
I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.
So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker
The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.
Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.
The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.
Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.
Anyway, just thought I’d share that with everyone.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things. It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Monday, 26 November 2018 6:51 PM To: koha-devel@lists.koha-community.org Cc: 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Building Koha Debian packages in Docker More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged) Chris On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository? I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient. I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: Building Koha Debian packages in Docker David, try this https://hub.docker.com/r/koha/koha-dpkg/ I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results. El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió: Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages. Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I’d share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
If you like, I note it's been working fine for over 8 years now. And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it Chris On 27 November 2018 1:32:33 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.
It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Monday, 26 November 2018 6:51 PM To: koha-devel@lists.koha-community.org Cc: 'koha-devel' <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Building Koha Debian packages in Docker
More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)
Chris
On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote:
Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?
I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.
I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: Building Koha Debian packages in Docker
David, try this
https://hub.docker.com/r/koha/koha-dpkg/
I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.
El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió:
Hi folks,
I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.
So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker
The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.
Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.
The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.
Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.
Anyway, just thought I’d share that with everyone.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough. I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details. I’m not really advocating for change. Really just asking questions about why things are done the way they’re done. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Chris Cormack [mailto:chrisc@catalyst.net.nz] Sent: Tuesday, 27 November 2018 11:35 AM To: David Cook <dcook@prosentient.com.au>; koha-devel@lists.koha-community.org Subject: RE: [Koha-devel] Building Koha Debian packages in Docker If you like, I note it's been working fine for over 8 years now. And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it Chris On 27 November 2018 1:32:33 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things. It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Monday, 26 November 2018 6:51 PM To: koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> Cc: 'koha-devel' <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Building Koha Debian packages in Docker More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged) Chris On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository? I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient. I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: Building Koha Debian packages in Docker David, try this https://hub.docker.com/r/koha/koha-dpkg/ I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results. El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió: Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages. Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I’d share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages. El lun., 26 nov. 2018 a las 21:55, David Cook (<dcook@prosentient.com.au>) escribió:
There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.
I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.
I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Chris Cormack [mailto:chrisc@catalyst.net.nz] *Sent:* Tuesday, 27 November 2018 11:35 AM *To:* David Cook <dcook@prosentient.com.au>; koha-devel@lists.koha-community.org *Subject:* RE: [Koha-devel] Building Koha Debian packages in Docker
If you like, I note it's been working fine for over 8 years now. And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it
Chris
On 27 November 2018 1:32:33 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.
It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* koha-devel-bounces@lists.koha-community.org [ mailto:koha-devel-bounces@lists.koha-community.org <koha-devel-bounces@lists.koha-community.org>] *On Behalf Of *Chris Cormack *Sent:* Monday, 26 November 2018 6:51 PM *To:* koha-devel@lists.koha-community.org *Cc:* 'koha-devel' <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Building Koha Debian packages in Docker
More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)
Chris
On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?
I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.
I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Tomas Cohen Arazi [mailto:tomascohen@gmail.com <tomascohen@gmail.com>] *Sent:* Thursday, 22 November 2018 8:38 PM *To:* David Cook <dcook@prosentient.com.au> *Cc:* koha-devel <koha-devel@lists.koha-community.org> *Subject:* Re: Building Koha Debian packages in Docker
David, try this
https://hub.docker.com/r/koha/koha-dpkg/
I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.
El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au> escribió:
Hi folks,
I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.
So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker
The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.
Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.
The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.
Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.
Anyway, just thought I’d share that with everyone.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
-- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F
If you shared some of your ideas, Tomas, perhaps I could do some work on that one? With https://gitlab.com/minusdavid/koha-deb-builder-docker, I’m not using a chroot. I just use the debian:jessie parent image, install build tools, take the upstream tarball (generated via git archive), use dch to update the changelog, currently use apt-get -y build-dep koha to install the build dependencies, then run “debuild”. The slowest bit of the process is installing those build dependencies. (Full disclosure: I’m using DEB_BUILD_OPTIONS=nocheck as I recall one of the tests causing a fatal error, and I put off analysing that until I had the build process nailed down in the docker. That’ll need revisiting.) David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Tuesday, 27 November 2018 11:57 AM To: David Cook <dcook@prosentient.com.au> Cc: Chris Cormack <chrisc@catalyst.net.nz>; koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Building Koha Debian packages in Docker If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages. El lun., 26 nov. 2018 a las 21:55, David Cook (<dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> >) escribió: There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough. I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details. I’m not really advocating for change. Really just asking questions about why things are done the way they’re done. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Chris Cormack [mailto:chrisc@catalyst.net.nz <mailto:chrisc@catalyst.net.nz> ] Sent: Tuesday, 27 November 2018 11:35 AM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> >; koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> Subject: RE: [Koha-devel] Building Koha Debian packages in Docker If you like, I note it's been working fine for over 8 years now. And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it Chris On 27 November 2018 1:32:33 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things. It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Chris Cormack Sent: Monday, 26 November 2018 6:51 PM To: koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> Cc: 'koha-devel' <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: [Koha-devel] Building Koha Debian packages in Docker More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged) Chris On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository? I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient. I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: Tomas Cohen Arazi [mailto:tomascohen@gmail.com] Sent: Thursday, 22 November 2018 8:38 PM To: David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > Cc: koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Re: Building Koha Debian packages in Docker David, try this https://hub.docker.com/r/koha/koha-dpkg/ I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results. El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > escribió: Hi folks, I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else. So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer. Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy. The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages. Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool. Anyway, just thought I’d share that with everyone. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ -- Tomás Cohen Arazi Theke Solutions (http://theke.io <http://theke.io/> ) ✆ +54 9351 3513384 GPG: B2F3C15F
It's all here: https://gitlab.com/tomascohen/koha-dpkg-docker I thought it was documented on the Docker hub. El lun., 26 nov. 2018 a las 22:10, David Cook (<dcook@prosentient.com.au>) escribió:
If you shared some of your ideas, Tomas, perhaps I could do some work on that one?
With https://gitlab.com/minusdavid/koha-deb-builder-docker, I’m not using a chroot. I just use the debian:jessie parent image, install build tools, take the upstream tarball (generated via git archive), use dch to update the changelog, currently use apt-get -y build-dep koha to install the build dependencies, then run “debuild”. The slowest bit of the process is installing those build dependencies.
(Full disclosure: I’m using DEB_BUILD_OPTIONS=nocheck as I recall one of the tests causing a fatal error, and I put off analysing that until I had the build process nailed down in the docker. That’ll need revisiting.)
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Tomas Cohen Arazi [mailto:tomascohen@gmail.com] *Sent:* Tuesday, 27 November 2018 11:57 AM *To:* David Cook <dcook@prosentient.com.au> *Cc:* Chris Cormack <chrisc@catalyst.net.nz>; koha-devel < koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Building Koha Debian packages in Docker
If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages.
El lun., 26 nov. 2018 a las 21:55, David Cook (<dcook@prosentient.com.au>) escribió:
There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.
I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.
I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Chris Cormack [mailto:chrisc@catalyst.net.nz] *Sent:* Tuesday, 27 November 2018 11:35 AM *To:* David Cook <dcook@prosentient.com.au>; koha-devel@lists.koha-community.org *Subject:* RE: [Koha-devel] Building Koha Debian packages in Docker
If you like, I note it's been working fine for over 8 years now. And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it
Chris
On 27 November 2018 1:32:33 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.
It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* koha-devel-bounces@lists.koha-community.org [ mailto:koha-devel-bounces@lists.koha-community.org <koha-devel-bounces@lists.koha-community.org>] *On Behalf Of *Chris Cormack *Sent:* Monday, 26 November 2018 6:51 PM *To:* koha-devel@lists.koha-community.org *Cc:* 'koha-devel' <koha-devel@lists.koha-community.org> *Subject:* Re: [Koha-devel] Building Koha Debian packages in Docker
More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)
Chris
On 26 November 2018 8:19:36 PM NZDT, David Cook <dcook@prosentient.com.au> wrote:
Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?
I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.
I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Tomas Cohen Arazi [mailto:tomascohen@gmail.com <tomascohen@gmail.com>] *Sent:* Thursday, 22 November 2018 8:38 PM *To:* David Cook <dcook@prosentient.com.au> *Cc:* koha-devel <koha-devel@lists.koha-community.org> *Subject:* Re: Building Koha Debian packages in Docker
David, try this
https://hub.docker.com/r/koha/koha-dpkg/
I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.
El 22 nov. 2018 04:59, "David Cook" <dcook@prosentient.com.au> escribió:
Hi folks,
I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.
So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker
The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.
Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.
The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.
Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.
Anyway, just thought I’d share that with everyone.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
--
Tomás Cohen Arazi
Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F
-- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F
participants (3)
-
Chris Cormack -
David Cook -
Tomas Cohen Arazi