[Koha-devel] Adopting CPAN and Carton
dcook at prosentient.com.au
dcook at prosentient.com.au
Wed Jun 10 02:32:15 CEST 2020
I agree with Tomas.
Julian, you're right about bundling deps in a Debian package not being very typical in Debian dogma, but there is some precedent (see https://wiki.debian.org/Javascript/Nodejs/Npm2Deb/Embedding and https://wiki.debian.org/Javascript/GroupSourcesTutorial). I think it's worthwhile to remember that we're not planning on upstreaming Koha into the official Debian repositories either (at least I haven't heard that as a goal for 10 years now). We (currently) use Debian packages as a deployment mechanism. We aren't bound by official Debian policy. (For an interesting look at Debian policy and modern apps, you could look at this blog about Debian's docker.io vs Docker's docker-ce packages. The Debian version separates out dependencies into separate packages while Docker's version bundles the dependencies in the package. Personally, I tend to use docker-ce myself, since it is always much more up-to-date than docker.io.)
Julian, you can use cpanm to install locally, but keep in mind that cpanm and carton were created by the same developer Tatsuhiko Miyagawa (who also created Plack). Based on the documentation, the cpanfile.snapshot includes not just the top level dependencies, but also all their root dependencies and versions. Without that cpanfile.snapshot file, two people running "cpanm --installdeps" could have very different versions of the dependencies below their top level dependencies. You'd want carton's cpanfile.snapshot to create consistency across all deployments. (Carton's bundling and caching abilities could also speed up builds so that local tarballs could be used instead of having to re-fetch everything over a network.)
I think using CPAN and Carton would be most powerful if we used it across all deployment methods, but I can understand a desire to just use it as an alternative. I think the only problem with it being an alternative is that, without community support, it's unlikely to succeed.
At the moment, I'm moving 99% of my Koha installations to Debian/Ubuntu, so I won't really have need of an unsupported alternative installation method for Koha.
I suppose that a first step could be for Tomas and me to do some proof-of-concepts before continuing the conversation. (As Jonathan said to me the other day, "more code, less talk".)
If people aren't in favour of the proof-of-concepts, we could leave some unofficial unsupported documentation, but then just let inertia continue to guide us as we are. Or if more people (but not all) accept the idea, maybe we could have a small group that support the alternative method.
I was just thinking using CPAN with Carton would be more modern, which is a bit funny since apt-get is from 1998 and CPAN is from 1995 (although I think cpanm was created in 2010 and carton in 2013).
Anyway, I'm happy to do work on this, but I'm looking for other interested parties with whom to collaborate.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
-----Original Message-----
From: Julian Maurice <julian.maurice at biblibre.com>
Sent: Wednesday, 10 June 2020 12:11 AM
To: Tomas Cohen Arazi <tomascohen at gmail.com>
Cc: David Cook <dcook at prosentient.com.au>; koha-devel <koha-devel at lists.koha-community.org>
Subject: Re: [Koha-devel] Adopting CPAN and Carton
Le 09/06/2020 à 15:47, Tomas Cohen Arazi a écrit :
> I think we should use carton and/or local::lib, and bundle all the
> dependencies on build time.
> Building a Docker image? Bundle all deps.
> Building a Debian package? Bundle all deps (we already need to build
> things for specific perl versions) Building a Snap/Flatpack? Bundlfe
> all deps
Bundle all deps in a Debian package ? It does not seem very "Debian-friendly" :-/
>
> cpanm means installing globally and is possible, but why not just...
> bundle them.
cpanm can install locally
(https://metacpan.org/pod/distribution/App-cpanminus/bin/cpanm#-L,-local-lib-contained)
>
>
> El mar., 9 jun. 2020 a las 3:47, Julian Maurice
> (<julian.maurice at biblibre.com <mailto:julian.maurice at biblibre.com>>)
> escribió:
>
> I'm all for installing Perl modules directly from CPAN. I'm less sure
> about Carton. Is it really needed ? I believe that `cpanm --installdeps
> .` does the same thing.
>
> Anyway, I think the first step would be to write documentation on
> how to
> use this installation method. Even if not adopted, it can't hurt anyone
> to have an alternate method documented.
> Can you write it ?
>
> Le 09/06/2020 à 01:13, dcook at prosentient.com.au
> <mailto:dcook at prosentient.com.au> a écrit :
> > It would mean that the Koha community would support "CPAN and
> Carton" as an installation method or use it as *the* installation
> method for Koha.
> >
> > David Cook
> > Systems Librarian
> > Prosentient Systems
> > 72/330 Wattle St
> > Ultimo, NSW 2007
> > Australia
> >
> > Office: 02 9212 0899
> > Online: 02 8005 0595
> >
> > -----Original Message-----
> > From: Koha-devel <koha-devel-bounces at lists.koha-community.org
> <mailto:koha-devel-bounces at lists.koha-community.org>> On Behalf Of
> Julian Maurice
> > Sent: Friday, 5 June 2020 5:12 PM
> > To: koha-devel at lists.koha-community.org
> <mailto:koha-devel at lists.koha-community.org>
> > Subject: Re: [Koha-devel] Adopting CPAN and Carton
> >
> > Hi,
> >
> > As you said it is already feasible and very easy to do. So what
> would it mean to "adopt CPAN and Carton" ?
> >
> > Le 05/06/2020 à 08:10, dcook at prosentient.com.au
> <mailto:dcook at prosentient.com.au> a écrit :
> >> Now that I think about it, embedding the Perl modules in the Debian
> >> package could be problematic due to dependencies on C libraries.
> >> (Currently building Text::Bidi on an OpenSUSE system and it's taking
> >> forever.)
> >>
> >> I actually found it pretty challenging to read replies
> interweaved in
> >> emails, so I might not respond to all that.
> >>
> >> But yeah I could see the most likely option being keeping the Debian
> >> packages the way they are and just having an alternative Carton
> style.
> >> All that is needed for it really is changing the PERL5LIB
> >> environmental variable, so it would be easy enough to do.
> >>
> >> David Cook
> >> Systems Librarian
> >> Prosentient Systems
> >> 72/330 Wattle St
> >> Ultimo, NSW 2007
> >> Australia
> >>
> >> Office: 02 9212 0899
> >> Online: 02 8005 0595
> >>
> >> -----Original Message-----
> >> From: Victor Grousset/tuxayo <victor at tuxayo.net
> <mailto:victor at tuxayo.net>>
> >> Sent: Friday, 5 June 2020 3:15 PM
> >> To: dcook at prosentient.com.au <mailto:dcook at prosentient.com.au>;
> 'Koha Devel'
> >> <koha-devel at lists.koha-community.org
> <mailto:koha-devel at lists.koha-community.org>>
> >> Subject: Re: Adopting CPAN and Carton
> >>
> >> Hi o/
> >>
> >> On 20-06-05 02:18, dcook at prosentient.com.au
> <mailto:dcook at prosentient.com.au> wrote:
> >>> What are people’s thoughts on CPAN and Carton (the Perl version of
> >>> Bundler, Composer, NPM, etc)?
> >>
> >> I hope that something like that would be on our reach. And that it
> >> will make the majority of the work done to support other Linux
> >> distributions, which would be amazing!
> >>
> >>> I like the idea as a way of more easily managing Perl dependencies
> >>> regardless of Linux distribution and version; in other words, we
> >>> could provide the same dependencies regardless of being on Debian
> >>> Jessie/Debian Stretch/Debian Buster or Ubuntu or Fedora or RHEL or
> >> OpenSUSE.
> >>>
> >>>
> >>>
> >>> I suppose the difficulty is how we do a community-friendly
> deployment?
> >>> I see two main options:
> >>>
> >>> 1. Have the Debian package run “carton install”
> automatically during
> >>> install/upgrade
> >>> 1. This could be error prone and difficult to support…
> 2. Have
> >>> the Koha Debian Package Manager run “carton install” when
> >>> building the package, and embed the Perl modules in the
> Koha package
> >>> 1. It’s unknown how much this would increase the size of
> the Koha
> >>> package
> >>
> >> Interesting, I hope such an unified way will be possible.
> >> If not, is the following a valid option?
> >> The current way of managing the perl dependencies for Debian
> doesn't change.
> >> And a carton way of managing the perl dependencies will coexist.
> (does
> >> it mean just maintaining the cpanfile? I guess I'm very very wrong
> >> here) And packages for the other distros will use carton.
> >>
> >> It would depend of which is more work:
> >> Integrating and maintaining the use of carton in the Debian packages
> >> Or Maintaining the "full" Debian packages in parallel.
> >>
> >> And also how much is valued the fact that for Debian, the perl deps
> >> comes from the repo? (we could keep the versions in sync with
> Debian,
> >> see [1]) Subparts of this:
> >> - value of Debian patches
> >> - anything else???
> >>
> >>
> >>> I think retaining Debian packages is important, because we have a
> >>> number of non-Perl dependencies, and the Debian packages provide a
> >>> lot of very useful automation.
> >>
> >> Thank for addressing the question of "If we can use Carton, why have
> >> packages at all?"
> >> I trust you and the others for that ^^" I have no idea of whether or
> >> not packages are the best option compared to others. Does anyone
> have
> >> feedback about the handling of applications with a lot of deps in
> >> other languages than Perl? (that have something similar to Carton)
> >>
> >>
> >>
> >>> Personally, I’m moving away from packaging Perl dependencies as
> >>> Debian packages for non-Koha projects, because it makes porting the
> >>> projects to a newer version of the same OS much more difficult.
> >>
> >>> I know using Debian’s Perl packages have a lot of advantages, as
> >>> they’re tested, reviewed, and patched, but I think it might be time
> >>> to consider a change.
> >>
> >> [1]The cpanfile could still evolve in sync with the Debian package
> >> versions of the perl libs.
> >> That doesn't address the point "patched" => are there a lot of
> patched
> >> perl packages in Debian?
> >> i.e. enough so that even getting the same versions (as the one in
> >> Debian stable or oldstable) from CPAN will lead to issues
> compared to
> >> getting them from the Debian packages?
> >>
> >>
> >>
> >> Unrelated idea: if we end up not using the Debian perl packages and
> >> not syncing the versions pulled from CPAN with those in Debian, here
> >> is how we could manage perl deps.
> >> We could have a script that would edit the cpanfile to bump the
> major
> >> version of one package. Or set it to the latest version.
> >> And run the tests. And if the tests are alright, keep the new
> package
> >> version and try updating the next package.
> >> This way (hopefully) most of the update work of perl deps would
> be done.
> >> Edge cases will be dealt with manually.
> >>
> >> Even if we don't directly need it, it could be a nice tool to detect
> >> regressions in our upstream deps way before they fall on us!
> >>
> >> Cheers,
> >>
> >> --
> >> Victor Grousset/tuxayo
> >>
> >>
> >> _______________________________________________
> >> Koha-devel mailing list
> >> Koha-devel at lists.koha-community.org
> <mailto:Koha-devel at lists.koha-community.org>
> >> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> >> website : http://www.koha-community.org/ git :
> >> http://git.koha-community.org/ bugs :
> http://bugs.koha-community.org/
> >>
> >
>
> --
> Julian Maurice
> BibLibre
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> <mailto:Koha-devel at lists.koha-community.org>
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
>
>
> --
> Tomás Cohen Arazi
> Theke Solutions (http://theke.io <http://theke.io/>) ✆ +54 9351
> 3513384
> GPG: B2F3C15F
--
Julian Maurice
BibLibre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 484 bytes
Desc: not available
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20200610/d7e3cba1/attachment.sig>
More information about the Koha-devel
mailing list