Prepackaged perl distro
Hi Developers, I worked in an environment where a specific set of Perl packages known to work together well with everything else, i.e. database and web services, were required to create a virtual machine. We cobbled together perl + all perl modules required. The virtual builder then simply installed perl and all the perl modules we required. No CPAN. No failed makes. No mismatches. Can't the same be done for Koha? cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor A lifetime of listening to disco music is a high price to pay for one's sexual preference. -- Quentin Crisp _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
* Rick Welykochy (rick@praxis.com.au) wrote:
Hi Developers,
I worked in an environment where a specific set of Perl packages known to work together well with everything else, i.e. database and web services, were required to create a virtual machine.
We cobbled together perl + all perl modules required.
The virtual builder then simply installed perl and all the perl modules we required.
No CPAN. No failed makes. No mismatches.
Can't the same be done for Koha?
Im sure it could, you volunteering Rick? :) Chris _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
No CPAN. No failed makes. No mismatches.
Can't the same be done for Koha?
Im sure it could, you volunteering Rick? :)
Chris
This is a brilliant idea _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
Chris Cormack wrote:
We cobbled together perl + all perl modules required.
The virtual builder then simply installed perl and all the perl modules we required.
No CPAN. No failed makes. No mismatches.
Can't the same be done for Koha?
Im sure it could, you volunteering Rick? :)
If in your opinion it would fly for Linux (as a start) I wouldn't mind giving it a try. I am familiar with care and feeding of Koha/2.2.9 but have not ventured very far into Koha/3. With 3.0 I got as far as prepping for system-wide Unicode support, as that was highly recommended by the install notes. I have not conquered Unicode on Linux/deb yet. I would imagine it is important for universal acceptance. (We have been spoilt by living in ASCII 7 bit land!) That said, my guess for next steps is to examine the Koha installation procedure for 3.0 and then unravel all the perl modules required. Then get some advice on rewriting the relevant parts of the installation. I can do the perl stuff no probs. Hopefully my solution would work with all linux distros that have gcc installed. Winders is another matter. I left Winders behind in 1998 so cannot directly comment, but my guess is that since C compilation on Windows is a bit "tedious" for an out of the box XP or Vista, that the Active-State precompiled perl modules are the go. Anywho, I could concentrate solely on the Linux way of installation and then a Windows guru can adapt that. Yeah, let's give it a go. And git ... I'll need to (finally) learn some git. Not a bad thing, imho. Anyone else want to come on board as perhaps an advisor? cheers rickw p.s. I just got a fresh new Asus EEE PC running Xandros, so I might use that for starters, just because it sounds challenging. Besides, my perl packaging idea should definitely not rely on anything debian-specific (as but one example) so the Asus is a good choice. I know nothing of its innards yet and thus will not rely on anything distro specific. I hope. In a word: it should be distro agnostic. -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor A lifetime of listening to disco music is a high price to pay for one's sexual preference. -- Quentin Crisp _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
On 2008/07/16, at 11:48 PM, Rick Welykochy wrote:
Chris Cormack wrote:
We cobbled together perl + all perl modules required.
The virtual builder then simply installed perl and all the perl modules we required.
No CPAN. No failed makes. No mismatches. Can't the same be done for Koha?
Im sure it could, you volunteering Rick? :)
If in your opinion it would fly for Linux (as a start) I wouldn't mind giving it a try.
Anyone else want to come on board as perhaps an advisor?
I do have a bit of a pet itch to automate a big chunk of the perl- module-install stuff that needs to be done before a koha install So I'm keen to do some work on this too My idea was to continue to use CPAN still - but your idea and mine dovetail quite nicely Perhaps something like.. do a search for a perl module in a tgz/par file if none is found then attempt a cpan search FYI: take a look at MJ Ray's 'install-CPAN.pl' script as a starting point http://git.koha.org/cgi-bin/gitweb.cgi?p=Koha;a=blob;f=install-CPAN.pl Mason.
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
I think this idea is easy to do for a single target platform and harder to do for heterogeneous platforms. For one thing you have current versions of Ubuntu shipping with perl 5.10 (and its variations in core). The available methods include: - Bundle::Koha - CPAN "autobundle" - platform-specific packages Making a Bundle::Koha is probably the easiest and most achievable option. The best evidence of this is that it has been done already: http://search.cpan.org/~koha/Bundle-KohaSupport-0.08/KohaSupport.pm<http://search.cpan.org/%7Ekoha/Bundle-KohaSupport-0.08/KohaSupport.pm> Unfortunately that version is way out of date (since the author doesn't appear to consider versioning, perhaps Koha v1.0!) and it contains duplicates, but the idea is the same. The autobundle approach would be more of a shotgun method, getting stuff that isn't Koha-related and potentially limiting portability. Building platform specific packages is the most difficult of the 3, but also the strongest solution for a given platform. This would be consistent with your desire for "no CPAN", but not distro-agnostic. I think the way to go is to make a current Bundle::Koha and let packagers build off of that. --Joe Atzberger On Wed, Jul 16, 2008 at 7:48 AM, Rick Welykochy <rick@praxis.com.au> wrote:
Chris Cormack wrote:
We cobbled together perl + all perl modules required.
The virtual builder then simply installed perl and all the perl modules we required.
No CPAN. No failed makes. No mismatches.
Can't the same be done for Koha?
Im sure it could, you volunteering Rick? :)
If in your opinion it would fly for Linux (as a start) I wouldn't mind giving it a try.
I am familiar with care and feeding of Koha/2.2.9 but have not ventured very far into Koha/3.
With 3.0 I got as far as prepping for system-wide Unicode support, as that was highly recommended by the install notes. I have not conquered Unicode on Linux/deb yet. I would imagine it is important for universal acceptance. (We have been spoilt by living in ASCII 7 bit land!)
That said, my guess for next steps is to examine the Koha installation procedure for 3.0 and then unravel all the perl modules required.
Then get some advice on rewriting the relevant parts of the installation. I can do the perl stuff no probs.
Hopefully my solution would work with all linux distros that have gcc installed.
Winders is another matter. I left Winders behind in 1998 so cannot directly comment, but my guess is that since C compilation on Windows is a bit "tedious" for an out of the box XP or Vista, that the Active-State precompiled perl modules are the go.
Anywho, I could concentrate solely on the Linux way of installation and then a Windows guru can adapt that.
Yeah, let's give it a go.
And git ... I'll need to (finally) learn some git. Not a bad thing, imho.
Anyone else want to come on board as perhaps an advisor?
cheers rickw
p.s. I just got a fresh new Asus EEE PC running Xandros, so I might use that for starters, just because it sounds challenging. Besides, my perl packaging idea should definitely not rely on anything debian-specific (as but one example) so the Asus is a good choice. I know nothing of its innards yet and thus will not rely on anything distro specific. I hope. In a word: it should be distro agnostic.
-- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
Rick Welykochy <rick@praxis.com.au> wrote: [...]
We cobbled together perl + all perl modules required.
The virtual builder then simply installed perl and all the perl modules we required.
No CPAN. No failed makes. No mismatches.
Can't the same be done for Koha?
Yes, and there are several ways to do it, including ones that I think are called PAR and PPM, but they should be able to build on the work we've done with Makefile.PL and they really need to be built on a stable release. However, some of the modules involve compiled code will be different on each binary platform. Also, some systems (debian, Fedora, Strawberry Perl, more?) have their own ways of bundling modules together which avoids having multiple maybe-incompatible versions on the system at the same time. Hopefully they can build on Makefile.PL too and we don't have to keep updating INSTALL.debian, INSTALL.fedora and so on forever: those files mostly suck time and should slowly die. Regards, -- MJ Ray (slef) Webmaster for hire, statistician and online shop builder for a small worker cooperative http://www.ttllp.co.uk/ http://mjr.towers.org.uk/ (Notice http://mjr.towers.org.uk/email.html) tel:+44-844-4437-237 _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
MJ Ray wrote:
Yes, and there are several ways to do it, including ones that I think are called PAR and PPM, but they should be able to build on the work we've done with Makefile.PL and they really need to be built on a stable release.
I will investigate PAR and PPM, whatever they are. I have looked at Bundle and it looks pretty brain dead IMHO.
However, some of the modules involve compiled code will be different on each binary platform. Also, some systems (debian, Fedora, Strawberry Perl, more?) have their own ways of bundling modules together which avoids having multiple maybe-incompatible versions on the system at the same time. Hopefully they can build on Makefile.PL too and we don't have to keep updating INSTALL.debian, INSTALL.fedora and so on forever: those files mostly suck time and should slowly die.
One solution is to use a different location for the set of Perl modules used by Koha. This avoids any entanglements with existing Perl installations on the box. It is a very clean soln :) cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor A lifetime of listening to disco music is a high price to pay for one's sexual preference. -- Quentin Crisp _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
One solution is to use a different location for the set of Perl modules used by Koha. This avoids any entanglements with existing Perl installations on the box. It is a very clean soln :)
We do that for Koha's internal perl modules, and I think it makes sense there, but I would be very against doing it for *external* dependencies. It doesn't avoid entanglements with existing perl installations, because if you have 5.10 installed (e.g. Ubuntu) you already have a different core and @INC. It may keep you out of the CPAN'd module tree, but none of us wants to step in to fill the role of CPAN for generally available modules like DBI, DBD:mysql, etc. Once you stick your own copies of modules in the system somewhere, you're responsible for updating them. At best, we'd be duplicating modules that the system may already have, and in any case *should* have installed properly. Inevitably, versions will fall out of joint and debugging the difference between one side of the system's perl modules and our side's nearly identical modules will be more headache than it is worth. By the same token, if we were to create any modules that were abstracted enough to be generally useful outisde of Koha, then we should post them to CPAN and keep them up to date. This would be like MARC::File or MARC::File::XML, currently maintained by Mike Rylander or Josh, respectively, and certainly useful in more than just one ILS setting. --Joe Atzberger _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
Joe Atzberger wrote:
We do that for Koha's internal perl modules, and I think it makes sense there, but I would be very against doing it for *external* dependencies. It doesn't avoid entanglements with existing perl installations, because if you have 5.10 installed (e.g. Ubuntu) you already have a different core and @INC.
There is a clear separation between the Perl CORE and so-called "site_perl" modules installed after the fact. The CORE should be (must be?) backwards compatible with all of CPAN, subject to the Perl versioning constraints inside the modules, i.e. the use 5.8 kind of specificier. I am proposing the following: @INC/PERL5LIB to contain: /usr/lib/perl stuff (the core only) + /usr/local/koha/site_perl (the Koha Perl Kit as I call it) + /usr/local/koha/intranet/modules (the Koha app modules) And yup, there might be two DBI packages on the system. So be it. If queries arise about which CPAN packages are installed, they must be directed at the /usr/local/koha/site_perl, not anywhere else. A mix of locally installed CPAN modules + just a subset from the Koha Perl Kit is just the thing I want to avoid, for version compatibility reasons. The overarching goal: install a set of CPAN modules alongside the Koha app modules that are *known* to be 100% compatible. Versions must be *exact* and known to work together. The PERL CORE is a different kettle of fish. If Larry Wall and Co have introduced backwards compat issues at that low level, I certainly want to know about it. For example, what are the differences introduced at Perl 10.0 in the CORE that you refer to? Would they affect an existing Koha/2.2.9 or Koha3.0 install? cheers rick -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor I forget what I was taught. I only remember what I have learnt. -- Patrick White _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
what are the differences introduced at Perl 10.0 in the CORE that you refer to? Would they affect an existing Koha/2.2.9 or Koha3.0 install?
The cause/culprit isn't clear, but we definitely have compatibility issues between 5.10 and MARC::File::XML, for example. See bug #2309: http://bugs.koha.org/cgi-bin/bugzilla/show_bug.cgi?id=2309 --joe
Joe Atzberger wrote:
what are the differences introduced at Perl 10.0 in the CORE that you refer to? Would they affect an existing Koha/2.2.9 or Koha3.0 install?
The cause/culprit isn't clear, but we definitely have compatibility issues between 5.10 and MARC::File::XML, for example. See bug #2309: http://bugs.koha.org/cgi-bin/bugzilla/show_bug.cgi?id=2309
I see no evidence of Perl CORE causing trouble. All that happened was a method was called on a undefined value. My experience tells me that this is an incompatiblity between MARC::File::SAX and XML::LibXML::SAX and MARC::File::XML, or else a bug in the first. Also, my experience with XML parsing in Perl (and Koha) has been less than "calming" due to the multiple implementations of XML parsing from CPAN, multiple incompatible C libraries (libxml*) that do that parsing and the interactions of C-based XML parsing and the native Perl parser. Methinks the problems in that bug report might be related to that. Why would you deduce that it is the Perl CORE at fault? I have been using Perl since 1995 and have *never* encountered incompatibilities going forward. cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor Blessed are the cracked for they let in the light. -- Spike Milligan
Why would you deduce that it is the Perl CORE at fault?
I didn't: It's not my bug. The conclusion was that the 5.10 installation as whole held some incompatibility. I agree the slippery XML parser is a likely culprit (and a serious pain), possibly because the "wrong" one is already pre-installed(?) for the OS.
using Perl since 1995 and have *never* encountered incompatibilities going forward.
See the "Incompatible Changes" section of perldelta for any release, including the current one: http://perldoc.perl.org/perldelta.html#Incompatible-Changes http://perldoc.perl.org/perl561delta.html#Incompatible-Changes http://perldoc.perl.org/perl58delta.html#Incompatible-Changes Perl is still a model to me of how things should go, but the number of times it has been necessary to break compatibility is non-zero. --joe
Joe Atzberger wrote:
I agree the slippery XML parser is a likely culprit (and a serious pain), possibly because the "wrong" one is already pre-installed(?) for the OS.
Yes, the XML mess has bitten me before. It is probably something beyond the scope of Koha installation to prevent. *UNLESS* perl and all its required C libs are plunked into a separate area and a known-to-be-right combination of everything perl is used. A big ask, I know. Esp. for those who want Koha to just work out of the box with their existing perl distro.
See the "Incompatible Changes" section of perldelta for any release, including the current one: http://perldoc.perl.org/perldelta.html#Incompatible-Changes http://perldoc.perl.org/perl561delta.html#Incompatible-Changes http://perldoc.perl.org/perl58delta.html#Incompatible-Changes
Excellent reading! Thanks for the links. As for progress of the prepackaged "Koha Perl Kit", I am investigating the various existing ways to do it and comparing to how I've accomplished it in the past with a home-grown script. I think I'll test my methodology on a clean Linux install and Koha/2.2.9, since that is what I am familiar with. It should then be easy to port to Koha/3. Once I have something ready to test, it would be great to dig up some testers on various platforms/distros to give it a try. I think a Windows version will be "out of scope" for this project. cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor Blessed are the cracked for they let in the light. -- Spike Milligan
Rick Welykochy <rick@praxis.com.au> wrote:
Yes, the XML mess has bitten me before. It is probably something beyond the scope of Koha installation to prevent.
*UNLESS* perl and all its required C libs are plunked into a separate area and a known-to-be-right combination of everything perl is used. A big ask, I know. Esp. for those who want Koha to just work out of the box with their existing perl distro.
Just to be as sure as we can be, shouldn't we bundle that perl and all its required C libs with copies of zebra, MySQL, Apache httpd, Linux and the tools needed to get it all running? We could use the debian or gobolinux live CDs as a starting point. No longer sure if I'm serious, -- MJ Ray (slef) Webmaster for hire, statistician and online shop builder for a small worker cooperative http://www.ttllp.co.uk/ http://mjr.towers.org.uk/ (Notice http://mjr.towers.org.uk/email.html) tel:+44-844-4437-237
MJ Ray wrote:
Just to be as sure as we can be, shouldn't we bundle that perl and all its required C libs with copies of zebra, MySQL, Apache httpd, Linux and the tools needed to get it all running?
Actually, that is the only way to guarantee a working Koha install. This is standard practice in enterprise systems. Everything from the ground up to the app must be fit together, tested, approved and then locked. Subject to security upgrades which is an orthogonal issue.
We could use the debian or gobolinux live CDs as a starting point. No longer sure if I'm serious,
I'm taking you seriously. There is no point of doing all this work on a Koha Perl Kit if there is a need for a Koha Live CD. I believe work has already been done on such a thing, and abandoned years ago. Efforts might better be spent on that: Koha out of the box. The installer for that would be a dream: wipe a hard drive and install the entire system. I'm lovin' it :) cheers rickw -- ________________________________________________________________ Rick Welykochy || Praxis Services || Internet Driving Instructor Blessed are the cracked for they let in the light. -- Spike Milligan
On Thu, Jul 24, 2008 at 8:45 PM, MJ Ray <mjr@phonecoop.coop> wrote:
Rick Welykochy <rick@praxis.com.au> wrote:
Yes, the XML mess has bitten me before. It is probably something beyond the scope of Koha installation to prevent.
*UNLESS* perl and all its required C libs are plunked into a separate area and a known-to-be-right combination of everything perl is used. A big ask, I know. Esp. for those who want Koha to just work out of the box with their existing perl distro.
Just to be as sure as we can be, shouldn't we bundle that perl and all its required C libs with copies of zebra, MySQL, Apache httpd, Linux and the tools needed to get it all running?
We could use the debian or gobolinux live CDs as a starting point. What ever happened to the idea of packaging Koha for one or more GNU/Linux distros (MJ, aren't you a Debian contributor?) ?
Cheers, -- Joshua Ferraro SUPPORT FOR OPEN-SOURCE SOFTWARE CEO migration, training, maintenance, support LibLime Featuring Koha Open-Source ILS jmf@liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS
"Joshua Ferraro" <jmf@liblime.com> wrote:
What ever happened to the idea of packaging Koha for one or more GNU/Linux distros (MJ, aren't you a Debian contributor?) ?
I'm a debian developer, if that's what you mean, but I've been stuck doing most development on OSX for the last few months. Chris Cormack has been blogging great work about debian packaging and I'd be happy to sponsor that into the main repository if possible. I don't think getting it in before this weekend's freeze is realistic, though - is it even desirable? Let's wait for Koha 3.0.0-final. Regards, -- MJ Ray (slef) Webmaster for hire, statistician and online shop builder for a small worker cooperative http://www.ttllp.co.uk/ http://mjr.towers.org.uk/ (Notice http://mjr.towers.org.uk/email.html) tel:+44-844-4437-237
participants (6)
-
Chris Cormack -
Joe Atzberger -
Joshua Ferraro -
Mason James -
MJ Ray -
Rick Welykochy