On Thu, Sep 27, 2012 at 12:36 PM, MJ Ray <mjr@phonecoop.coop> wrote:

Tomas Cohen Arazi <tomascohen@gmail.com>

...

This simple test probes dselect is currently broken on Ubuntu amd64 12.04/10.04 (as root):

I don't think it is generally broken. "dselect install" runs without problems on a 10.04 server near me (through sudo).

Running through sudo wont make any difference (for this testing purposes). I think that being broken for multiarch, which has become a default for ubuntu, means this install method is actually broken.

...

$ echo "apache2 install" | dpkg --set-selections $ dselect install Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: gcc-4.6-base:i386 libasn1-8-heimdal:i386 libc6:i386 libcap2:i386 libcomerr2:i386 libdb5.1:i386 libexpat1:i386 libgcc1:i386 libgcrypt11:i386 libgnutls26:i386 libgpg-error0:i386 libgssapi3-heimdal:i386 libhcrypto4-heimdal:i386 libheimbase1-heimdal:i386 libheimntlm0-heimdal:i386 libhx509-5-heimdal:i386 libkrb5-26-heimdal:i386 libldap-2.4-2:i386 libp11-kit0:i386 libpcre3:i386 libroken18-heimdal:i386 libsasl2-2:i386 libsasl2-modules:i386 libsqlite3-0:i386 libssl1.0.0:i386 libtasn1-3:i386 libuuid1:i386 libwind0-heimdal:i386 zlib1g:i386 0 upgraded, 29 newly installed, 0 to remove and 0 not upgraded. Need to get 8,500 kB of archives. After this operation, 22.2 MB of additional disk space will be used. Do you want to continue [Y/n]?

That just looks like some other dselect selections are set. They might be security updates and other available upgrades, because I think dselect just does those without asking. Try running dselect and picking "Select" from its menu to check what is set and, if needed, clearing any that aren't wanted. Maybe there is no suitable apache2 to install. I suspect the key thing is the ":i386" on there. I never see anything like that when running dselect. Maybe this is https://bugs.launchpad.net/ubuntu/+source/apt/+bug/879324

Belive me: 1) There's no upgrades scheduled in my PVM (did apt-get update ; apt-get dist-upgrade first) 2) There are no schedulled selections: root@koha-dev:/usr/local/src/koha-community-src# dselect install Reading package lists... Done Building dependency tree Reading state information... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Do you want to erase any previously downloaded .deb files? [Y/n] Press enter to continue. root@koha-dev:/usr/local/src/koha-community-src# echo "apache2 install" | dpkg --set-selections root@koha-dev:/usr/local/src/koha-community-src# dselect install Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: gcc-4.6-base:i386 libasn1-8-heimdal:i386 libc6:i386 libcap2:i386 libcomerr2:i386 libdb5.1:i386 libexpat1:i386 libgcc1:i386 libgcrypt11:i386 libgnutls26:i386 libgpg-error0:i386 libgssapi3-heimdal:i386 libhcrypto4-heimdal:i386 libheimbase1-heimdal:i386 libheimntlm0-heimdal:i386 libhx509-5-heimdal:i386 libkrb5-26-heimdal:i386 libldap-2.4-2:i386 libp11-kit0:i386 libpcre3:i386 libroken18-heimdal:i386 libsasl2-2:i386 libsasl2-modules:i386 libsqlite3-0:i386 libssl1.0.0:i386 libtasn1-3:i386 libuuid1:i386 libwind0-heimdal:i386 zlib1g:i386 0 upgraded, 29 newly installed, 0 to remove and 0 not upgraded. Need to get 8,500 kB of archives. After this operation, 22.2 MB of additional disk space will be used. Do you want to continue [Y/n]? n Abort. root@koha-dev:/usr/local/src/koha-community-src# apt-get install apache2 Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libaprutil1-dbd-sqlite3 libaprutil1-ldap Suggested packages: www-browser apache2-doc apache2-suexec apache2-suexec-custom The following NEW packages will be installed: apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libaprutil1-dbd-sqlite3 libaprutil1-ldap 0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded. Need to get 1,679 kB of archives. After this operation, 5,064 kB of additional disk space will be used. Do you want to continue [Y/n]?

...

So I propose the removal of dselect usage from the docs and use of simple apt-get commands (perhaps providing a convenient script).

Ouch! That seems like an overreaction to strange output on some system.

As installing on (amd64) Ubuntu is currently broken using those files/docs (as I see it), and this files are Ubuntu-specific I'm not sure its just *some* system, but the target system of those files. This is not the first time we talk about this (http://koha.1045719.n5.nabble.com/Ubuntu-support-td5720379.html) but I understand its not a issue for most of the people on the list. I've been actually doing this to get things installed: cat install_misc/ubuntu.12.04.packages | grep -v "#" | grep [a-A] | cut -f1 | xargs apt-get install --assume-yes so I don't need this, but the non-tech user does.

I would hope that QA rejects most scripts designed to be run as root. It is very wrong to encourage root to do "sh script.sh" or "./script.sh" during installation, because it could do absolutely anything to the system (think man-in-the-middle attack, or some unforseen situation which leads to a script doing rm -rf /). It's far safer to ask people to load in package lists to an apt frontend.

We could use "apt-get dselect-upgrade" as dselect is just an apt frontend, but it's one we can manipulate in bulk through dpkg.

If dselect isn't working for some people, can we bulk-update apt another way, then just let it run?

I agree. And understand why you prefer the dselect front-end. Its just broken on Ubuntu right now. As we are encouraging people to use packages I understand that changing this wouldn't affect anyone also. Regards To+

"Mark Tompsett" <mtompset@hotmail.com>

[...] I also don 't think a wrapper for sudo apt-get install calls based on an existing file we provide is any different:

$ ./install_misc/ubuntu-packages.sh -i

vs.

$ sudo apt-get install dselect $ sudo dpkg --set-selections < install_misc/ubuntu.packages $ sudo dselect

Frankly, not having to install dselect (one less thing on the server = good), less typing (less prone to errors = good), less figuring out problems (=good -- oh, use ubuntu.10.04.packages on ubuntu 10.04 instead = bad). As for your security concern, where does the line of trust get drawn? I don't think a retooled script is a big issue.

There is a big difference between asking people to run a downloaded script as an admin user and asking them to use already-installed system utilities. The system utilities have man pages, are signed by the distributor and are usually limited in the tasks that they do. Downloaded scripts have variable documentation, are rarely signed and can do absolutely everything. I feel this is a big issue: we shouldn't be encouraging people in bad security habits, like to run random scripts as root or let them sudo. Where does the line of trust get drawn? For me, it's "in god we trust... all others must bring data." It's not paranoia when they're really out to get you and there's a lot of people on the internet who really ARE out to get you. Spend some time working for Internet Service Providers (which is how our co-op started), deal with a few cracked websites and servers (which was often why people called us in - too late, but not necessarily beyond help), it might change one's view on this. One might say that an admin has to trust us to run Koha, so they should just run the script as part of that trust. But they don't need to trust us to run Koha - I've installed it without root before, although it wasn't pretty - and there's always the risk of someone setting up a spoof download.koha-community.org (I hear someone is already using koha.org for a fork...) and doing Really Bad Things in the admin script included in that one, to punish people for something or other. Or one might say that any good admin should check the script thoroughly before use, but those scripts quickly become long as they cope with more edge cases or feature creeps. How many admins would read it all? And then they've got to understand the scripting language too, which is more complex than just a few apt-get and dpkg commands as it copes with stuff like default values or stopping if an earlier command fails, which humans are pretty good at, as standard. Most of the time I dealt with such installation scripts during packaging for various distributions, I was horrified by the bad habits in them - they usually needed rewriting to stand any chance of passing QA and working in reasonably-forseeable situations. So, please, avoid a "sudo ./ubuntu-install.sh" or similar. Would it still be necessary to install dselect if the "dselect install" command was changed to "apt-get dselect-upgrade"? However, that would still suffer from ubuntu's multiarch bug. Can we bulk-edit the package selections another way? I've not yet found one. Worst case, could we use a script that suggests the right "apt-get install ..." command? Is there a flaw in that I'm not seeing? Hope that explains, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and library systems developer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire (including development) at http://www.software.coop/

Mark Tompsett schreef op za 29-09-2012 om 01:50 [+0800]:

You did a koha install WITHOUT using sudo or logging in as root? How did you configure apache, mysql, and the hundreds of perl dependencies?

Those aren't random scripts though. He's really saying that we shouldn't encourage the behaviour of doing 'sudo ./something-we-gave-you.sh' as it's bad practice. Instead, we should use the system tools to provide everything necessary. On the subject of multiarch breaking on x86_64 (which I knew aptitude had/has issues with, didn't realise apt-get did), has someone tried making the list of packages on a 64-bit machine to see a) if they differ, b) if that list works on 64-bit? FWIW, the dependency generation for the packages I tend to do on a 32-bit VM, but when I deploy I'm usually doing to a 64-bit machine and it works happily. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

"Mark Tompsett" <mtompset@hotmail.com>

...

I feel this is a big issue: we shouldn't be encouraging people in bad security habits, like to run random scripts as root or let them sudo.

You did a koha install WITHOUT using sudo or logging in as root? How did you configure apache, mysql, and the hundreds of perl dependencies?

Apache was configured through some control panel (ispconfig maybe?) and the sysadmin was happy enough to configure mysql to utf-8 which was all that was needed back then. Perl dependencies weren't fun, but cpan shell supports PREFIX installs, then you just add a bit to the SetEnv PERL5LIB in the Apache config. [...]

Take a look at bug 8840. I think this script could be generalized to include Debian. However, since the problem was Ubuntu, and not Debian, we didn't.

It's OK but I agree it does seem to be rather Ubuntu-specific at the mo. By the way (looking at the other replies), I think it is apt-get that is breaking, as "sudo dselect install" and "sudo apt-get dselect-upgrade" are the same thing. So if this multiarch bug isn't fixed in the OS, it may bite us later no matter what we do. Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and library systems developer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire (including development) at http://www.software.coop/

On 2012-10-2, at 1:25 PM, Tomas Cohen Arazi wrote:

On Mon, Oct 1, 2012 at 8:19 PM, MJ Ray <mjr@phonecoop.coop> wrote:

...

...

Take a look at bug 8840. I think this script could be generalized to include Debian. However, since the problem was Ubuntu, and not Debian, we didn't.

It's OK but I agree it does seem to be rather Ubuntu-specific at the mo.

By the way (looking at the other replies), I think it is apt-get that is breaking, as "sudo dselect install" and "sudo apt-get dselect-upgrade" are the same thing. So if this multiarch bug isn't fixed in the OS, it may bite us later no matter what we do.

You may take a look at this, they talk about wheeze and thus will bite us soon.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=664893

I'm curious, the bugs report says… "Well, your bug report shows there are still people using dselect..:-) It's of course not a big surprise that dselect doesn't support multi arch." so… whats the better alternative to dselect, that we should be using (that i know nothing about?) dpkg or aptitude, i guess?? anyone know?

Greetings, The “good” thing about this bug is that it only affects people who do git or tarball installs in a multi-arch environment. That is, it affects us as developers, not really the average user of Koha who has hopefully been transitioned to a package install. With bug 8840, instead of writing: $ sudo apt-get install dselect $ sudo dpkg --set-selections < install_misc/ubuntu.packages $ sudo dselect The instructions would look more like (it will look prettier, promise): $ ./install_misc/ubuntu-packages.sh -ic [ output trimmed, but copy the apt-get install command to the command line ] $ sudo apt-get install ... If we generalized the script to work with Debian, we could change the name again to just: check-packages.sh Generalizing to work with Debian wouldn’t be too difficult. apt-get should already support multi-architecture, as there is a configuration option and file somewhere (sorry, forgot where and haven’t started my VM) that specifically states it. Interesting related reading: http://wiki.debian.org/Multiarch/HOWTO (see the Availability section) https://wiki.ubuntu.com/MultiarchCross (linked to from the above article. See Transition section) GPML, Mark Tompsett

On Mon, Oct 1, 2012 at 11:44 PM, Chris Cormack <chris@bigballofwax.co.nz>wrote:

On 2 October 2012 15:33, Mark Tompsett <mtompset@hotmail.com> wrote:

...

Greetings,

The “good” thing about this bug is that it only affects people who do git or tarball installs in a multi-arch environment. That is, it affects us as developers, not really the average user of Koha who has hopefully been transitioned to a package install.

Not so unfortunately, most users are still using tarballs. In fact the vast majority of users install from tarball.

Only a very small amount (and it is mostly us developers and those that stumble into #koha) that install from packages. The tarball install should always work too. The packages will do more, but the tarball method should not fail to install.

As to reinforce what Chris said, let me add my grain of salt: http://koha-community.org/download-koha/ says that Koha runs on Linux, that what you need is a LAMP server plus some Perl modules. And if what you want is to download something, the you go to a page filled with tarballs. Install from ".deb" might be easier, if you use Debian or the like, but installing from tarballs should not be impossible, nor faulty! Koha is NOT a Debian only software. And the same thing is about INSTALL(s) file(s). In almost 20 years of downloading tarballs, I expect to read "the" INSTALL file to learn how to get a program up and running. In the current situation, you must read ALL install files, then get a cup of coffee and think a lot, and with luck you will find a proper way (if you are not on Debian). Bernardo

On Mon, Oct 1, 2012 at 11:33 PM, Mark Tompsett <mtompset@hotmail.com> wrote:

Greetings,

The “good” thing about this bug is that it only affects people who do git or tarball installs in a multi-arch environment. That is, it affects us as developers, not really the average user of Koha who has hopefully been transitioned to a package install.

Any 'new' Ubuntu setup (amd64) is multiarch by default. All our desktops are +4GB RAM and run 64bit OS's. Is not that rare. Multiarch is set when writing in /etc/dpkg/dpkg.cfg.d/multiarch the instruction to accept another architecture as foreign: <code> foreign-architecture i386 </code> Regards To+

El oct 2, 2012 4:56 p.m., "Mason James" <mtj@kohaaloha.com> escribió:

On 2012-10-3, at 6:53 AM, Tomas Cohen Arazi wrote:

...

On Mon, Oct 1, 2012 at 11:33 PM, Mark Tompsett <mtompset@hotmail.com>

wrote:

...

...

Greetings,

The “good” thing about this bug is that it only affects people who do git or tarball installs in a multi-arch environment. That is, it affects us as developers, not really the average user of Koha who has hopefully been transitioned to a package install.

Any 'new' Ubuntu setup (amd64) is multiarch by default. All our desktops are +4GB RAM and run 64bit OS's. Is not that rare.

Multiarch is set when writing in /etc/dpkg/dpkg.cfg.d/multiarch the instruction to accept another architecture as foreign: <code> foreign-architecture i386 </code>

Regards To+

the other workaround might be to disable multiarch, (if its enabled by default)

Really? To keep the use of dselect in the instructions? I vote for using another method for installing dependencies. I know its an overreaction, but I don't get the point of sticking to dselect anyway. If generating the deps list during install (read using a script for either triggering apt-get or printing it) is problematic, I'd rather provide qa'd copy and paste super big apt-get oneliner in the INSTALL.* files. To+

Mark Tompsett schreef op di 02-10-2012 om 10:33 [+0800]:

With bug 8840, instead of writing: $ sudo apt-get install dselect $ sudo dpkg --set-selections < install_misc/ubuntu.packages $ sudo dselect

You shouldn't really use dselect. You can use apt-get dselect-upgrade however. I've found this bug, which seems to the be the issue: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/879324 I tested a workaround, Ubuntu 12.04.1 64-bit: $ sudo dpkg --set-selections < ubuntu.packages $ sudo apt-get update # Turn off multiarch $ sudo perl -p -i -e 's/^/#/' /etc/dpkg/dpkg.cfg.d/multiarch $ sudo apt-get dselect-upgrade -u --no-install-recommends # Turn multiarch back on $ sudo perl -p -i -e 's/^#//' /etc/dpkg/dpkg.cfg.d/multiarch this worked. I don't have a 32-bit install handy but, I think this shouldn't cause problems there either (although there may be no multiarch file, so that bit may error.) It isn't necessary on Debian. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

Greetings, But the question is: will those instructions be the same for all debian-based OSes? Generic instructions are so much nicer. GPML, Mark Tompsett

Mason James wrote:

"Well, your bug report shows there are still people using dselect..:-) It's of course not a big surprise that dselect doesn't support multi arch."

so… whats the better alternative to dselect, that we should be using (that i know nothing about?)

dpkg or aptitude, i guess??

We're already using dpkg, specifically --set-selections, but dpkg doesn't do the downloads for you. I think this is actually a bug in apt-get, not dselect, as I think "dselect install" just runs "apt-get dselect-upgrade" now. dselect may refuse to run, but I bet apt-get dselect-upgrade still might try. I fear that the list may get too long for a command line. I don't see anything in "man aptitude" or "man apt-get" which suggests how to load a package list file for installation. At best, could we print multiple aptitude --schedule-only install ... lines, followed by a plain aptitude install? Not sure on aptitude now, because I think the last two debian release notes have changed back to apt-get. Hope that helps, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and library systems developer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire (including development) at http://www.software.coop/