Questions from the great wall of install...
Greetings, As I am putting finishing touches on the Ubuntu great wall of install, for the purposes of *splitting* and organizing (packages/git/tarball – the latter of which will be hidden), I have written down questions that I thought perhaps the list may be able to clarify. 1) When doing a packages install, a message about upgrading from 3.2 to 3.4 pops up. What is the recommended upgrade path from pre-3.4 to current? Is that message accurate? 2) I noticed that Mime::Lite (forgive wrong capitalizations) triggers the install of nullmailer under a fresh install of Ubuntu 12.04 LTS Server. Seeing as Koha has cronjobs that do mailing, are their any external links (like Vimal Kumar V.’s link to a gmail relay set up) which would be recommended for getting those functions to work properly? I understand that a “configuring mailing” wiki page is beyond the scope of Koha, but surely a “look here for suggested hints, but configuration of mailing is out of scope” page is okay? 3) In the Ubuntu documentation the ordering of /etc/perl/XML/SAX/ParserDetails.ini is supposedly important, as “You must be sure you're using the XML::LibXML SAX parser, not Expat or PurePerl, both of which have outstanding bugs with pre-composed characters.” During a packages installation, I decided to look at the ini file, and found it wasn’t in the recommended order. Is this a problem with the packages or has this point become irrelevant? 4) Does the packages installation method set up apache2 correctly, or does AddCharset UTF-8 .utf8 AddDefaultCharset UTF-8 Still need to be added to a configuration file and a restart issued? It’s not in the Debian instructions for packages. 5) In MySQL “select variables like ‘%colla%’;” and “select variables like ‘%char%’;” generate non-utf8 entries. Is this correct? Nothing like this is mentioned in the Debian instructions for packages. I think that’s it for now. I hope to start the splitting in the near future, and be done before the end of the week. I’ve got an upgrade (3.6.3 tarball to 3.8.5 packages) to do this weekend. GPML, Mark Tompsett
On 16 September 2012 19:23, Mark Tompsett <mtompset@hotmail.com> wrote:
Greetings,
As I am putting finishing touches on the Ubuntu great wall of install, for the purposes of *splitting* and organizing (packages/git/tarball – the latter of which will be hidden), I have written down questions that I thought perhaps the list may be able to clarify.
1) When doing a packages install, a message about upgrading from 3.2 to 3.4 pops up. What is the recommended upgrade path from pre-3.4 to current? Is that message accurate?
Yes, you must run that if you are coming from pre 3.4
2) I noticed that Mime::Lite (forgive wrong capitalizations) triggers the install of nullmailer under a fresh install of Ubuntu 12.04 LTS Server. Seeing as Koha has cronjobs that do mailing, are their any external links (like Vimal Kumar V.’s link to a gmail relay set up) which would be recommended for getting those functions to work properly? I understand that a “configuring mailing” wiki page is beyond the scope of Koha, but surely a “look here for suggested hints, but configuration of mailing is out of scope” page is okay?
I'd prefer it linked to a more generic best practices for configuring and running a mail server. Something that is in no way Koha specific.
3) In the Ubuntu documentation the ordering of /etc/perl/XML/SAX/ParserDetails.ini is supposedly important, as “You must be sure you're using the XML::LibXML SAX parser, not Expat or PurePerl, both of which have outstanding bugs with pre-composed characters.” During a packages installation, I decided to look at the ini file, and found it wasn’t in the recommended order. Is this a problem with the packages or has this point become irrelevant?
No idea
4) Does the packages installation method set up apache2 correctly, or does AddCharset UTF-8 .utf8 AddDefaultCharset UTF-8 Still need to be added to a configuration file and a restart issued? It’s not in the Debian instructions for packages.
Pass, take a look and see :)
5) In MySQL “select variables like ‘%colla%’;” and “select variables like ‘%char%’;” generate non-utf8 entries. Is this correct? Nothing like this is mentioned in the Debian instructions for packages.
I don't know what this question means. Chris
I think that’s it for now. I hope to start the splitting in the near future, and be done before the end of the week. I’ve got an upgrade (3.6.3 tarball to 3.8.5 packages) to do this weekend.
GPML, Mark Tompsett
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Greetings, Thank you for the replies. Let me clarify/ask differently. And anyone else, please feel free to jump in with answers and insights.
1) When doing a packages install, a message about upgrading from 3.2 to 3.4 pops up. What is the recommended upgrade path from pre-3.4 to current? Is that message accurate?
Yes, you must run that if you are coming from pre 3.4
So, the message is confusing then. Because, from what I gather from your reply that if I had 3.0.0 and wanted to upgrade to 3.8.5 this coming weekend, I would need to run this script, but the message implies only if the script is only applicable if upgrading to 3.4.x specifically. And of course, this is all prior to the web-install step, after the person has loaded their old data into the database created by the koha-create command, right? (Anyone feel free to jump in) [snip 2 -- mailing configuration issues]
I'd prefer it linked to a more generic best practices for configuring and running a mail server. Something that is in no way Koha specific.
Yes, that would be great. Does anyone have such links? [ 3 - ParserDetails.ini inquiry snip] [ 4 - Apache UTF8 configuration inquiry snip ]
5) In MySQL “select variables like ‘%colla%’;” and “select variables like ‘%char%’;” generate non-utf8 entries. Is this correct? Nothing like this is mentioned in the Debian instructions for packages.
I don't know what this question means.
Those two MySQL queries confirm how default databases and tables are created, as far as I know. Latin1 is not UTF-8, and may cause issues with non-Latin charaters, correct? The question was what steps, if any, need to be taken to ensure everything is UTF-8 compliant? I didn't read the koha-create script yet, but I assume it would have forced UTF-8 compliance on the database generation. Am I making a bad assumption? GPML, Mark Tompsett
On 2012-09-17, at 6:37 PM, Mark Tompsett wrote:
Greetings,
Thank you for the replies. Let me clarify/ask differently. And anyone else, please feel free to jump in with answers and insights.
1) When doing a packages install, a message about upgrading from 3.2 to 3.4 pops up. What is the recommended upgrade path from pre-3.4 to current? Is that message accurate?
Yes, you must run that if you are coming from pre 3.4
So, the message is confusing then.
yeah, i guess it should say 'upgrading from 3.2.x or earlier, to 3.4.x or later'
Op 17-09-12 10:03, Mason James schreef:
yeah, i guess it should say 'upgrading from 3.2.x or earlier, to 3.4.x or later'
Actually, it should be changed so that it detects that you're coming from 3.2 or earlier and only tells you about it then. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
At 02:37 PM 9/17/2012 +0800, Mark Tompsett wrote: [snip]
5) In MySQL âselect variables like â%colla%â;â and âselect variables like â%char%â;â generate non-utf8 entries. Is this correct? Nothing like this is mentioned in the Debian instructions for packages.
I don't know what this question means.
Mysql and character sets can be daunting, because of the dominance over the years of Latinx (tnx WG) but is improving in the direction of UTF8 as more and more RFCs and developers wake up (on one of our servers I even have to use a dedicated, non-UTF8 instance of Mysql to take care of our "legacy" 7 year-old, updated every year, accounting programs.) What you should ideally find is: mysql> show variables like 'char%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | utf8 | | character_set_connection | utf8 | | character_set_database | utf8 | | character_set_filesystem | binary | | character_set_results | utf8 | | character_set_server | utf8 | | character_set_system | utf8 | | character_sets_dir | /usr/share/mysql/charsets/ | +--------------------------+----------------------------+ 8 rows in set (0.00 sec) What you often find (particularly over a network, even using quite up-to-date tunneling) is:mysql> show variables like 'char%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | latin1 | | character_set_connection | latin1 | | character_set_database | utf8 | | character_set_filesystem | binary | | character_set_results | latin1 | | character_set_server | utf8 | | character_set_system | utf8 | | character_sets_dir | /usr/share/mysql/charsets/ | +--------------------------+----------------------------+ 8 rows in set (0.00 sec)
Those two MySQL queries confirm how default databases and tables are created, as far as I know. Latin1 is not UTF-8, and may cause issues with non-Latin charaters, correct? The question was what steps, if any, need to be taken to ensure everything is UTF-8 compliant? I didn't read the koha-create script yet, but I assume it would have forced UTF-8 compliance on the database generation. Am I making a bad assumption?
I was fairly certain that so long as character_set_database | system | server are UTF8, Koha should not be in trouble (the potential danger is not being able to *restore* a dump, as Mysql does not necessarily report corruption while actually making the backup.) BUT ... Mysql also uses "collation" char sets for textual fields. I've just had a looked at our older 3.6.7 db on the sandbox and the production 3.8.4, and find: mysql> show table status where collation like '%latin%' \G *************************** 1. row *************************** Name: fundmapping [snip] Collation: latin1_swedish_ci [snip] 1 row in set (0.09 sec) I *think* (see <http://wiki.koha-community.org/wiki/DB_schema_bugs>) this is probably a legacy from 3.2 and is no longer required, then: 'DROP TABLE IF EXISTS fundmapping;' If it's required (I just restored after DROPping the table) then: 'ALTER TABLE fundmapping CHARACTER SET utf8 COLLATE utf8_general_ci;' Both command lines work on our sandbox: mysql> show table status where collation like '%latin%' \G Empty set (0.08 sec) Note that we do not use acquisitions in Koha, so I really have no data to test with. Maybe someone else could confirm the 'DROP' *if* the table is not required, or test whether non-latin utf8 entries in this table are correctly restorable? Best - Paul
Op 16-09-12 09:23, Mark Tompsett schreef:
1) When doing a packages install, a message about upgrading from 3.2 to 3.4 pops up. What is the recommended upgrade path from pre-3.4 to current? Is that message accurate?
Replied about this already. It could do with editing.
2) I noticed that Mime::Lite (forgive wrong capitalizations) triggers the install of nullmailer under a fresh install of Ubuntu 12.04 LTS Server. Seeing as Koha has cronjobs that do mailing, are their any external links (like Vimal Kumar V.’s link to a gmail relay set up) which would be recommended for getting those functions to work properly?
There are a billion ways to set up a mail server, but if someone finds a good "follow this to get something secure and running" guide, I could see linking to it being a good thing.
3) In the Ubuntu documentation the ordering of /etc/perl/XML/SAX/ParserDetails.ini is supposedly important, as “You must be sure you're using the XML::LibXML SAX parser, not Expat or PurePerl, both of which have outstanding bugs with pre-composed characters.” During a packages installation, I decided to look at the ini file, and found it wasn’t in the recommended order. Is this a problem with the packages or has this point become irrelevant?
It's never been an issue, I think it stopped being such with the release of Squeeze.
4) Does the packages installation method set up apache2 correctly, or does AddCharset UTF-8 .utf8 AddDefaultCharset UTF-8 Still need to be added to a configuration file and a restart issued? It’s not in the Debian instructions for packages.
It's not set by the packages. I wouldn't include it in instructions, I'd include it in the package configuration. I'd also note the manual for it: http://httpd.apache.org/docs/2.2/mod/core.html#adddefaultcharset
This directive specifies a default value for the media type charset parameter (the name of a character encoding) to be added to a response if and only if the response's content-type is either text/plain or text/html. ... AddDefaultCharset should only be used when all of the text resources to which it applies are known to be in that character encoding and it is too inconvenient to label their charset individually. One such example is to add the charset parameter to resources containing generated content, such as legacy CGI scripts, that might be vulnerable to cross-site scripting attacks due to user-provided data being included in the output. Note, however, that a better solution is to just fix (or delete) those scripts, since setting a default charset does not protect users that have enabled the "auto-detect character encoding" feature on their browser.
5) In MySQL “select variables like ‘%colla%’;” and “select variables like ‘%char%’;” generate non-utf8 entries. Is this correct? Nothing like this is mentioned in the Debian instructions for packages.
I've not noticed it being a problem. Test it and see if it's an issue? Also, if it was determined to be useful, it'd be better off being set in the package's mysql configuration rather that putting it instructions. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
participants (5)
-
Chris Cormack -
Mark Tompsett -
Mason James -
Paul -
Robin Sheat