- Koha-devel - lists.koha-community.org

Problematic Zebra Charmaps Equivalences
by David Cook 24 Nov '15

24 Nov '15

Hi all: Yet another Zebra email from this guy. I don’t know how many of you are using CHR vs ICU, but CHR is the default for installs, so I’m guessing that it’s quite a few. Well, there are some issues with how we use the equivalent directive. Hopefully the UTF-8 won’t be stripped out of this message, although I’m guessing it might… Here’s all instances of the directive in word-phrase-utf.chr: # Characters to be considered equivalent for sorting purposes equivalent aáàãåâăąȧǎȁȃ equivalent ӕä(ae) equivalent ā(aa) equivalent iíìîịĩĭįǐȉȋ equivalent ï(ie) equivalent ī(ii) equivalent uúùûũŭųűǔȕȗ equivalent ü(ue) equivalent ū(uu) equivalent eéèêẽĕęėěȅȇ equivalent ëē(ee) equivalent oóòõôŏǫȯőǒȍȏ equivalent Œœöø(oe) equivalent ō(oo) Firstly, that comment is wrong. “equivalent” isn’t just for sorting purposes. It’s for searching purposes. Indexdata have confirmed that the documentation is wrong about the sorting thing. So “ie” and ï (if you can’t see this character, it’s the UTF-8 representation of ï) are equivalent. That means searches for “siemon” will get results for “siemon” and “sïmon”. Now, there is also a “map” directive: map ï i This means that “sïmon” is the same as “simon”. Now, “map” affects both indexing and searching. If you have “sïmon” in a record, you can see that it is actually stored as “simon” in Zebra, if you do a search for it and use “format xml” and “elements zebra::index”. So your search for “siemon” will really get results for “siemon” and “simon”. This really isn’t ideal. However, you can see why you’d want equivalences. In Scandinavian languages, I think “å” and “aa” are roughly equivalent. They’re spelled differently but they’re the same sound. So if you search for “Gaard”, you might want hits for “Gård” as well. But you might not want “career” to be equivalent to “carer” as they’re two different words. Or “choose” to be equivalent to “chose”, “sloop” - "slop”, “reef” - "ref”, etc. -- Unfortunately, I don’t really know what the solution is. For one client, I’ve disabled the equivalent directive where it creates an equivalence between any two letter combination with a one letter combination, as they only have records in English, and it’ll just cause them headaches. I can see this being useful for multilingual records… although I think many people with multilingual records use ICU. I don’t know ICU well enough to know how it manages characters that English speakers would think of as accents or ligatures. I know you can provide your own normalization with ICU, but I think it does a fair amount on its own as well… I think some of the difficulties are mentioned here: http://userguide.icu-project.org/collation/icu-string-search-service. It also mentions the Danish å/aa example. I don’t know how ICU would know how to handle particular languages… that webpage seems to indicate you can provide a locale to deal with it. Of course, that doesn’t necessarily solve things. If you have multilingual records with multilingual users, how do you choose your rules? Sure, you might be able to specify a locale at search time (note you can’t do this with Zebra), but what rules did you specify at index time? As anyone who has watched this video (https://www.youtube.com/watch?v=0j74jcxSunY) would know, internationalis(z)ing code has many challenges… -- Anyway, the reason for this email is mostly just to make you all aware of this issue, and how “equivalent” and “map” work in the Charmap files when using CHR indexing. Oh, also, if you look at “default.idx”, you’ll see that “sort s” references “charmap sort-string-utf.chr”, but I don’t think sort-string-utf.chr actually exists anywhere… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

3 3

Help about bug 15207
by Zeno Tajoli 21 Nov '15

21 Nov '15

Hi to all, I trying to resolve bug 15207, Error on upgrade from 3.20.5 to 3.22 beta http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15207. The second error (insert defaults in the table 'permissions') it easy, I change 'INSERT' with 'INSERT IGNORE' and all is OK. The first one is much more difficult: insert a CONSTRAINT but if you update from 3.20.5 this CONSTRAINT is just present. I don't find a similar situation in others parts of installer/data/mysql/updatedatabase.pl I try a solution (check the presence of the CONSTRAINT using information_schema.table_constraints) but it doesn't work always. For example it doesn't work for Katrin. I don't know what to do now. Do you have any idea about this topic ? Bye Zeno Tajoli

2 1

About level of collaction in MySQL
by Zeno Tajoli 21 Nov '15

21 Nov '15

Hi to all, working on bug 15207, Error on upgrade from 3.20.5 to 3.22 beta, http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15207 Bernardo finds same missing bit about collate info in sound alerts table and he propose a patch about it. As I see collate info are not mandatory for us at column or table level, we setup it at database level. I see that mysql works with levels to define collation [server, database, table, column]. So which level of collation do we want ? Bye Zeno Tajoli

1 0

E noho rā, Koha
by Robin Sheat 20 Nov '15

20 Nov '15

Hi, Koha folks. Tomorrow is my last day at Catalyst, and the end of my last week in New Zealand. From December, I'm starting a new job that is located only 1,450km distance from as far away as it is possible to get from Wellington on earth. That is, in Amsterdam*. It's been a pleasure meeting those of you that I've met, emailing and IRCing those that I haven't, and I'm sure you'll all keep the project heading on its current course to taking over the world :) Interesting note: my first commit was on April 20th, 2010, signed off by Galen, who has happily taken over my package manager role with almost no blackmai^Wbriber^Wconvincing needed. I'll probably still be lurking in IRC though, just in a quite different timezone to what it was, and skimming emails trying not to answer them. Ka kite anō au i a koutou, -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF * Marcel, I'm going to have to get you to show me the Rijksmuseum library some time :)

5 4

Re: [Koha-devel] incremental backup of the DB
by Paul A 19 Nov '15

19 Nov '15

At 02:34 AM 11/18/2015 +0530, Indranil Das Gupta wrote: >Hi all, > >I'm facing a situation that between full backups, I would like to take >incremental backups. Anyone have played with mysql incremental backup >and Koha? any known issues? Yup -- [probably, depends, no compression] and if you ever "restore" you'll get back all your deletes since the last full backup... We just clean the MySQL db a couple of times a day and dump it -- takes 2104ms for a 754 Meg dump, so there's really no down time at all. Write some sort of [bash] script along the following lines for your Koha user in /usr/share/koha: ./bin/cronjobs/cleanup_database.pl --import 1 && ./bin/cronjobs/cleanup_database.pl --sessions && ./bin/cronjobs/cleanup_database.pl --logs 10 && ./bin/cronjobs/cleanup_database.pl --zebraqueue 1 && mysqldump --user=whomever --password=whatever your_koha_db > /mnt/data7/backup/your_koha_db_dump_[rotate_number].sql Prepend ./bin/cronjobs/cleanup_database.pl --mail -v && if required, vary your variables 'au goût du jour' and bingo... Best -- Paul >thanks in advance > >-- >Indranil Das Gupta > >Phone : +91-98300-20971 >Blog : http://indradg.randomink.org/blog >IRC : indradg on irc://irc.freenode.net >Twitter : indradg > >-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=- >Please exchange editable Office documents only in ODF Format. No other >format is acceptable. Support Open Standards. > >For a free editor supporting ODF, please visit LibreOffice - >http://www.documentfoundation.org >_______________________________________________ >Koha-devel mailing list >Koha-devel(a)lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/

5 8

Getting an IP address for a koha server
by Cajetan Onyeneke 18 Nov '15

18 Nov '15

Please does anyone have idea of how to get or issue dynamic IP address to a koha server running on Linux environment. The server is configured but cannot be accessed within the local network because of this IP issue. Thanks. Cajetan Onyeneke Digital Librarian Imo State University, Owerri, Nigeria

2 1

incremental backup of the DB
by Indranil Das Gupta 17 Nov '15

17 Nov '15

Hi all, I'm facing a situation that between full backups, I would like to take incremental backups. Anyone have played with mysql incremental backup and Koha? any known issues? thanks in advance -- Indranil Das Gupta Phone : +91-98300-20971 Blog : http://indradg.randomink.org/blog IRC : indradg on irc://irc.freenode.net Twitter : indradg -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=- Please exchange editable Office documents only in ODF Format. No other format is acceptable. Support Open Standards. For a free editor supporting ODF, please visit LibreOffice - http://www.documentfoundation.org

1 0

Working around annoying OpacSuppression 114 error in Zebra using special attribute 14
by David Cook 17 Nov '15

17 Nov '15

Hey all: I just opened Bug 15198 (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15198) which contains some instructions for how to make it so that you can turn on OpacSuppression and still retrieve search results, even if you don't have any suppressed records (942$n = 1) in Zebra. I ramble about it below but everything you actually need to know is in the bug report. I would supply a patch myself, but I'm having issues building Perl dependencies for Master, so I won't be contributing any patches until I get that sorted. Cheers, -David -- So I was reading the Zebra docs again (as you do), and I noticed a special attribute type of 14 which could help out with our OpacSuppression issue (whereby you get 0 results - well actually a 114 error - if you don't have any records suppressed but are sending queries checking for suppression). Observe: Z> find @attr 14=1 @not @attr 1=4 test @attr 1=9011 1 Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 8, setno 20 SearchResult-1: term=test cnt=8, term=1 cnt=0 records returned: 0 Elapsed: 0.000678 Z> find @not @attr 1=4 test @attr 1=9011 1 Sent searchRequest. Received SearchResponse. Search was a bloomin' failure. Number of hits: 0, setno 21 Result Set Status: none records returned: 0 Diagnostic message(s) from database: [114] Unsupported Use attribute -- v2 addinfo '9011' Elapsed: 0.000650 Here's the info about attribute type 14 from the Zebra docs (http://www.indexdata.com/zebra/doc/querymodel-zebra.html) Specifies whether un-indexed fields should be ignored. A zero value (default) throws a diagnostic when an un-indexed field is specified. A non-zero value makes it return 0 hits. Cheers to Jesse Weaver for realizing the syntax was @attr 14=1. For those of you who don't read PQF, I'll do it in CCL too: Z> find ignore-empty=(kw=test not Suppress=1) Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 97, setno 2 SearchResult-1: term=test cnt=97, term=1 cnt=0 records returned: 0 Elapsed: 0.000969 Z> find kw=test not Suppress=1 Sent searchRequest. Received SearchResponse. Search was a bloomin' failure. Number of hits: 0, setno 3 Result Set Status: none records returned: 0 Diagnostic message(s) from database: [114] Unsupported Use attribute -- v2 addinfo '9011' Elapsed: 0.000841 *Note that I just added "ignore-empty 14=1" to my ccl.properties to get that ignore-empty=() to work. Actually, the lightest touch for this issue would be to change: Suppress 1=9011 To Suppress 1=9011 14=1 Check that out: Z> find kw=test not Suppress=1 Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 97, setno 1 SearchResult-1: term=test cnt=97, term=1 cnt=0 records returned: 0 Elapsed: 0.005849 David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

2 2

Call for Development News: November Newsletter
by kohanews 14 Nov '15

14 Nov '15

Let the community hear what cool things you're working on! Send me some bugs that need testing, sign-offs and user feedback. k o h a news AT gmail dot com Thank you! -- Chad Roseburg Editor, Koha Newsletter

1 0

Call for News: November Newsletter
by kohanews 14 Nov '15

14 Nov '15

Fellow Koha users ~ I'm collecting news for the November newsletter. Send anything noteworthy to: k o h a news AT gmail dot com News criteria: --------------------------- ** For events **: - Please include dates for past events. If I can't find dates I may not add it. - Announcements for future events with dates T.B.A. are fine ...Eg., Kohacon - For past events -- **** one month back is the cut-off ***. * News items can be of any length. * Images are fine * Anything and everything Koha. * Submit by the 26th of the month. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. -- Chad Roseburg

1 0