May 2016 - Koha-devel - lists.koha-community.org

Fuzzy searching in ICU
by David Cook 06 May '16

06 May '16

Hi all, I thought I was going crazy(er) last night when search wasn't working for me in my Koha dev install... I kept seeing this message in yaz-client: [120] Unsupported Truncation attribute -- v2 addinfo '103' I started going through my configuration files with a fine-tooth comb, when suddenly I realized that I was using the ICU tokenizer rather than the CHR tokenizer. Sure enough, when I changed that over and re-built Koha, it was working again. I'm sure everyone who uses ICU knows that you can't use QueryFuzzy searching, but this was a painful reminder. Perhaps we should add a warning to the text for QueryFuzzy in the system preferences, or maybe tie use of QueryFuzzy to the "UseICU" system preference, although I don't know if anyone actually uses that preference. I don't know if mine was turned on or off. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Office: 02 9212 0899 Direct: 02 8005 0595

3 5

Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago
by David Cook 05 May '16

05 May '16

Hey Barton, Yes, the year 2038 for timestamps is a problem even on 64 bit machines. Here’s a test you can do on a 64 bit MySQL install on a 64 bit OS: select unix_timestamp('2038-01-20'); select from_unixtime(2548990800); Both will return null. You can also try storing a value of ‘2039-01-01’ into a timestamp column, and you’ll get the following message : Out of range value for column 'timestamp' at row 1 Here’s some good explanations: <http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bi…> http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bi… <https://bugs.mysql.com/bug.php?id=12654> https://bugs.mysql.com/bug.php?id=12654 Also, I’d never heard of date_t before your email, and the only references I can see to it are for specific projects which have implemented it as a new data structure themselves. I think time_t is the standard and that depends on your system architecture, although as you can see in those above links – it’s more complicated than that in MySQL. -- issues.date_due is already a DATETIME data type, so we don’t need to worry about 2038. As for time zones, why would it be tricky storing it in UTC? At the moment, you can only have due dates in one time zone. That means you can’t have library systems spanning more than one timezone. Maybe that’s a rare or uncommon scenario, but here in the state of New South Wales, we actually do have 2 different timezones. In that case, when you’re doing a check out, you choose a local time, convert it to UTC for storage, and then render it in local time for the library user and library staff. I imagine the trickiest part would be the Reports module, as users wouldn’t know to use something like the CONVERT_TZ() function. Calculating overdues against holidays might be tricky… although that’s a case of just getting your timezone right as well. I reckon your timezone should be set per library, so you just fetch your UTC datetime, convert it to your local library timezone, probably convert it to floating to do math, and then convert it back to local timezone, and then whatever else you need. It sounds complex, but it’s really not that bad when you have it structured well. -- Barton, I’m not sure I understand your reaction :p. I also field support tickets for a living, and I think it would be a lot easier to field tickets for a re-architected system that is created off years of experience rather than years of technical debt. Of course, I know re-architecting a system is also a lot easier said than done. But I think that’s why you see folks like EBSCO thinking about creating a microservice based system which can integrate with Koha. Incremental change. I suppose dates are most important when it comes to circulation and patrons… and I think that’s what people probably think about when they think of core Koha. I suppose acquisitions and serials require a lot of careful date handling as well… I was actually thinking about Koha and “asset tracking” in general this morning on the way to the office. I don’t know. Maybe Koha is fine just being a 1 timezone system. I imagine library systems spanning over multiple timezones are probably the exception rather than the rule. We need to re-think some of our DateTime logic though. Whether we use ‘floating’ timezones more consistently, or are careful with creating local times that don’t exist… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Office: 02 9212 0899 Direct: 02 8005 0595 From: Barton Chittenden [mailto:barton@bywatersolutions.com] Sent: Thursday, 5 May 2016 12:04 AM To: David Cook <dcook(a)prosentient.com.au> Subject: Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago On Thu, Apr 28, 2016 at 7:27 PM, David Cook <dcook(a)prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: I don’t think using “timestamp” would be preferable to “datetime”. First, MySQL’s timestamp seems to suffer from the Year 2038 problem ( <https://en.wikipedia.org/wiki/Year_2038_problem> https://en.wikipedia.org/wiki/Year_2038_problem)(http://bugs.mysql.com/bug.…) while datetime does not. Even on 64 bit machines? Oh MySQL, MySQL, knowest thou not of date_t? Second, you’d be handling timezones at the data level rather than at the application level. I’d rather have a convention of storing DATETIME as UTC, having Koha fetch that DATETIME as UTC, and then doing timezone conversions from there. I think that could be handled *relatively* transparently in DateUtils.pm, but there are a lot of corner cases like due dates that get ... tricky. As for the same server running different time zones, you can change the timezone on a per connection setting I think, but that could be a hassle. I think it would be easier just to store things clearly as UTC. This is done on a different project I work on, and it works well. Oh, I think it’ll be a nightmare changing timezone/timestamp/datetime handling in Koha. I don’t even know if it’s possible, to be honest. Well, it would be possible, but it would likely be an error-prone process. It’s things like this which make me think that we need to start thinking about a version of Koha that doesn’t have a clear upgrade path. A Koha with some major architectural changes which will need a skilled hand to migrate to. It won’t be nice for a lot of people… but I don’t know how else we’re going to make some major changes without dramas. >From a design standpoint, I agree. >From the point of view of someone who fields support tickets for a living: NOOOOOOOOOOOOOOOOOOOOO!!!! ... but that's just an emotional response. Cheers, --Barton

3 4

Re: [Koha-devel] Searching with the question mark wildcard (Part two)
by David Cook 05 May '16

05 May '16

My last email may have been a bit over the top, although I still stand by the fact that Search.pm butchers queries before they get to Zebra. Hopefully, the integration of ElasticSearch will make a re-write of the Zebra-related code to be unnecessary. While we’re still planning to hold on to Zebra, I think it’s mostly as a Z39.50/SRU server? I actually quite like Zebra as a search engine, but the way the search code in Koha was implemented makes it impossible to use Zebra as it should be used. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces(a)lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Francois Charbonnier Sent: Wednesday, 4 May 2016 11:00 PM To: koha-devel(a)lists.koha-community.org >> koha-devel <koha-devel(a)lists.koha-community.org> Subject: [Koha-devel] Searching with the question mark wildcard Hello, Has anyone tried to implement the question mark (?) wildcard ? The documentation says "Two of the wildcard characters that can be used in Koha searches are the asterisk ('*') and the percent sign ('%'). However, these two characters act differently when used in searching." (http://manual.koha-community.org/3.22/en/searchfaq.html) But, CQL also supports the question mark wildcard "? : Matches any single character, so that c?t will match any of the words cat, cot or cut, but not coat or indeed ct." (http://zing.z3950.org/cql/intro.html#6) We are currently evaluating this implementation. If you already tried, your feebacks will be appreciated. And, any advice on how you'll do it will be welcome as well! Thanks! -- François Charbonnier, Bibl. prof. / Chef de produits Tél. : (888) 604-2627 <mailto:francois.charbonnier@inLibro.com> francois.charbonnier(a)inLibro.com inLibro | pour esprit libre | <http://www.inLibro.com> www.inLibro.com

1 0

Hackfest in Marseille in October
by Paul Poulain 04 May '16

04 May '16

Hello koha-developers, For the 5th year, BibLibre is happy to welcome in Marseille anyone wanting to hack Koha. Because the KohaCon is in June, the Marseille hackfest has been switched to fall. We are waiting for you in October, 10th - 14th ! Some people have already told they'll be here. Kohala, the french NPO, and BibLibre have decided to sponsor Owen, so he'll be with us. Thanks to NPL for letting him come ! I also know that our loved RM will be here too. If you plan to join us, just drop me an email ! PS: for those who came previous years: we have a new NPO as neighbors. They've a large meeting room, that can host 15+ ppl we already have booked, plus a small one with sofas. So we have space for 70+ persons ! -- Paul Poulain, Associé-gérant / co-owner BibLibre, Services en logiciels libres pour les bibliothèques BibLibre, Open Source software and services for libraries

1 0

Dev a app with android studio
by dave_meyer＠web.de 04 May '16

04 May '16

1 0

Searching with the question mark wildcard
by Francois Charbonnier 04 May '16

04 May '16

Hello, Has anyone tried to implement the question mark (?) wildcard ? The documentation says "Two of the wildcard characters that can be used in Koha searches are the asterisk ('*') and the percent sign ('%'). However, these two characters act differently when used in searching." (http://manual.koha-community.org/3.22/en/searchfaq.html) But, CQL also supports the question mark wildcard "? : Matches any single character, so that c?t will match any of the words cat, cot or cut, but /not/ coat or indeed ct." (http://zing.z3950.org/cql/intro.html#6) We are currently evaluating this implementation. If you already tried, your feebacks will be appreciated. And, any advice on how you'll do it will be welcome as well! Thanks! -- François Charbonnier, Bibl. prof. / Chef de produits Tél. : (888) 604-2627 francois.charbonnier(a)inLibro.com <mailto:francois.charbonnier@inLibro.com> inLibro | pour esprit libre | www.inLibro.com <http://www.inLibro.com>

1 0

Password stored in borrowers
by yash verma 03 May '16

03 May '16

I need to retrieve the data from the borrowers table, but the password is encrypted. How do I decrypt it in MySQL?

3 4

Test Suite
by Marcel de Rooy 02 May '16

02 May '16

Thanks for some extensive pushing, Kyle and Brendan. We still have some tests failing. I resolved a few trivial ones: bug 16403, 16404, 16405, 16407 (hardcoded borrowers), 16408 16406 Context.t is a harder one (it should not fail only for not having a cache). I also tested on top of bug 16155. Note that we probably still need some DBIx updates ! With DBIx update and some of the above fixes all TestBuilder related tests pass: All tests successful. Files=69, Tests=1929, 85 wallclock secs ( 0.53 usr 0.13 sys + 75.33 cusr 5.92 csys = 81.91 CPU) Result: PASS And finally what about: Can't locate Catmandu/Importer/MARC.pm Can't locate object method "_convert_marc_to_json" Can't locate Catmandu/Store/ElasticSearch.pm Marcel

2 1