[Koha-devel] Fuzzy searching in ICU

[David Cook]

Hey Barton,

 

I didn’t change a configuration file per se. I edited my koha-install-log to change ZEBRA_TOKENIZER from “icu” to “chr” then ran “perl Makefile.PL --prev-install-log /path/to/koha-install-log”, “make”, “make upgrade”. I could’ve tweaked the default.idx file, but then my change would’ve been splatted the next time I did a source build. 

 

“Unsupported truncation attribute” is coming from Zebra. I’m not sure I understand your question about whether it comes from ICU or from queryfuzzy… neither of those has the power to return error messages. The system preference “QueryFuzzy” is used as part of QueryWeighted or whatever it’s called. Basically it hacks the query and adds something like “wrdl,fuzzy=$term”. Fuzzy then gets translated into @attr 5=103 by the ZOOM::CCL2RPN function. 

 

I’ve actually realized that QueryStemming incorrectly uses the ? wildcard (I mentioned that in a ridiculously long listserv email to François at inLibro). But QueryStemming uses the right-Truncation CCL qualifier and ICU supports right-Truncation, so it’s all good on that front.

 

No worries. I’ll add it here as well:  <https://wiki.koha-community.org/wiki/Troubleshooting_Zebra> https://wiki.koha-community.org/wiki/Troubleshooting_Zebra. I think Robin created the page ages ago, and I try to add to it when I can. There’s probably a lot I could still add to it, rather than leaving Zebra information littered through listserv emails…

 

 

David Cook

Systems Librarian

 

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Barton Chittenden [mailto:barton at bywatersolutions.com] 
Sent: Friday, 6 May 2016 12:16 AM
To: David Cook <dcook at prosentient.com.au>
Cc: Koha-devel <koha-devel at lists.koha-community.org>
Subject: Re: [Koha-devel] Fuzzy searching in ICU

 

On Wed, May 4, 2016 at 10:28 PM, David Cook <dcook at prosentient.com.au <mailto:dcook at prosentient.com.au> > wrote:

Hi all,

 

I thought I was going crazy(er) last night when search wasn’t working for me in my Koha dev install...

 

I kept seeing this message in yaz-client:

[120] Unsupported Truncation attribute -- v2 addinfo '103'

 

I started going through my configuration files with a fine-tooth comb, when suddenly I realized that I was using the ICU tokenizer rather than the CHR tokenizer.

 

Sure enough, when I changed that over and re-built Koha, it was working again.

 

Just so I'm clear, which configuration file did you change, and how did you change it?

 

 

I’m sure everyone who uses ICU knows that you can’t use QueryFuzzy searching, but this was a painful reminder.

 

Hm... apparently I'm not a subset of "Everyone e who uses ICU", then. ;-)

 

Is the "Unsupported truncation attribute" coming from ICU, or from queryfuzzy?

 

Also, is QueryStemming affected?

 

I think that I may have been bitten by this recently, although I'm going to have to do some more research to be sure.

 

Thanks for the heads up.

 

--Barton

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20160506/6a27acb8/attachment.html>