[Bug 13751] New: "Begins with" search not possible with ICU indexing
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 Bug ID: 13751 Summary: "Begins with" search not possible with ICU indexing Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: katrin.fischer@bsz-bw.de QA Contact: testopia@bugs.koha-community.org With ICU activated, a "begins with" search like ti,ext= will return the same results as a ti= search. This is a problem for libraries with big collections where a title you look for can be pretty unspecific and you need a way to do more exact searches. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hm, I have to correct: ti= gives more results than ti,ext= But: results don't match the expected result. I am not really sure what is happening there. On a non-ICU installation something like: ti,ext=Pädagogik Works perfectly, all titles shown start with Pädagogik. The manual: http://manual.koha-community.org/3.18/en/searchguide.html#searchguide-advanc... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=6425 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=11158 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #2 from David Cook <dcook@prosentient.com.au> --- It would be interesting to look at this using yaz-client... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I'd be happy to help testing - just not sure how to do it. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #3)
I'd be happy to help testing - just not sure how to do it.
1) Find the unix socket for your zebra database Find "koha-conf.xml" and get the value from <listen id="biblioserver" > e.g. "unix:/blah/blah/zebradb/bibliosocket" 2) Connect to the socket using yaz-client "yaz-client unix:/blah/blah/zebradb/bibliosocket" 3) Select the correct database "base biblios" 3) Run some queries "find @attr 1=4 Pädagogik" "find @attr 1=4 @attr 4=1 @attr 6=3 Pädagogik" @attr 1=4 @attr 4=1 @attr 6=3 is the PQF equivalent of the "ti,ext" CCL, while @attr 1=4 is the same as "ti". 4) Compare the results to see if they return the same results or not -- This should show whether or not these queries are returning the same results. If they are returning the same results, it's a bug in Zebra. If they aren't returning the same results using yaz-client, but they are when using Koha, then it's down to the "extra stuff" that we do to queries before sending them off to Zebra. In that case, you'd need to add some debugging code to see the query before it gets sent off to Zebra. This is probably best done in search.pl or opac-search.pl. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- For what it's worth, I just tried: Z> base biblios Z> find @attr 1=4 test Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 5, setno 4 SearchResult-1: term=test cnt=5 records returned: 0 Elapsed: 0.000979 Z> find @attr 1=4 @attr 4=1 @attr 6=3 test Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 0, setno 5 SearchResult-1: term=test cnt=0 records returned: 0 Elapsed: 0.000381 This is using Zebra 2.0.60 from IndexData's Debian repository. -- I just tried in the staff client of Koha 3.8 and got the same results as I did in Zebra. I've tried changing the QueryAutoTruncate, QueryStemming, and QueryWeightFields system preferences to see if they made a difference, but there was no difference. -- Maybe I just don't have the right test string though. What do you mean when you say that "results don't match the expected result"? Can you share the URL of the site where you're experiencing the problem? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi David, I am not sure how to interpret your test results. The problem I see is that the search seems to work... but the records titles DON'T BEGIN all with with the search term I used ("Pädagogik..."), when ICU is enabled. I our other installations using chr indexing, this works correctly. It should not depend on the search term. It would be interesting to see the titles for your results maybe? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #6)
Hi David,
I am not sure how to interpret your test results. The problem I see is that the search seems to work... but the records titles DON'T BEGIN all with with the search term I used ("Pädagogik..."), when ICU is enabled. I our other installations using chr indexing, this works correctly. It should not depend on the search term.
It would be interesting to see the titles for your results maybe?
Ah, I think I wasn't understanding your original question. My test was just to show that a "ti" and a "ti,ext" search weren't returning the same results. In my example, the former returned 5 results while the latter returned 0. I'll have to look more at this another time... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |abl@biblos.pk.edu.pl -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #8 from Jacek Ablewicz <abl@biblos.pk.edu.pl> --- (In reply to Katrin Fischer from comment #1)
Hm, I have to correct:
ti= gives more results than ti,ext= But: results don't match the expected result. I am not really sure what is happening there.
On a non-ICU installation something like: ti,ext=Pädagogik Works perfectly, all titles shown start with Pädagogik.
I guess Zebra version being used may be somehow relevant. Such searches were indeed not working correctly w/ ICU in some older versions (like 2.0.44, shipped by default with Debian Wheezy). AFAIRC, in particular 'ext' (4=1 6=3) was affected by this, but also other bib-1 structure, position and trucation attributes (or maybe just some specific combinations of thereof ?) were not yet properly supported in 2.0.44 with ICU enabled. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi Jacek, thx for looking into this. Do you know if it works for newer versions and if it's safe to use a newer version? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #9)
Hi Jacek, thx for looking into this. Do you know if it works for newer versions and if it's safe to use a newer version?
I was using 2.0.60, which seemed to work fine, but I'm not sure I was testing the issue correctly... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- Suppose it's worth mentioning that the upcoming versions of Debian/Ubuntu will ship with Zebra 2.0.59, which has a few issues as noted by myself and Tomas, I believe. 2.0.59 has a fairly major problem with ICU where strings containing hyphens aren't tokenized properly. That is to say, when the strings are tokenized, only the first token is used and everything after it is discarded. This was fixed in Zebra 2.0.60, but I think OpenSuse is the only Linux distro which has upgraded its packages to 2.0.60... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 Than H. S. (Punsarn.asia) <thanhtutsoetgi@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |thanhtutsoetgi@gmail.com --- Comment #12 from Than H. S. (Punsarn.asia) <thanhtutsoetgi@gmail.com> --- I have tried upgrading to latest Zebra version available (2.0.61). I have also performed forced re-building of the index after upgrade. It doesn't seem to fix this bug at all. Does anyone has some leads on how to fix this problem? It affects not only "Begins with" search but it also affects "Is exactly" search as well. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org