[Bug 16581] New: ICU tokenization bug in idzebra-2.0 2.0.59-1
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Bug ID: 16581 Summary: ICU tokenization bug in idzebra-2.0 2.0.59-1 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: barton@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org The version of idzebra-2.0 idzebra-2.0 2.0.59-1 which is the version in the debian-jesse repository, has a known ICU-Tokenization bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777515 This has been seen to cause search errors with, for instance, the Chinese language search for 心意更新. Searches for the string of all four characters, without spaces, will give hundreds of hits, whereas separating the ideograms with spaces will limit the search to 8 search results, as intended. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Hi Barton, I was the one who found that bug in Zebra last year, and while I got Indexdata to fix it in 2.0.60 (http://www.indexdata.com/zebra/doc/NEWS https://github.com/indexdata/idzebra/commit/704fd190292cb771df94553b0ed6f9f4...), I'm not sure that's the same issue that you're having. Have you tested your query with newer versions of Zebra (2.0.60+) to see if the issue still exists in newer versions? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Robin and I mentioned this bug between Feb-May 2015 but no one really seemed to care at the time (http://permalink.gmane.org/gmane.education.libraries.koha.devel/11966). I don't know if that means that people are using the Indexdata repositories or if they're just ignoring erroneous behaviour in Zebra when using ICU... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Jacek Ablewicz <abl@biblos.pk.edu.pl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=16921 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtompset@hotmail.com --- Comment #3 from M. Tompsett <mtompset@hotmail.com> --- (In reply to David Cook from comment #2)
I don't know if that means that people are using the Indexdata repositories or if they're just ignoring erroneous behaviour in Zebra when using ICU...
Probably the latter. :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Eric Bégin <eric.begin@inLibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eric.begin@inLibro.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |WONTFIX CC| |z.tajoli@cineca.it --- Comment #4 from Zeno Tajoli <z.tajoli@cineca.it> --- In fact I use ICU on jessie and I install Zebra from Indexdata repositories. I think it is the only option avaible. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |http://ftp.indexdata.dk/pub | |/zebra/debian/README --- Comment #5 from M. Tompsett <mtompset@hotmail.com> --- I've added a link to the IndexData Debian README file into the URL field above. The Ubuntu one is here: http://ftp.indexdata.dk/pub/zebra/ubuntu/README -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 --- Comment #6 from David Cook <dcook@prosentient.com.au> --- (In reply to Zeno Tajoli from comment #4)
In fact I use ICU on jessie and I install Zebra from Indexdata repositories. I think it is the only option avaible.
Agreed. I don't know the Koha community's official position on the subject, but I think we should be advocating the use of IndexData's repositories or providing a fully functional Zebra package ourselves. Relying on the Debian repository is clearly not a good option :/. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Barton Chittenden <barton@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|WONTFIX |--- Status|RESOLVED |REOPENED --- Comment #7 from Barton Chittenden <barton@bywatersolutions.com> --- 09:34 < barton> I'm looking at bug 16581... I think there's still a number of un-answered questions there... 09:34 < huginn> Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 enhancement, P5 - low, ---, gmcharlt, RESOLVED WONTFIX, ICU tokenization bug in idzebra-2.0 2.0.59-1 09:35 < marcelr> barton: i did not understand why it was closed either 09:36 < barton> so here are the things that I'm wondering about: 1) is it possible to require the use of the indexdata repo? 2) what version of idzebra-2.0 *should* we be using? 09:37 < mtompset> 1) why not? It's a good idea, 2) the current one -- I think something 62? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |dependency CC| |katrin.fischer@bsz-bw.de, | |mirko@abunchofthings.net --- Comment #8 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Adding Mirko to the discussion. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- Overall, I'm in favour of using Indexdata's repository. As Mark pointed out, 2.0.62 appears to be the latest version: http://www.indexdata.com/zebra/doc/NEWS You'll notice that 2.0.60 is where the tokenization bug was fixed: Fix ICU phrase searches for terms split by ICU ZEB-664 You'll probably notice a big gap between 2.0.59 and 2.0.60. My emails to Indexdata prompted the fix for 2.0.60. You may also notice "Allow @attr 2=102 for numeric index" in 2.0.62. I had noticed that relevance ranking didn't work with a query containing a numeric index, so Indexdata fixed it. I don't think it really affects the ranking much per se, but it prevents the Zebra error. Note also that you an now browse Zebra's code on Github: https://github.com/indexdata/idzebra I don't know if they accept pull requests, but I wouldn't be surprised if they did. Of course, I don't know if Indexdata will be doing much work on Zebra going forward. I don't think they're answering the Zebra-list anymore: http://lists.indexdata.dk/pipermail/zebralist/2016-May/002645.html I think they're focusing their efforts on the FOLIO project these days. Tomas mentioned some time ago that Indexdata considered Zebra a legacy project, and that seems to be the case. All the more reason to move to Elastic, I suppose, although I see the utility of keeping Zebra around if only as a Z39.50 server. Makes you wonder if we should be thinking about turning Zebra into an optional component once Elastic takes over fully... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #10 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to David Cook from comment #9)
Overall, I'm in favour of using Indexdata's repository.
We at Biblibre are using this repo with Ubuntu Xenial since Koha 16.11. Its Zebra 2.1.3 now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Searching |Packaging -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=12430 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- We should probably update debian/control to require a newer version of Zebra since we know that 2.0.59 is buggy... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |WONTFIX Status|REOPENED |RESOLVED --- Comment #12 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #11)
We should probably update debian/control to require a newer version of Zebra since we know that 2.0.59 is buggy...
It looks like Debian 11 Bullseye finally updated Zebra to 2.2.2 which alleviates this problem... I'm tired of talking about 2.0.59 being buggy, so I'm going to close it. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org