https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion --- Comment #1 from David Cook <dcook@prosentient.com.au> --- However, there is a way. If you comment out all our current truncation detection code and do the following... if ($auto_truncation){ - unless ( $index =~ /,(st-|phr|ext)/ ) { + unless ( $index =~ /,(st-|ext)/ ) { + my $trunc_count = ($operand =~ s/\Q*\E/#/g); + if ($trunc_count){ + $index .= ",process-in-search-term"; + } Zebra does support # as a mask/wildcard in lieu of *. It does mean it would be wise to s/#//g the operand before doing truncation handling, so that "C# Program*" doesn't match "CGI Programming". But that wouldn't be that hard to do. The # would work with both left and right truncation. -- Anyway, food for thought. At this point, I don't think that we're going to pursue this change. Something like 99% of our libraries are using Elasticsearch now. But it's an idea... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Truncation doesn't work |Truncation doesn't work as |with Zebra phrase searches |expected with Zebra phrase | |searches -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- (In reply to Fridolin Somers from comment #2)

...

Zebra doesn't support * for doing a masked/wildcard search

Strange I never noticed this. Looks correct indeed in CCL doc : https://software.indexdata.com/yaz/doc/tools.html#ccl.syntax

I hadn't thought about the CCL syntax specifically. That's interesting. "2.4.4. Truncation Attributes (type = 5)" on https://software.indexdata.com/zebra/doc/querymodel-rpn.html Our Koha CCL qualifier "process-in-search-term" maps to "Process # in search term" for the RPN truncation. I couldn't get that CCL syntax to work but the RPN one does. In CCL, the # is a mask for one character whereas in the RPN it's translated into .* and then does a regexp search. Although as I say that... I wonder if that means other characters in the query that could be a regular expression character would also be interpreted. Probably.

Not sure where to start on this complex topic

Yeah, that's why I'm thinking of not pursuing this one. Zebra is on the way out, and changing a search fundamental seems... problematic, even if it could be an improvement. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #4)

curl -u <username:password> -X POST "<ES server>" -H 'Content-Type: application/json' -d '{"query": {"query_string": {"query": "handb*","analyze_wildcard": true}}}'

Sorry that's the non-phrase which does work and gets 4 results. This is the phrase query which gets 0 results: -d '{"query": {"query_string": {"query": "\"handb*\"","analyze_wildcard": true}}}' This phrase query will get 3 results though: -d '{"query": {"query_string": {"query": "\"handbook\"","analyze_wildcard": true}}}' -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #6 from David Cook <dcook@prosentient.com.au> --- I'm thinking that maybe we just add a warning to the syspref "QueryAutoTruncate" saying that truncation doesn't work for phrase searches for either Zebra or Elasticsearch. (Of course, we should probably also remove the "X as phrase" advanced search options when using Elasticsearch since they don't actually do anything...) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #8 from David Cook <dcook@prosentient.com.au> --- Of course, allowing regular expressions also comes with peril. So that's fun too. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #9)

(In reply to David Cook from comment #6)

...

I'm thinking that maybe we just add a warning to the syspref "QueryAutoTruncate" saying that truncation doesn't work for phrase searches for either Zebra or Elasticsearch.

(Of course, we should probably also remove the "X as phrase" advanced search options when using Elasticsearch since they don't actually do anything...)

Are you saying phrase search doesn't work at all or only not working as expected with QueryAutoTruncate?

It's complicated haha. Zebra: If you use "phr" with * but don't use double-quotes, Koha mangles the query and separates the phrase into individual terms that are searched against the "phrase" register. Technically truncation does apply to the terms, but it's essentially broken the phrase search, so it's not really a phrase search anymore. If you use "phr" with * and use double quotes, Koha keeps it as a phrase search, but the truncation won't work. Elasticsearch: "phr" just gets erased from the query, so it's a useless adding it to a query in Koha. Elasticsearch can only use * with individual terms. If you use double quotes, which is ES's Query String Syntax for a phrase search, it will silently ignore the * and won't perform the truncation. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38749 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #11)

It sure sounds complicated.

Would it make sense then to ignore * on phrase searches as that would at least phrase search?

For Zebra, that could make sense. For Elastic, I wonder if we should be trying to enforce double quotes when the phr index is used... more thinking to do one that one. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.