[Bug 12430] New: Truncation disables relevance ranking (when not using QueryParser)
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 Bug ID: 12430 Summary: Truncation disables relevance ranking (when not using QueryParser) Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: dcook@prosentient.com.au QA Contact: testopia@bugs.koha-community.org Scenario: 1) NOT using QueryParser 2) Using truncation (either by explicitly using * in the query or when 'QueryAutoTruncate' is set to 'automatically') Result: 1) Search results ARE ordered by biblionumber ascending and NOT by relevance -- This appears not to be an issue when using QueryParser as it seems to parse queries much better than the traditional...method. However, quite a major issue for those not using QueryParser, as everyone likes relevant searches. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Truncation disables |Truncation disables |relevance ranking (when not |relevance ranking (and has |using QueryParser) |issues when using | |QueryParser) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- So the QueryParser actually acts a bit oddly as well. In some cases, it will add the relevance attribute to a query with a couple weighted fields, as defined in this section of queryparser.yaml: relevance_bumps: keyword: titleext: enabled: 1 bib1_mapping: biblioserver: 34 titlekw: enabled: 1 bib1_mapping: biblioserver: 20 If you type in "test" into the OPAC as a "Library catalog" search with QueryAutoTruncate (or as test*), QueryParser will generate something like this: @or @or @attr 1=1016 @attr 5=1 @attr 4=6 "test" @attr 9=20 @attr 2=102 @attr 5=1 @attr 4=6 "test" @attr 9=34 @attr 2=102 @attr 5=1 @attr 4=6 "test" Let me re-organize that a bit so it looks a bit more readable by humans... @attr 1=1016 @attr 5=1 @attr 4=6 "test" @or @attr 9=20 @attr 2=102 @attr 5=1 @attr 4=6 "test" @or @attr 9=34 @attr 2=102 @attr 5=1 @attr 4=6 "test" Basically, it says it's doing a keyword (@attr 1=1016) wordlist (@attr 4=6) search for a right truncated (@attr 5=1) "test" in 3 branches of this query. In two branches, it's looking for relevance (@attr 2=102) with weighted fields (@attr 9=20) and (@attr 9=34) where the former is worth fewer points than the latter in the ranking. Source: http://www.indexdata.com/zebra/doc/administration-ranking.html -- If you do a "Title" search for "test", you'll get the following: @attr 1=4 @attr 5=1 @attr 4=6 "test" (@attr 5=1 means 'right truncate', @attr 4=6 means 'wordlist', and @attr 1=4 means 'title') Notice that we no longer have relevance ranking (via @attr 2=102) and no weighted fields (@attr 9=VALUE) -- So...it seems to me that relevance only works with QueryParser for keywords...and even then in an unexpected way... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- There's a bit of odd code in Koha\QueryParser\Driver\PQF\query_plan\node.pm, which is where a lot of the action happens in QP... When doing a "Library Catalog" search, it tries to use the following code to create a relevance query branch: $fieldobj = $self->plan->QueryParser->bib1_mapping_by_name('field', $relbump->{'classname'}, $relbump->{'field'}, $server); This means $fieldobj, which should contain our CCL2RPN type mapping, will always be null because $relbump->{'classname'} and $relbump->{'field'} don't exist. This is why the 2nd line in the following code doesn't have an @attr 1=1016 in it. @attr 1=1016 @attr 5=1 @attr 4=6 "test" @or @attr 9=20 @attr 2=102 @attr 5=1 @attr 4=6 "test" But we don't notice the error in the OPAC, as I think Zebra/Yaz/Zoom/whatever silently treats the second branch as having @attr 1=1016 as a bare search will usually be treated as a keyword (ie @attr 1=1016). -- Here is another odd bit of code, which is responsible for QueryParser's current "support" for relevance. my $relbumps = $self->plan->QueryParser->bib1_mapping_by_name('relevance_bump', $self->classname, '', $server); This line will return ALL the relevance_bump mappings under the keyword class...which are 'titleext' and 'titlekw'. I think these may have been example mappings...as it doesn't really make sense to have them under a keyword class search. It should be a "keyword" class search with a "keyword" field rather than a "titleext" field, in my opinion. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- In any case, you can add relevance mappings to QueryParser by adding additional config in queryparser.yaml. For instance: title: title: enabled: 1 bib1_mapping: biblioserver: 30 This will add relevance ranking to a title search with a field weight of 30. However, it's worth noting that QueryParser seems to often fall back to non-QueryParser mode for more complicated indexes like "ti,phr", so...I wouldn't go out and make a lot of QP mappings just yet... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Truncation disables |Truncation disables |relevance ranking (and has |relevance ranking when not |issues when using |using QueryParser |QueryParser) | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- Created attachment 28880 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28880&action=edit Bug 12430 - Truncation disables relevance ranking when not using QueryParser The system preference "QueryWeightFields" is mutually exclusive with the system preference "QueryAutoTruncate" and the * truncation modifier, when not using QueryParser. If you use truncation, relevance won't work anymore. (N.B. Relevance doesn't work probably when using QueryParser, but for a very different reason beyond the current scope of this bug.) This patch adds relevance ranking when using truncation (or basically when QueryWeightFields is disabled). _TEST PLAN_ 1) Turn on "QueryWeightFields" and "QueryAutoTruncate" 2) Turn off "TryQueryParser" 3) Do a keyword search in the OPAC 4) Note that the results are ordered by biblionumber ascending 5) Apply patch 6) Do the same keyword search in the OPAC 7) Note that the results are never ordered differently (presumably in relevance order :P) If you're a stickler, throw in some warns so that you can see what CCL query is being sent to Zebra. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |normal --- Comment #5 from David Cook <dcook@prosentient.com.au> --- Changing the importance of this one as I suppose its severity is debateable... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #6 from David Cook <dcook@prosentient.com.au> --- It appears that this patch could break the "Publication Date (yyyy)" search, when using "yr,st-numeric" (which might be the case after bug 9368). If we were to use "yr,st-year" instead, you might get results for a "2000-" query but they would only be for "2000". If you specified a full range "2000-2012", it would break. It seems that relevance can't be run with date ranges (which is usually fine, as we use date ranges as limits which are not affected by relevance ranking). -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- Putting this on hold for the moment while I try to get some refactoring done on "buildQuery". We shouldn't be adding relevance searching to date ranges (it will cause errors that produce no results or results only matching the first date in the range) or item-level indexes (as the relevance score will be artificially inflated by bib records with higher numbers of items). -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |12742 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12430 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|12742 | Depends on| |12742 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org