Problems with "LOC" authorized values and tokenization
Hi all, I have a library with lots of problematic data in their "LOC" authorized value codes, and it's led me to a few issues. 1. LOC authorized values can contain "*", "(", and ")" but all of these characters will cause the ZOOM library to fail to compile the CCL2RPN query. 2. mc-loc (from the "Shelving location" tab on /cgi-bin/koha/catalogue/search.pl) seems to be doing an incomplete search. That means if you have two locations like "Big Box" and "Big Box - Drawer", "Big Box" will return items for both locations. The reason for #2 is actually kind of interesting. While the RPN/PQF query is "@attr 1=8013 @attr 4=1 apple_street" and @attr 4=1 represents a "phrase" search, Zebra is actually using the "w" register and not the "p" register. It's because of the "Completeness Attributes (type = 6)": "Incomplete subfield (1) is the default, and makes Zebra use register type="w", whereas Complete field (3) triggers search and scan in index type="p"." (https://software.indexdata.com/zebra/doc/querymodel-rpn.html). If we wanted to actually just get the exact location, we'd want to use "mc-loc,complete-field", but then we'd get Zebra errors because mc-loc doesn't use the "p" type register currently, although that would be easy enough to add. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595
On 2019-02-20 7:10 p.m., David Cook wrote: [snip]
2.mc-loc (from the “Shelving location” tab on /cgi-bin/koha/catalogue/search.pl) seems to be doing an incomplete search. That means if you have two locations like “Big Box” and “Big Box – Drawer”, “Big Box” will return items for both locations.
I fully agree -- this problem is getting very close to the top of my list of "things to fix". My analysis (so far) is very close to yours in that it's a question of getting the correct request in, and answer out of, Zebra, but while I am very happy with Zebra, I've never tried (yet) to play with it. Are you suggesting that this would require PERL changes? or can Zebra alone handle modifying a request at input time? Thanks and best regards, Paul
The reason for #2 is actually kind of interesting. While the RPN/PQF query is “@attr 1=8013 @attr 4=1 apple_street” and @attr 4=1 represents a “phrase” search, Zebra is actually using the “w” register and not the “p” register. It’s because of the “Completeness Attributes (type = 6)”:
“Incomplete subfield (1) is the default, and makes Zebra use register type="w", whereas Complete field (3) triggers search and scan in index type="p".” (https://software.indexdata.com/zebra/doc/querymodel-rpn.html).
If we wanted to actually just get the exact location, we’d want to use “mc-loc,complete-field”, but then we’d get Zebra errors because mc-loc doesn’t use the “p” type register currently, although that would be easy enough to add.
David Cook
One solution would be to change "mc-loc" to "mc-loc,ext" (or mc-loc,complete-field) in the Koha template and then to update the Zebra XSLT to index location into the "p" register and do a full re-index (or re-index every record that has items with a location). That would solve the problem. I was thinking of writing a patch, but I have a lot on my plate at the moment. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Paul A Sent: Thursday, 21 February 2019 11:33 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Problems with "LOC" authorized values and tokenization On 2019-02-20 7:10 p.m., David Cook wrote: [snip]
2.mc-loc (from the "Shelving location" tab on /cgi-bin/koha/catalogue/search.pl) seems to be doing an incomplete search. That means if you have two locations like "Big Box" and "Big Box - Drawer", "Big Box" will return items for both locations.
I fully agree -- this problem is getting very close to the top of my list of "things to fix". My analysis (so far) is very close to yours in that it's a question of getting the correct request in, and answer out of, Zebra, but while I am very happy with Zebra, I've never tried (yet) to play with it. Are you suggesting that this would require PERL changes? or can Zebra alone handle modifying a request at input time? Thanks and best regards, Paul
The reason for #2 is actually kind of interesting. While the RPN/PQF query is "@attr 1=8013 @attr 4=1 apple_street" and @attr 4=1 represents a "phrase" search, Zebra is actually using the "w" register and not the "p" register. It's because of the "Completeness Attributes
(type = 6)":
"Incomplete subfield (1) is the default, and makes Zebra use register type="w", whereas Complete field (3) triggers search and scan in index type="p"." (https://software.indexdata.com/zebra/doc/querymodel-rpn.html).
If we wanted to actually just get the exact location, we'd want to use "mc-loc,complete-field", but then we'd get Zebra errors because mc-loc doesn't use the "p" type register currently, although that would be easy enough to add.
David Cook
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi, Authorized values works really better if you stick to [0-9A-Z] in values. I know its not always easy ;) Le 21/02/2019 à 01:10, David Cook a écrit :
Hi all,
I have a library with lots of problematic data in their "LOC" authorized value codes, and it's led me to a few issues.
1. LOC authorized values can contain "*", "(", and ")" but all of these characters will cause the ZOOM library to fail to compile the CCL2RPN query.
2. mc-loc (from the "Shelving location" tab on /cgi-bin/koha/catalogue/search.pl) seems to be doing an incomplete search. That means if you have two locations like "Big Box" and "Big Box - Drawer", "Big Box" will return items for both locations.
The reason for #2 is actually kind of interesting. While the RPN/PQF query is "@attr 1=8013 @attr 4=1 apple_street" and @attr 4=1 represents a "phrase" search, Zebra is actually using the "w" register and not the "p" register. It's because of the "Completeness Attributes (type = 6)":
"Incomplete subfield (1) is the default, and makes Zebra use register type="w", whereas Complete field (3) triggers search and scan in index type="p"." (https://software.indexdata.com/zebra/doc/querymodel-rpn.html).
If we wanted to actually just get the exact location, we'd want to use "mc-loc,complete-field", but then we'd get Zebra errors because mc-loc doesn't use the "p" type register currently, although that would be easy enough to add.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> BibLibre, France - software and system maintainer
I totally agree. Unfortunately, it's difficult to control hundreds of librarians by words alone. I've been thinking perhaps a good way forward would be some Javascript to prevent people from adding anything non-alphanumeric. It wouldn't stop vendors from doing problematic data migrations but it would prevent library staff from making it worse? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of Fridolin SOMERS Sent: Tuesday, 26 February 2019 7:49 PM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Problems with "LOC" authorized values and tokenization Hi, Authorized values works really better if you stick to [0-9A-Z] in values. I know its not always easy ;) Le 21/02/2019 à 01:10, David Cook a écrit :
Hi all,
I have a library with lots of problematic data in their "LOC" authorized value codes, and it's led me to a few issues.
1. LOC authorized values can contain "*", "(", and ")" but all of these characters will cause the ZOOM library to fail to compile the CCL2RPN query.
2. mc-loc (from the "Shelving location" tab on /cgi-bin/koha/catalogue/search.pl) seems to be doing an incomplete search. That means if you have two locations like "Big Box" and "Big Box - Drawer", "Big Box" will return items for both locations.
The reason for #2 is actually kind of interesting. While the RPN/PQF query is "@attr 1=8013 @attr 4=1 apple_street" and @attr 4=1 represents a "phrase" search, Zebra is actually using the "w" register and not the "p" register. It's because of the "Completeness Attributes (type = 6)":
"Incomplete subfield (1) is the default, and makes Zebra use register type="w", whereas Complete field (3) triggers search and scan in index type="p"." (https://software.indexdata.com/zebra/doc/querymodel-rpn.html).
If we wanted to actually just get the exact location, we'd want to use "mc-loc,complete-field", but then we'd get Zebra errors because mc-loc doesn't use the "p" type register currently, although that would be easy enough to add.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Direct: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> BibLibre, France - software and system maintainer _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
participants (3)
-
David Cook -
Fridolin SOMERS -
Paul A