June 2021 - Koha-bugs - lists.koha-community.org

[Bug 15400] Display patron age in useful places in the staff interface
by bugzilla-daemon＠bugs.koha-community.org 17 Jun '21

17 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15400 --- Comment #90 from Michal Denar <black23(a)gmail.com> --- I don't think that displaying age and date of birth in the interface for librarians is privacy issue or against GDPR rules. It depends on how the library has the area covered in their documents. Our libraries use the age information in the normal course of business in some cases to verify the reader if they don't have a library card, etc. Certainly this feature can be optional, but it should be available. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15400] Display patron age in useful places in the staff interface
by bugzilla-daemon＠bugs.koha-community.org 17 Jun '21

17 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15400 --- Comment #89 from Michal Denar <black23(a)gmail.com> --- I don't think that displaying age and date of birth in the interface for librarians is in privacy with security and GDPR. It depends on how the library has the area covered in their documents. Our libraries use the age information in the normal course of business in some cases to verify the reader if they don't have a library card, etc. Certainly this feature can be optional, but it should be available. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 15400] Display patron age in useful places in the staff interface
by bugzilla-daemon＠bugs.koha-community.org 17 Jun '21

17 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15400 --- Comment #88 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- This is privacy issue too. GDPR etc. Note that we (as a special library) removed all birthdates from our system. Hopefully we dont have lots of uninitialized warnings too? -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #11 from David Cook <dcook(a)prosentient.com.au> --- Of course, some characters also have special significance in the CCL query language as well, so even getting them to Zebra would be part of the first step. Not a simple problem by any stretch. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #10 from David Cook <dcook(a)prosentient.com.au> --- Oh I've had some fun playing with ICU... chain.xml: <icu_chain locale=""> <tokenize rule="l"/> <transliterate rule="[:Punctuation:] } [:WhiteSpace:] > ''"/> <transform rule="[:WhiteSpace:] Remove "/> <display/> <casemap rule="l"/> </icu_chain> echo -n '.NET. test' | yaz-icu -c chain.xml 1 1 '.net'' '.NET'' 2 1 'test' 'test' -- Here we tokenize based on the line break (ie space), and then we perform our transliteerate and transform rules as per http://userguide.icu-project.org/transforms/general. With the transliterate, we can use the following syntax: "before_context { text_to_replace } after_context > completed_result | result_to_revisit ;" So here the "text_to_replace" is the [:Punctuation:] and the "after_context" is [:WhiteSpace:], and the completed result is transliterating the punctuation into nothing. So we trim the "." from the end of NET but we don't trim the "." from the start. Of course, that doesn't really work in practice, because it misses sooo many other scenarios: echo -n 'Was that a good idea?' | yaz-icu -c chain.xml 1 1 'was' 'Was' 2 1 'that' 'that' 3 1 'a' 'a' 4 1 'good' 'good' 5 1 'idea?' 'idea?' I'm not really sure how to solve this problem in an efficient way. We could just map "C#", "C++", and ".NET" to "csharp", "cplusplus', and 'dotnet', but that's not a very scalable or comprehensive solution for all Koha users. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #9 from David Cook <dcook(a)prosentient.com.au> --- (In reply to Katrin Fischer from comment #3) > Would be interesting to test with Elasticsearch. I agree. It looks like a bit of a known issue: https://stackoverflow.com/questions/63477160/elastic-analyzer-to-enable-mat… https://discuss.elastic.co/t/searching-and-indexing-of-c-c-net-asp-net-does… -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #8 from David Cook <dcook(a)prosentient.com.au> --- (In reply to David Cook from comment #5) > With words in ICU, it > looks like we might only be removing punctuation that is preceded by > whitespace (which seems to imitate the example chain.xml from > https://software.indexdata.com/yaz/doc/yaz.pdf) Ooops. I misread that. Actually, it just strips both whitespace and punctuation regardless of position. Yeah, I don't think this will work with Zebra, since both the word and phrase index registers are supposed to be normalized. I am curious how Google does it. Historically, Google used to normalize its queries and strip out symbols, but the past few years it seems to be more intelligent (it will even convert "+" into "plus" so there is some good AI at work there I reckon). -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #7 from David Cook <dcook(a)prosentient.com.au> --- After some experimenting, it seems YAZ ICU will tokenize based on the "+" without any normalization when using the "line" tokenize rule: echo -n "C++" | yaz-icu -c chain.xml 1 1 'c+' 'C+' 2 1 '+' '+' echo -n "C#" | yaz-icu -c chain.xml 1 1 'c#' 'C#' echo -n ".NET" | yaz-icu -c chain.xml 1 1 '.net' '.NET' I wonder if that's a bug in YAZ because it doesn't do that for all punctuation/symbols... echo -n 'C--' | yaz-icu -c chain.xml 1 1 'c--' 'C--' echo -n 'C???' | yaz-icu -c chain.xml 1 1 'c???' 'C???' -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 --- Comment #6 from David Cook <dcook(a)prosentient.com.au> --- Hmm nope... echo "C++" | yaz-icu -c words-icu.xml 1 1 'c+' 'C+' 2 1 '+' '+' echo "C#" | yaz-icu -c words-icu.xml 1 1 'c' 'C' echo ".NET" | yaz-icu -c words-icu.xml 1 1 'net' 'NET' -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 9729] Unable to use IT search terms such as C#, .NET, C++ in searching
by bugzilla-daemon＠bugs.koha-community.org 16 Jun '21

16 Jun '21

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9729 David Cook <dcook(a)prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook(a)prosentient.com.au --- Comment #5 from David Cook <dcook(a)prosentient.com.au> --- (In reply to Marjorie Barry-Vila from comment #4) > Still valid with Zebra and ElasticSearch in 20.05 > > Marjorie It would be interesting to know if it's CHR or ICU indexing for Zebra. These symbols get stripped out in the normalization process. With phrases in ICU, I see we're arbitrarily removing all punctuation. With words in ICU, it looks like we might only be removing punctuation that is preceded by whitespace (which seems to imitate the example chain.xml from https://software.indexdata.com/yaz/doc/yaz.pdf) So that might be OK. In fact I should be able to test that... With CHR indexing, symbols get converted into spaces, so it won't work. -- You are receiving this mail because: You are watching all bug changes.

1 0