Z3950 authority server integrated with ElasticSearch
Hello everyone, I'm currently facing an issue with a new Z3950 authority server integrated with ElasticSearch that I'm trying to set up. My basic searches are functioning properly, but when I attempt to use @attr attributes, it doesn't seem to work. Interestingly, attribute 1=1 (personal name) works fine. I've attempted to address this by adding the attributes in the /etc/z3950/attribute_mappings.yaml file, but unfortunately, that hasn't resolved the issue. This situation has led me to question whether it's possible, with ElasticSearch for authorities, to perform a search on a specific field, such as a title by adding @attr 1=4 on yaz-client command line. In contrast, I haven't encountered this problem with an authority server under ZEBRA. For example, when I use @attr 1=4, it works seamlessly, but with ElasticSearch, it's not functioning as expected. Example with a Z3950 authority server under Zebra $ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Zebra Information Server/GFS/YAZ Version: 2.0.44/4.2.69 026ad3d737be4cbba4e98c6b6dc753f8029e3655 Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets Elapsed: 0.446096 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 172856, setno 1 SearchResult-1: term=canada cnt=172856 records returned: 0 Elapsed: 0.208821 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 1920, setno 2 SearchResult-1: term=canada cnt=1920 records returned: 0 Elapsed: 0.858340 Z> With a Z3950 authority server under ElasticSearch $ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Koha/GFS/YAZ Version: 21.05.11.000/5.27.1 872b6f92a024bb53bc1c11dfeccd47f065f98238 Options: search present triggerResourceCtrl namedResultSets Elapsed: 0.425683 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 1 records returned: 0 Elapsed: 0.478937 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 2 records returned: 0 Elapsed: 0.261396 Z> Utilizing a Z3950 authority server with ElasticSearch, the yaz-client connects successfully to Koha/GFS/YAZ (Version 21.05.11.000/5.27.1). However, when attempting searches for 'canada' with the attribute '@attr 1=4', the number of results (56811) remains the same, '@attr 1=4' does not have effect. I aim to enable the functionality of attributes to be able to search via these field nameany => '@attr 1=1002 "#term" ', authorany => '@attr 1=1003 "#term" ', authorcorp => '@attr 1=2 "#term" ', authorpersonal => '@attr 1=1 "#term" ', authormeetingcon => '@attr 1=3 "#term" ', subject => '@attr 1=21 "#term" ', subjectsubdiv => '@attr 1=47 "#term" ', title => '@attr 1=4 "#term" ', uniformtitle => '@attr 1=6 "#term" ', srchany => '@attr 1=1016 "#term" ', controlnumber => '@attr 1=12 "#term" ', Do you have any suggestion/check to fix this situation ? Cheers Hammat Wele
I believe, for Zebra, the file /etc/koha/zebradb/authorities/etc/bib1.att maps attributes to indexes. I don't know elasticsearch that well, so I can't say whether there is or isn't a similar configuration there. Or even whether it's possible to do that. In the past it was deemed unfeasible to have elasticsearch handle the Z39.50 protocol, and zebra was kept around for that. Maybe that's changed since then? On Wed, Feb 28, 2024 at 2:37 PM Hammat Wele via Koha-devel < koha-devel@lists.koha-community.org> wrote:
Hello everyone,
I'm currently facing an issue with a new Z3950 authority server integrated with ElasticSearch that I'm trying to set up. My basic searches are functioning properly, but when I attempt to use @attr attributes, it doesn't seem to work. Interestingly, attribute 1=1 (personal name) works fine. I've attempted to address this by adding the attributes in the /etc/z3950/attribute_mappings.yaml file, but unfortunately, that hasn't resolved the issue.
This situation has led me to question whether it's possible, with ElasticSearch for authorities, to perform a search on a specific field, such as a title by adding @attr 1=4 on yaz-client command line. In contrast, I haven't encountered this problem with an authority server under ZEBRA. For example, when I use @attr 1=4, it works seamlessly, but with ElasticSearch, it's not functioning as expected.
Example with a Z3950 authority server under Zebra
$ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Zebra Information Server/GFS/YAZ Version: 2.0.44/4.2.69 026ad3d737be4cbba4e98c6b6dc753f8029e3655 Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets Elapsed: 0.446096 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 172856, setno 1 SearchResult-1: term=canada cnt=172856 records returned: 0 Elapsed: 0.208821 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 1920, setno 2 SearchResult-1: term=canada cnt=1920 records returned: 0 Elapsed: 0.858340 Z>
With a Z3950 authority server under ElasticSearch $ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Koha/GFS/YAZ Version: 21.05.11.000/5.27.1 872b6f92a024bb53bc1c11dfeccd47f065f98238 Options: search present triggerResourceCtrl namedResultSets Elapsed: 0.425683 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 1 records returned: 0 Elapsed: 0.478937 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 2 records returned: 0 Elapsed: 0.261396 Z>
Utilizing a Z3950 authority server with ElasticSearch, the yaz-client connects successfully to Koha/GFS/YAZ (Version 21.05.11.000/5.27.1). However, when attempting searches for 'canada' with the attribute '@attr 1=4', the number of results (56811) remains the same, '@attr 1=4' does not have effect.
I aim to enable the functionality of attributes to be able to search via these field nameany => '@attr 1=1002 "#term" ', authorany => '@attr 1=1003 "#term" ', authorcorp => '@attr 1=2 "#term" ', authorpersonal => '@attr 1=1 "#term" ', authormeetingcon => '@attr 1=3 "#term" ', subject => '@attr 1=21 "#term" ', subjectsubdiv => '@attr 1=47 "#term" ', title => '@attr 1=4 "#term" ', uniformtitle => '@attr 1=6 "#term" ', srchany => '@attr 1=1016 "#term" ', controlnumber => '@attr 1=12 "#term" ',
Do you have any suggestion/check to fix this situation ?
Cheers Hammat Wele _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
-- Michael Hafen Washington County School District Technology Department Systems Analyst
Hi Hammat, you probably already found this, but some information on the use and config of the z3950-responder can be found here: https://wiki.koha-community.org/wiki/Setting_up_the_Z39.50_and_SRU_Server The original bug introducing it was: *Bug 13937* <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13937> - Add an Elasticsearch-compatible Z39.50/SRU daemon There are some open bugs and some closed ones (search for "ALL responder"), but I couldn't find information on searching authorities. Hope this helps, Katrin On 28.02.24 22:37, Hammat Wele via Koha-devel wrote:
Hello everyone,
I'm currently facing an issue with a new Z3950 authority server integrated with ElasticSearch that I'm trying to set up. My basic searches are functioning properly, but when I attempt to use @attr attributes, it doesn't seem to work. Interestingly, attribute 1=1 (personal name) works fine. I've attempted to address this by adding the attributes in the /etc/z3950/attribute_mappings.yaml file, but unfortunately, that hasn't resolved the issue.
This situation has led me to question whether it's possible, with ElasticSearch for authorities, to perform a search on a specific field, such as a title by adding @attr 1=4 on yaz-client command line. In contrast, I haven't encountered this problem with an authority server under ZEBRA. For example, when I use @attr 1=4, it works seamlessly, but with ElasticSearch, it's not functioning as expected.
Example with a Z3950 authority server under Zebra
$ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Zebra Information Server/GFS/YAZ Version: 2.0.44/4.2.69 026ad3d737be4cbba4e98c6b6dc753f8029e3655 Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets Elapsed: 0.446096 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 172856, setno 1 SearchResult-1: term=canada cnt=172856 records returned: 0 Elapsed: 0.208821 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 1920, setno 2 SearchResult-1: term=canada cnt=1920 records returned: 0 Elapsed: 0.858340 Z>
With a Z3950 authority server under ElasticSearch $ yaz-client xxx.xxx.xxx.xxx:xxxx/authorities Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : 81 Name : Koha/GFS/YAZ Version: 21.05.11.000/5.27.1 872b6f92a024bb53bc1c11dfeccd47f065f98238 Options: search present triggerResourceCtrl namedResultSets Elapsed: 0.425683 Z> f canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 1 records returned: 0 Elapsed: 0.478937 Z> f @attr 1=4 canada Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 56811, setno 2 records returned: 0 Elapsed: 0.261396 Z>
Utilizing a Z3950 authority server with ElasticSearch, the yaz-client connects successfully to Koha/GFS/YAZ (Version 21.05.11.000/5.27.1). However, when attempting searches for 'canada' with the attribute '@attr 1=4', the number of results (56811) remains the same, '@attr 1=4' does not have effect.
I aim to enable the functionality of attributes to be able to search via these field nameany => '@attr 1=1002 "#term" ', authorany => '@attr 1=1003 "#term" ', authorcorp => '@attr 1=2 "#term" ', authorpersonal => '@attr 1=1 "#term" ', authormeetingcon => '@attr 1=3 "#term" ', subject => '@attr 1=21 "#term" ', subjectsubdiv => '@attr 1=47 "#term" ', title => '@attr 1=4 "#term" ', uniformtitle => '@attr 1=6 "#term" ', srchany => '@attr 1=1016 "#term" ', controlnumber => '@attr 1=12 "#term" ',
Do you have any suggestion/check to fix this situation ?
Cheers Hammat Wele _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
participants (3)
-
Hammat Wele -
Katrin Fischer -
Michael Hafen