[Koha-devel] libsearch-elasticsearch-perl and liblog-any-perl

Stefano Bargioni bargioni at pusc.it
Fri Jul 1 17:02:44 CEST 2016 

Hi, Jonathan:
first of all, I started a fresh Koha 16.05 installation, on Debian 8.5.
This means that tables for Elasticsearch are not populated by the installer.

I populated them using the attached script, quite completely copied from the 3.23.00.050 entry of updatedatabase.pl, as you suggested.
Then I rebuilt the Elasticsearch indices, and tested some queries.
Here are some results:

- query "a" returns 362 bibs using ES, and 1949 (all bib records) using Zebra
- sn:N returns 0 records with ES; you have to use sn,wrdl:N
- a query on auth record fails using ES, tracing the error in opac-error.log
- in queries with ES you have to use the exact accented word (università, études, ...)

Zeno Tajoli suggested me to ask you and Frédéric Demians to backport on 16.05 the progress on Elasticsearch you'll add to the master. Thanks for your opinion.
Unfortunately, as a deputy director of a library, I can't help you much more than this, especially with new bug reports. Sorry.
Cheers. Stefano


> On 27 giu 2016, at 14:09, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
> 
> No, I have much more mappings defined: http://pastebin.com/YJFNa79Q
> 
> You should try to repopulate the tables used by elastic search (see
> entry 3.23.00.050 of updatedatabase.pl) and then do a full reindex.
> 
> 2016-06-27 10:37 GMT+01:00 Stefano Bargioni <bargioni at pusc.it>:
>> Hi, Jonathan,
>> sorry: I typed "..." to cut off the MARC record. Bibs are in the source field and look good.
>> 
>> About mapping:
>> 
>> # curl 'http://localhost:9200/_all/_mapping'
>> {"koha_robin_biblios":{"mappings":{"data":{"properties":{"_all.phrase":{"type":"string","analyzer":"analyser_phrase"},"record":{"type":"string","store":true,"include_in_all":false}}}}}}
>> 
>> Is it correct?
>> Thx. Stefano
>> 
>>> On 24 giu 2016, at 18:33, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
>>> 
>>> "_source":{"record":...,"_id":"1"}
>>> Is it the entire data? :)
>>> If yes, your first record is ":...;" and nothing else.
>>> It should be something like
>>> "_source":{"title":[["Méga
>>> casse-tête"]],"homebranch":[["MAURES"]],"publisher__facet":[["Nathan"]],
>>> [....]}
>>> 
>>> Check at the field/mapping tables: search_field, search_marc_map,
>>> search_marc_to_field. They should be fully populated.
>>> Then rebuild the biblio index using -d (for delete!)
>>> 
>>> 2016-06-24 17:21 GMT+01:00 Stefano Bargioni <bargioni at pusc.it>:
>>>> koha-conf.xml seems correct:
>>>> 
>>>> <elasticsearch>
>>>>    <server>localhost:9200</server>      <!-- may be repeated to include all servers on your cluster -->
>>>>    <index_name>koha_robin</index_name>  <!-- should be unique amongst all the indices on your cluster. _biblios and _authorities will be appended. -->
>>>> </elasticsearch>
>>>> 
>>>> sb
>>>> 
>>>>> On 24 giu 2016, at 18:09, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
>>>>> 
>>>>> Hi Stephano,
>>>>> 
>>>>> Are you sure "koha_robin" is the index_name defined in your koha-conf.xml file?
>>>>> Search for the "elasticsearch" tag in this file.
>>>>> 
>>>>> Cheers,
>>>>> Jonathan
>>>>> 
>>>>> 2016-06-24 16:51 GMT+01:00  <bargioni at pusc.it>:
>>>>>> Nope, MARC21.
>>>>>> I verified that Elasticsearch contains my records:
>>>>>> 
>>>>>> $ curl 'http://localhost:9200/koha_robin_biblios/data/1?pretty'
>>>>>> {
>>>>>> "_index" : "koha_robin_biblios",
>>>>>> "_type" : "data",
>>>>>> "_id" : "1",
>>>>>> "_version" : 1,
>>>>>> "found" : true,
>>>>>> "_source":{"record":...,"_id":"1"}
>>>>>> }
>>>>>> 
>>>>>> Trying to debugging a search with "perl -d", it seems that the query is not correctly passed to Elasticsearch, because it always returns no results.
>>>>>> Some hints for a better debug?
>>>>>> Thx again. Stefano
>>>>>> 
>>>>>>> On 23 giu 2016, at 22:24, Chris Cormack <chris at bigballofwax.co.nz> wrote:
>>>>>>> 
>>>>>>> Hi Stefano
>>>>>>> 
>>>>>>> I wonder if this is a bug in the UNIMARC support, are you using UNIMARC?
>>>>>>> 
>>>>>>> Chris
>>>>>>> 
>>>>>>> On 24 June 2016 at 04:01,  <bargioni at pusc.it> wrote:
>>>>>>>> Thx, Mark, everything is fine, including the rebuild_elastic_search.pl.
>>>>>>>> Now Elasticsearch contains my ~1000 bib records.
>>>>>>>> But no results from searches in Koha, both opac and staff; and nothing in /var/log/koha, unfortunately.
>>>>>>>> Any idea?
>>>>>>>> Stefano
>>>>>>>> 
>>>>>>>>> On 23 giu 2016, at 13:08, Mark Tompsett <mtompset at hotmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Greetings,
>>>>>>>>> 
>>>>>>>>> While working on
>>>>>>>>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16618, I had
>>>>>>>>> issues installing underlying Elastic Search libraries too. This is how I
>>>>>>>>> solved them, though this may
>>>>>>>>> or may not be the best way.
>>>>>>>>> 
>>>>>>>>> --- BEGIN SNIPPET ---
>>>>>>>>> And in case you didn't have elastic search stuff installed:
>>>>>>>>> echo deb http://packages.elastic.co/elasticsearch/1.7/debian stable main |
>>>>>>>>> sudo tee /etc/apt/sources.list.d/elasticsearch.list
>>>>>>>>> wget -O- https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key
>>>>>>>>> add -
>>>>>>>>> sudo apt-get update
>>>>>>>>> 
>>>>>>>>> cd ~
>>>>>>>>> wget
>>>>>>>>> http://debian.koha-community.org/koha/otherthings/elasticsearch_deps.tar.gz
>>>>>>>>> tar xvf elasticsearch_deps.tar.gz
>>>>>>>>> cd es_deps
>>>>>>>>> sudo dpkg i lib*
>>>>>>>>> sudo apt-get install -f
>>>>>>>>> --- END SNIPPET ---
>>>>>>>>> 
>>>>>>>>> GPML,
>>>>>>>>> Mark Tompsett
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Hi,
>>>>>>>>>> I'd like to show Koha running with Elasticsearch at the incoming KohaLa's Symposium in Lyon.
>>>>>>>>>> I have a Debian 8.5 + Koha 16.05 virtual machine.
>>>>>>>>>> Following [1] and [2], I installed Elasticsearch.
>>>>>>>>>> However, installing related Perl modules, I get this error (translated from Italian):
>>>>>>>>>> 
>>>>>>>>>> libsearch-elasticsearch-perl depends on liblog-any-perl (>= 1.02); but:
>>>>>>>>>> installed version of liblog-any-perl is 0.15-1.
>>>>>>>>>> 
>>>>>>>>>> There is something about liblog-any-perl >0.15-1 in [3], but it seems not stable.
>>>>>>>>>> 
>>>>>>>>>> Any help?
>>>>>>>>>> Thx. Stefano
>>>>>>>>>> 
>>>>>>>>>> [1] https://wiki.koha-community.org/wiki/Elasticsearch
>>>>>>>>>> [2] http://debian.koha-community.org/koha/otherthings/
>>>>>>>>>> [3] https://packages.debian.org/search?keywords=liblog-any-perl
>>>>>>>> _______________________________________________
>>>>>>>> Koha-devel mailing list
>>>>>>>> Koha-devel at lists.koha-community.org
>>>>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>>>> website : http://www.koha-community.org/
>>>>>>>> git : http://git.koha-community.org/
>>>>>>>> bugs : http://bugs.koha-community.org/
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Koha-devel mailing list
>>>>>> Koha-devel at lists.koha-community.org
>>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>> website : http://www.koha-community.org/
>>>>>> git : http://git.koha-community.org/
>>>>>> bugs : http://bugs.koha-community.org/
>>>>> _______________________________________________
>>>>> Koha-devel mailing list
>>>>> Koha-devel at lists.koha-community.org
>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>> website : http://www.koha-community.org/
>>>>> git : http://git.koha-community.org/
>>>>> bugs : http://bugs.koha-community.org/
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> Koha-devel mailing list
>>>> Koha-devel at lists.koha-community.org
>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>> website : http://www.koha-community.org/
>>>> git : http://git.koha-community.org/
>>>> bugs : http://bugs.koha-community.org/
>>> _______________________________________________
>>> Koha-devel mailing list
>>> Koha-devel at lists.koha-community.org
>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>> website : http://www.koha-community.org/
>>> git : http://git.koha-community.org/
>>> bugs : http://bugs.koha-community.org/
>> 
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: populate_tables_elasticsearch.pl.zip
Type: application/zip
Size: 786 bytes
Desc: not available
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20160701/5b7ddeea/attachment.zip>

More information about the Koha-devel mailing list