[Koha-devel] libsearch-elasticsearch-perl and liblog-any-perl

Jonathan Druart jonathan.druart at bugs.koha-community.org
Fri Jul 1 17:22:18 CEST 2016 

2016-07-01 16:02 GMT+01:00 Stefano Bargioni <bargioni at pusc.it>:
> Hi, Jonathan:
> first of all, I started a fresh Koha 16.05 installation, on Debian 8.5.
> This means that tables for Elasticsearch are not populated by the installer.

I have opened a new bug report:
Bug 16838 - Elasticsearch - mapping tables are not populated on new installs

> I populated them using the attached script, quite completely copied from the 3.23.00.050 entry of updatedatabase.pl, as you suggested.
> Then I rebuilt the Elasticsearch indices, and tested some queries.
> Here are some results:
>
> - query "a" returns 362 bibs using ES, and 1949 (all bib records) using Zebra
> - sn:N returns 0 records with ES; you have to use sn,wrdl:N
> - a query on auth record fails using ES, tracing the error in opac-error.log
> - in queries with ES you have to use the exact accented word (università, études, ...)
>
> Zeno Tajoli suggested me to ask you and Frédéric Demians to backport on 16.05 the progress on Elasticsearch you'll add to the master. Thanks for your opinion.
> Unfortunately, as a deputy director of a library, I can't help you much more than this, especially with new bug reports. Sorry.
> Cheers. Stefano
>
>
>> On 27 giu 2016, at 14:09, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
>>
>> No, I have much more mappings defined: http://pastebin.com/YJFNa79Q
>>
>> You should try to repopulate the tables used by elastic search (see
>> entry 3.23.00.050 of updatedatabase.pl) and then do a full reindex.
>>
>> 2016-06-27 10:37 GMT+01:00 Stefano Bargioni <bargioni at pusc.it>:
>>> Hi, Jonathan,
>>> sorry: I typed "..." to cut off the MARC record. Bibs are in the source field and look good.
>>>
>>> About mapping:
>>>
>>> # curl 'http://localhost:9200/_all/_mapping'
>>> {"koha_robin_biblios":{"mappings":{"data":{"properties":{"_all.phrase":{"type":"string","analyzer":"analyser_phrase"},"record":{"type":"string","store":true,"include_in_all":false}}}}}}
>>>
>>> Is it correct?
>>> Thx. Stefano
>>>
>>>> On 24 giu 2016, at 18:33, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
>>>>
>>>> "_source":{"record":...,"_id":"1"}
>>>> Is it the entire data? :)
>>>> If yes, your first record is ":...;" and nothing else.
>>>> It should be something like
>>>> "_source":{"title":[["Méga
>>>> casse-tête"]],"homebranch":[["MAURES"]],"publisher__facet":[["Nathan"]],
>>>> [....]}
>>>>
>>>> Check at the field/mapping tables: search_field, search_marc_map,
>>>> search_marc_to_field. They should be fully populated.
>>>> Then rebuild the biblio index using -d (for delete!)
>>>>
>>>> 2016-06-24 17:21 GMT+01:00 Stefano Bargioni <bargioni at pusc.it>:
>>>>> koha-conf.xml seems correct:
>>>>>
>>>>> <elasticsearch>
>>>>>    <server>localhost:9200</server>      <!-- may be repeated to include all servers on your cluster -->
>>>>>    <index_name>koha_robin</index_name>  <!-- should be unique amongst all the indices on your cluster. _biblios and _authorities will be appended. -->
>>>>> </elasticsearch>
>>>>>
>>>>> sb
>>>>>
>>>>>> On 24 giu 2016, at 18:09, Jonathan Druart <jonathan.druart at bugs.koha-community.org> wrote:
>>>>>>
>>>>>> Hi Stephano,
>>>>>>
>>>>>> Are you sure "koha_robin" is the index_name defined in your koha-conf.xml file?
>>>>>> Search for the "elasticsearch" tag in this file.
>>>>>>
>>>>>> Cheers,
>>>>>> Jonathan
>>>>>>
>>>>>> 2016-06-24 16:51 GMT+01:00  <bargioni at pusc.it>:
>>>>>>> Nope, MARC21.
>>>>>>> I verified that Elasticsearch contains my records:
>>>>>>>
>>>>>>> $ curl 'http://localhost:9200/koha_robin_biblios/data/1?pretty'
>>>>>>> {
>>>>>>> "_index" : "koha_robin_biblios",
>>>>>>> "_type" : "data",
>>>>>>> "_id" : "1",
>>>>>>> "_version" : 1,
>>>>>>> "found" : true,
>>>>>>> "_source":{"record":...,"_id":"1"}
>>>>>>> }
>>>>>>>
>>>>>>> Trying to debugging a search with "perl -d", it seems that the query is not correctly passed to Elasticsearch, because it always returns no results.
>>>>>>> Some hints for a better debug?
>>>>>>> Thx again. Stefano
>>>>>>>
>>>>>>>> On 23 giu 2016, at 22:24, Chris Cormack <chris at bigballofwax.co.nz> wrote:
>>>>>>>>
>>>>>>>> Hi Stefano
>>>>>>>>
>>>>>>>> I wonder if this is a bug in the UNIMARC support, are you using UNIMARC?
>>>>>>>>
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> On 24 June 2016 at 04:01,  <bargioni at pusc.it> wrote:
>>>>>>>>> Thx, Mark, everything is fine, including the rebuild_elastic_search.pl.
>>>>>>>>> Now Elasticsearch contains my ~1000 bib records.
>>>>>>>>> But no results from searches in Koha, both opac and staff; and nothing in /var/log/koha, unfortunately.
>>>>>>>>> Any idea?
>>>>>>>>> Stefano
>>>>>>>>>
>>>>>>>>>> On 23 giu 2016, at 13:08, Mark Tompsett <mtompset at hotmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Greetings,
>>>>>>>>>>
>>>>>>>>>> While working on
>>>>>>>>>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16618, I had
>>>>>>>>>> issues installing underlying Elastic Search libraries too. This is how I
>>>>>>>>>> solved them, though this may
>>>>>>>>>> or may not be the best way.
>>>>>>>>>>
>>>>>>>>>> --- BEGIN SNIPPET ---
>>>>>>>>>> And in case you didn't have elastic search stuff installed:
>>>>>>>>>> echo deb http://packages.elastic.co/elasticsearch/1.7/debian stable main |
>>>>>>>>>> sudo tee /etc/apt/sources.list.d/elasticsearch.list
>>>>>>>>>> wget -O- https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key
>>>>>>>>>> add -
>>>>>>>>>> sudo apt-get update
>>>>>>>>>>
>>>>>>>>>> cd ~
>>>>>>>>>> wget
>>>>>>>>>> http://debian.koha-community.org/koha/otherthings/elasticsearch_deps.tar.gz
>>>>>>>>>> tar xvf elasticsearch_deps.tar.gz
>>>>>>>>>> cd es_deps
>>>>>>>>>> sudo dpkg i lib*
>>>>>>>>>> sudo apt-get install -f
>>>>>>>>>> --- END SNIPPET ---
>>>>>>>>>>
>>>>>>>>>> GPML,
>>>>>>>>>> Mark Tompsett
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I'd like to show Koha running with Elasticsearch at the incoming KohaLa's Symposium in Lyon.
>>>>>>>>>>> I have a Debian 8.5 + Koha 16.05 virtual machine.
>>>>>>>>>>> Following [1] and [2], I installed Elasticsearch.
>>>>>>>>>>> However, installing related Perl modules, I get this error (translated from Italian):
>>>>>>>>>>>
>>>>>>>>>>> libsearch-elasticsearch-perl depends on liblog-any-perl (>= 1.02); but:
>>>>>>>>>>> installed version of liblog-any-perl is 0.15-1.
>>>>>>>>>>>
>>>>>>>>>>> There is something about liblog-any-perl >0.15-1 in [3], but it seems not stable.
>>>>>>>>>>>
>>>>>>>>>>> Any help?
>>>>>>>>>>> Thx. Stefano
>>>>>>>>>>>
>>>>>>>>>>> [1] https://wiki.koha-community.org/wiki/Elasticsearch
>>>>>>>>>>> [2] http://debian.koha-community.org/koha/otherthings/
>>>>>>>>>>> [3] https://packages.debian.org/search?keywords=liblog-any-perl
>>>>>>>>> _______________________________________________
>>>>>>>>> Koha-devel mailing list
>>>>>>>>> Koha-devel at lists.koha-community.org
>>>>>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>>>>> website : http://www.koha-community.org/
>>>>>>>>> git : http://git.koha-community.org/
>>>>>>>>> bugs : http://bugs.koha-community.org/
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Koha-devel mailing list
>>>>>>> Koha-devel at lists.koha-community.org
>>>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>>> website : http://www.koha-community.org/
>>>>>>> git : http://git.koha-community.org/
>>>>>>> bugs : http://bugs.koha-community.org/
>>>>>> _______________________________________________
>>>>>> Koha-devel mailing list
>>>>>> Koha-devel at lists.koha-community.org
>>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>> website : http://www.koha-community.org/
>>>>>> git : http://git.koha-community.org/
>>>>>> bugs : http://bugs.koha-community.org/
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Koha-devel mailing list
>>>>> Koha-devel at lists.koha-community.org
>>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>> website : http://www.koha-community.org/
>>>>> git : http://git.koha-community.org/
>>>>> bugs : http://bugs.koha-community.org/
>>>> _______________________________________________
>>>> Koha-devel mailing list
>>>> Koha-devel at lists.koha-community.org
>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>> website : http://www.koha-community.org/
>>>> git : http://git.koha-community.org/
>>>> bugs : http://bugs.koha-community.org/
>>>
>> _______________________________________________
>> Koha-devel mailing list
>> Koha-devel at lists.koha-community.org
>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : http://www.koha-community.org/
>> git : http://git.koha-community.org/
>> bugs : http://bugs.koha-community.org/

More information about the Koha-devel mailing list