[Koha-devel] Elasticsearch not searching.

Tomas Cohen Arazi tomascohen at gmail.com
Thu Sep 29 15:08:33 CEST 2016 

Its ok if you want to try ElasticSearch even though it is experimental. But
aren't you migrating a production service?

El jue., 29 sept. 2016 a las 9:57, Rodrigo Santellan (<rsantellan at gmail.com>)
escribió:

> Tomas,
>
> I have attached all the vagrant files.
>
> For installing it I use:
> - vagrant up
> - vagrant ssh
> - sudo sh /vagrant/provisioning/enviromentsSettings.sh
> - exit
> - vagrant ssh
> - sh /vagrant/provisioning/fullProvisioning.sh
> - sh /vagrant/provisioning/makeKoha.sh #Choose the dev installation
> - sudo sh /vagrant/provisioning/apacheEnviroment.sh
> - sudo systemctl restart apache2
> - sudo systemctl restart mysql
> - cd /vagrant/provisioning/kohaMigrationData/
> - gunzip < koha.sql.gz | mysql -u root -p # Old instance of koha 2.2.3
> - cat /vagrant/provisioning/koha3-schema.sql | mysql -u root -p
> - nohup ./importData.sh
> - mv nohup nohup.importdata
> - nohup ./runscripts.sh
>
> I'm running all the scripts that I need by hand.
>
> I'm trying to understand the new (for me) Koha so I can maximize the time
> I have to work on it, thats why I don't run the packages installer. I also
> have the problem that I know I have to modify it :(
>
> Thanks for the help!
>
>
> On Thu, Sep 29, 2016 at 9:38 AM, Tomas Cohen Arazi <tomascohen at gmail.com>
> wrote:
>
>> Dood, just:
>> - launch a clean VBox with Debian 8 or Ubuntu 16.04
>> - install mysql-server (in Ubuntu 16.04 install mariadb-server)
>> - Add the repo, install koha-common
>> - $ sudo koha-create --create-db test_library
>>
>> And that's all of it. It is working for everyone else, so please try to
>> write some more detailed install/setup steps so we understand what's going
>> on.
>>
>> PS: if this was a dev box, you need https://github.com/mkfifo/koha-gitify
>>
>> El jue., 29 sept. 2016 a las 9:31, Rodrigo Santellan (<
>> rsantellan at gmail.com>) escribió:
>>
>>> David,
>>>
>>> I tried running the Kohadevbox but I have fail because I'm behind a
>>> proxy and the provisioning fails and I can't start up the machine.
>>>
>>> So my next step was to create a empty Vagrant with the box:
>>> bento/ubuntu-16.04 and run post install scripts.
>>>
>>> The default version installed on ubuntu 16.04 is:
>>>
>>> idzebra-2.0:
>>>   Installed: 2.0.59-1
>>>   Candidate: 2.0.59-1
>>>
>>> I have successfully run the searches with Elasticsearch. Now I have to
>>> see if I'm missing something or not because using Elasticsearch instead of
>>> Zebra!
>>>
>>> Thanks for your help!
>>>
>>> Regards
>>>
>>>
>>> On Wed, Sep 28, 2016 at 10:04 PM, David Cook <dcook at prosentient.com.au>
>>> wrote:
>>>
>>>> Also, if you're having issues with Zebra, you can try posting to this
>>>> list,
>>>> as there are a number of us who can probably help you.
>>>>
>>>> I see that you posted before about trying to see how many biblios are
>>>> indexed in Zebra, and that you had a problem using _ALLRECORDS. Did you
>>>> have
>>>> any other problems? Were you able to successfully perform other Zebra
>>>> searches?
>>>>
>>>> I admit that the error "[114] Unsupported Use attribute -- v2 addinfo
>>>> '_ALLRECORDS'" is rather difficult to figure out. Usually, if you see
>>>> that
>>>> 114 code, it means that a particular index hasn't been created yet
>>>> within
>>>> the Zebra database*. However, since you're using the special index
>>>> _ALLRECORDS, it should be created.
>>>>
>>>> It looks like you're using Zebra 2.0.44. That should support
>>>> _ALLRECORDS.
>>>> I'm on Zebra 2.0.60, and this is the output I see:
>>>>
>>>> Z> find @attr 1=_ALLRECORDS @attr 2=103 ""
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a success.
>>>> Number of hits: 956, setno 2
>>>> SearchResult-1: term= cnt=956
>>>> records returned: 0
>>>> Elapsed: 0.010064
>>>>
>>>> I just tried It on a Debian server running Zebra 2.0.44 and I see this
>>>> output:
>>>> Z> find @attr 1=_ALLRECORDS @attr 2=103 ""
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a success.
>>>> Number of hits: 14882, setno 1
>>>> SearchResult-1: term= cnt=14882
>>>> records returned: 0
>>>> Elapsed: 0.028209
>>>>
>>>> Even if I delete all the records in my database, I still see:
>>>>
>>>> Z> find @attr 1=_ALLRECORDS @attr 2=103 ""
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a success.
>>>> Number of hits: 0, setno 1
>>>> SearchResult-1: term= cnt=0
>>>> records returned: 0
>>>> Elapsed: 0.007409
>>>>
>>>> Your Zebra database "biblios" appears to be created otherwise you'd see:
>>>>
>>>> Z> find @attr 1=_ALLRECORDS @attr 2=103 ""
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a bloomin' failure.
>>>> Number of hits: 0, setno 1
>>>> Result Set Status: none
>>>> records returned: 0
>>>> Diagnostic message(s) from database:
>>>>     [109] Database unavailable -- v2 addinfo 'biblios'
>>>> Elapsed: 0.001682
>>>>
>>>> Hmm... my best guess is that somehow your Zebra install has been
>>>> corrupted
>>>> maybe. I'm not really sure. This is a very bizarre error.
>>>>
>>>> How did you install Koha and Zebra? I notice the home directory for a
>>>> vagrant user... is this a Kohadevbox?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *
>>>> In this particular database, I've only ever had 1 record, and it
>>>> contained
>>>> very little data. It contained a title, so the following search works:
>>>> Z> find @attr 1=4 "test"
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a success.
>>>> Number of hits: 0, setno 5
>>>> SearchResult-1: term=test cnt=0
>>>> records returned: 0
>>>> Elapsed: 0.002783
>>>>
>>>> However, the record never contained an author, so the following search
>>>> fails
>>>> with that Error 114:
>>>> Z> find @attr 1=1 "Bob"
>>>> Sent searchRequest.
>>>> Received SearchResponse.
>>>> Search was a bloomin' failure.
>>>> Number of hits: 0, setno 6
>>>> Result Set Status: none
>>>> records returned: 0
>>>> Diagnostic message(s) from database:
>>>>     [114] Unsupported Use attribute -- v2 addinfo '1'
>>>> Elapsed: 0.000289
>>>>
>>>> David Cook
>>>> Systems Librarian
>>>> Prosentient Systems
>>>> 72/330 Wattle St
>>>> Ultimo, NSW 2007
>>>> Australia
>>>>
>>>> Office: 02 9212 0899
>>>> Direct: 02 8005 0595
>>>>
>>>>
>>>> > -----Original Message-----
>>>> > From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel-
>>>> > bounces at lists.koha-community.org] On Behalf Of Chris Cormack
>>>> > Sent: Thursday, 29 September 2016 6:53 AM
>>>> > To: Rodrigo Santellan <rsantellan at gmail.com>
>>>> > Cc: koha-devel <koha-devel at lists.koha-community.org>
>>>> > Subject: Re: [Koha-devel] Elasticsearch not searching.
>>>> >
>>>> > * Rodrigo Santellan (rsantellan at gmail.com) wrote:
>>>> > > Chris I will install Elastic 1.7 and try again!! It should be on the
>>>> > > wiki or the bugzilla issue that tracks the elasticsearch
>>>> development.
>>>> > >
>>>> > > Thank a lot for the quick response!
>>>> >
>>>> > You could also follow this bug
>>>> >
>>>> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17255
>>>> >
>>>> > It may be that with the new packages there, it might work already.
>>>> > It hasnt been tested yet though
>>>> >
>>>> > Chris
>>>> >
>>>> > >
>>>> > > On Wed, Sep 28, 2016 at 5:43 PM, Chris Cormack <
>>>> chrisc at catalyst.net.nz>
>>>> > wrote:
>>>> > >
>>>> > >     Hi Rodrigo
>>>> > >
>>>> > >     ElasticSearch in Koha currently only works with Elastic 1.7
>>>> > >     We are working on upgrading it to working the latest version but
>>>> there
>>>> > >     were some non backward compatible changes. That is what you are
>>>> > bumping
>>>> > >     into
>>>> > >
>>>> > >     Support in Koha is still experimental, but it once we upgrade it
>>>> should be
>>>> > >     more
>>>> > >     solid.
>>>> > >
>>>> > >     You could try installing 1.7 and seeing how you go
>>>> > >
>>>> > >     Chris
>>>> > >
>>>> > >
>>>> > >     * Rodrigo Santellan (rsantellan at gmail.com) wrote:
>>>> > >     > Hi,
>>>> > >     >
>>>> > >     > I have installed Elasticsearch after giving up on Zebra. I
>>>> have
>>>> follow
>>>> > >     the the
>>>> > >     > wiki: https://wiki.koha-community.org/wiki/Elasticsearch#
>>>> > >     Install_elasticsearch
>>>> > >     > and installed latest Elasticsearch through the official
>>>> repository.
>>>> > >     >
>>>> > >     > My installed version is:
>>>> > >     > curl -XGET 'localhost:9200'
>>>> > >     > {
>>>> > >     >   "name" : "Her",
>>>> > >     >   "cluster_name" : "elasticsearch",
>>>> > >     >   "cluster_uuid" : "yuU4XtZwT1WtK1SH-3Xofw",
>>>> > >     >   "version" : {
>>>> > >     >     "number" : "2.4.1",
>>>> > >     >     "build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
>>>> > >     >     "build_timestamp" : "2016-09-27T18:57:55Z",
>>>> > >     >     "build_snapshot" : false,
>>>> > >     >     "lucene_version" : "5.5.2"
>>>> > >     >   },
>>>> > >     >   "tagline" : "You Know, for Search"
>>>> > >     > }
>>>> > >     >
>>>> > >     > I have run the script perl
>>>> misc/search_tools/rebuild_elastic_search.pl -v
>>>> > >     -d
>>>> > >     > like the documentation says and I have this error:
>>>> > >     > Failed to parse mapping [data]: analyzer on field
>>>> [Local-number]
>>>> must
>>>> > be
>>>> > >     set
>>>> > >     > when search_analyzer is set
>>>> > >     >
>>>> > >     > The log on the elasticsearch.log is
>>>> > >     >
>>>> > >     > [2016-09-28 20:27:55,272][DEBUG][action.admin.indices.create]
>>>> [Her]
>>>> > >     > [koha_dev_biblios] failed to create
>>>> > >     > MapperParsingException[Failed to parse mapping [data]:
>>>> analyzer on
>>>> > field
>>>> > >     > [Local-number] must be set when search_analyzer is set];
>>>> nested:
>>>> > >     > MapperParsingException[analyzer on field [Local-number] must
>>>> be
>>>> set
>>>> > when
>>>> > >     > search_analyzer is set];
>>>> > >     >         at org.elasticsearch.index.mapper.MapperService.merge
>>>> > >     > (MapperService.java:291)
>>>> > >     >         at
>>>> > >     >
>>>> >
>>>> org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute
>>>> > >     > (MetaDataCreateIndexService.java:317)
>>>> > >     >         at
>>>> org.elasticsearch.cluster.ClusterStateUpdateTask.execute
>>>> > >     > (ClusterStateUpdateTask.java:45)
>>>> > >     >         at
>>>> > >     > org.elasticsearch.cluster.service.InternalClusterService.
>>>> > >     runTasksForExecutor
>>>> > >     > (InternalClusterService.java:468)
>>>> > >     >         at
>>>> > >     >
>>>> >
>>>> org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run
>>>> > >     > (InternalClusterService.java:772)
>>>> > >     >         at
>>>> > >     >
>>>> >
>>>> org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$
>>>> > >     TieBreakingPrioritizedRunnable.runAndClean
>>>> > >     > (PrioritizedEsThreadPoolExecutor.java:231)
>>>> > >     >         at
>>>> > >     >
>>>> >
>>>> org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$
>>>> > >     TieBreakingPrioritizedRunnable.run
>>>> > >     > (PrioritizedEsThreadPoolExecutor.java:194)
>>>> > >     >         at java.util.concurrent.ThreadPoolExecutor.runWorker
>>>> > >     > (ThreadPoolExecutor.java:1142)
>>>> > >     >         at java.util.concurrent.ThreadPoolExecutor$Worker.run
>>>> > >     > (ThreadPoolExecutor.java:617)
>>>> > >     >         at java.lang.Thread.run(Thread.java:745)
>>>> > >     > Caused by: MapperParsingException[analyzer on field
>>>> [Local-number]
>>>> > must
>>>> > >     be set
>>>> > >     > when search_analyzer is set]
>>>> > >     >         at
>>>> > >     > org.elasticsearch.index.mapper.core.TypeParsers.
>>>> > >     parseAnalyzersAndTermVectors
>>>> > >     > (TypeParsers.java:234)
>>>> > >     >         at
>>>> org.elasticsearch.index.mapper.core.TypeParsers.parseTextField
>>>> > >     > (TypeParsers.java:250)
>>>> > >     >         at
>>>> > >     >
>>>> > org.elasticsearch.index.mapper.core.StringFieldMapper$TypeParser.parse
>>>> > >     > (StringFieldMapper.java:165)
>>>> > >     >         at
>>>> > >     > org.elasticsearch.index.mapper.object.ObjectMapper$
>>>> > >     TypeParser.parseProperties
>>>> > >     > (ObjectMapper.java:309)
>>>> > >     >         at
>>>> > >     > org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.
>>>> > >     parseObjectOrDocumentTypeProperties
>>>> > >     > (ObjectMapper.java:222)
>>>> > >     >         at
>>>> > >     >
>>>> > org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.pars
>>>> > e
>>>> > >     > (RootObjectMapper.java:139)
>>>> > >     >         at
>>>> org.elasticsearch.index.mapper.DocumentMapperParser.parse
>>>> > >     > (DocumentMapperParser.java:118)
>>>> > >     >         at
>>>> org.elasticsearch.index.mapper.DocumentMapperParser.parse
>>>> > >     > (DocumentMapperParser.java:99)
>>>> > >     >         at org.elasticsearch.index.mapper.MapperService.parse
>>>> > >     > (MapperService.java:549)
>>>> > >     >         at org.elasticsearch.index.mapper.MapperService.merge
>>>> > >     > (MapperService.java:319)
>>>> > >     >         at org.elasticsearch.index.mapper.MapperService.merge
>>>> > >     > (MapperService.java:289)
>>>> > >     >         ... 9 more
>>>> > >     >
>>>> > >     > I'm again at a loss, I believe the problem is on the method:
>>>> > >     > get_elasticsearch_mappings in Koha::ElasticSearch but I don't
>>>> know
>>>> > how to
>>>> > >     > follow it.!
>>>> > >     >
>>>> > >     > Regards and thank for any help!!
>>>> > >     >
>>>> > >     > Rodrigo
>>>> > >
>>>> > >     > _______________________________________________
>>>> > >     > Koha-devel mailing list
>>>> > >     > Koha-devel at lists.koha-community.org
>>>> > >     >
>>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>> > >     > website : http://www.koha-community.org/
>>>> > >     > git : http://git.koha-community.org/
>>>> > >     > bugs : http://bugs.koha-community.org/
>>>> > >
>>>> > >
>>>> > >     --
>>>> > >     Chris Cormack
>>>> > >     Catalyst IT Ltd.
>>>> > >     +64 4 803 2238
>>>> > >     PO Box 11-053, Manners St, Wellington 6142, New Zealand
>>>> > >
>>>> > >
>>>> >
>>>> > > _______________________________________________
>>>> > > Koha-devel mailing list
>>>> > > Koha-devel at lists.koha-community.org
>>>> > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>> > > website : http://www.koha-community.org/ git :
>>>> > > http://git.koha-community.org/ bugs :
>>>> http://bugs.koha-community.org/
>>>> >
>>>> >
>>>> > --
>>>> > Chris Cormack
>>>> > Catalyst IT Ltd.
>>>> > +64 4 803 2238
>>>> > PO Box 11-053, Manners St, Wellington 6142, New Zealand
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Koha-devel mailing list
>>> Koha-devel at lists.koha-community.org
>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>> website : http://www.koha-community.org/
>>> git : http://git.koha-community.org/
>>> bugs : http://bugs.koha-community.org/
>>
>> --
>> Tomás Cohen Arazi
>> Theke Solutions (https://theke.io <http://theke.io/>)
>> ✆ +54 9351 3513384
>> GPG: B2F3C15F
>>
>
> --
Tomás Cohen Arazi
Theke Solutions (https://theke.io <http://theke.io/>)
✆ +54 9351 3513384
GPG: B2F3C15F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20160929/4617bf4b/attachment-0001.html>

More information about the Koha-devel mailing list