[Bug 18085] New: Elasticsearch ids are not being stored correctly
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 Bug ID: 18085 Summary: Elasticsearch ids are not being stored correctly Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org The upgrade from 1.X to 5.X works very well for indexing existing records, however, it seems the '_id' field is not being given to elastic at save so it is generating a random id. The problem with this is that each record update will save a new record as opposed to overwriting the existing record and you will get multiple items in your search. To reproduce: 1 - Have ES5 running and have some records 2 - Choose a biblionumber and reindex just that record, deleting the index before: perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -b -bn 2365 3 - View that record on the command line: curl -XGET 'localhost:9200/koha_kohadev_biblios/_search/?pretty=true&q=es_id:2365' 4 - Note the '_id' field is of the form: "_id" : "AVoemHc9LtgvIQ1yOOU6", 5 - Search for the record in koha - should be one 6 - Add or edit the items on that record 7 - Search again, the record returns multiple times This should be a small fix, not sure where the disconnect is though -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chris@bigballofwax.co.nz, | |jonathan.druart@bugs.koha-c | |ommunity.org, | |kyle@bywatersolutions.com, | |olli-antti.kivilahti@jns.fi | |, tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |17255 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17255 [Bug 17255] Upgrade Elastic Search code to work with version 5.1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Olli, any chances you can try and fix this one? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 --- Comment #2 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Sorry. Busy. I (or somebody in Koha-Suomi) will solve the ES issues when we decide to go to ES. Can't tell from the top of my hat, where the problem is. Sounds like something to do with interfacing via Catmandu. Catmandu needs some special care to handle _id properly. Maybe the issue is with one of the multiple Catmandu modules needed to get the MARC::Record to ES. Do we really need Catmandu at this phase? We can directly index to ES, or any search engine because Koha already has the infrastucture to facilitate multiple search engines. Now we need to wait for the Catmandu-guys to port Catmandu::Store::ElasticSearch to 5.2 or supply a patch (if the problem is in Catmandu) Also We need to implement Catmandu in the JavaScript-side if we want to consume the results directly to the browser. We would have to implement any parser there though. AJAX-searches would be nice! I guess Catmandu defines some kind of a "standard" for working with library data, but so does MARCXML. Turning MARCXML to JSON is really simple. Or even better... JSON-LD!! But this is not an answer to the bug at hand :) My bet is Catmandu. Giving it the right parameters to handle _id. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |INVALID --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Nvm - this is a kohadevbox issue - version of Catmandu installed is not the .deb from bug 17255 was using version 3.03 - need .5 or greater -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 --- Comment #4 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Ran into this too. For the record: ES log says "MapperParsingException[Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.]" You need to cpan Catmandu::Store::ElasticSearch, not Catmandu::Store::Elasticsearch S vs s! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18085 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18130 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org