[Bug 31652] New: Add geo-search
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Bug ID: 31652 Summary: Add geo-search Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: new feature Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: domm@plix.at A customer wants to add a geo-search to Koha: * Store lat/lng in biblio 034$s and 034$t * Add lat/lng to ElasticSearch index as geo_point: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/geo-point.html * Add support for ElasticSearch geo-queries: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/geo-queries.htm... * Use Leaflet.js to render maps and pins We (HKS3, koha-support.eu) have already implemented most of it. We will provide our changes as a patch (or a set of patches) and hope that they will be merged into the main Koha source code. It will take a bit more time to actually provide the code, but I create this ticket now so we have an "anchor" if anybody wants to discuss this feature etc. Greetings, domm -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |domm@plix.at |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #1 from Thomas Klausner <domm@plix.at> --- Created attachment 141406 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=141406&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SaerchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ This is a combination of 17 commits. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Change sponsored?|--- |Sponsored Status|NEW |Needs Signoff Patch complexity|--- |Medium patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply CC| |david@davidnind.com --- Comment #2 from David Nind <david@davidnind.com> --- The patch no longer applies 8-(... git bz apply 31652 Bug 31652 - Add geo-search 141406 - Bug 31652: Add geo-search Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 31652: Add geo-search .git/rebase-apply/patch:323: trailing whitespace. # can be single or multiple parameters separated by comma: kw,right-Truncation warning: 1 line adds whitespace errors. Using index info to reconstruct a base tree... M Koha/SearchEngine/Elasticsearch.pm M Koha/SearchEngine/Elasticsearch/QueryBuilder.pm M Koha/SearchEngine/Elasticsearch/Search.pm M admin/searchengine/elasticsearch/field_config.yaml M admin/searchengine/elasticsearch/mappings.yaml M installer/data/mysql/kohastructure.sql M koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt M opac/opac-search.pl .git/rebase-apply/patch:323: trailing whitespace. # can be single or multiple parameters separated by comma: kw,right-Truncation warning: 1 line adds whitespace errors. Falling back to patching base and 3-way merge... Auto-merging opac/opac-search.pl Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt Auto-merging installer/data/mysql/kohastructure.sql CONFLICT (content): Merge conflict in installer/data/mysql/kohastructure.sql Auto-merging admin/searchengine/elasticsearch/mappings.yaml Auto-merging admin/searchengine/elasticsearch/field_config.yaml CONFLICT (content): Merge conflict in admin/searchengine/elasticsearch/field_config.yaml Auto-merging Koha/SearchEngine/Elasticsearch/Search.pm Auto-merging Koha/SearchEngine/Elasticsearch/QueryBuilder.pm Auto-merging Koha/SearchEngine/Elasticsearch.pm error: Failed to merge in the changes. Patch failed at 0001 Bug 31652: Add geo-search -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #141406|0 |1 is obsolete| | --- Comment #3 from Thomas Klausner <domm@plix.at> --- Created attachment 153720 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153720&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SaerchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ This is a combination of 17 commits. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff --- Comment #4 from Thomas Klausner <domm@plix.at> --- I've updated the patch and it should apply now (again) on current master. Please try to signoff again, I'd love to have this in master before Perl Koha Conference (where I'll do a talk on this patch!) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- Comment on attachment 153720 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153720 Bug 31652: Add geo-search Review of attachment 153720: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=31652&attachment=153720) ----------------------------------------------------------------- ::: Koha/SearchEngine/Elasticsearch/Indexer.pm @@ +117,4 @@
}
my $documents = $self->marc_records_to_documents($records); +
It looks like this file might not be altered beyond whitespace? Looks like the whitespace change could be removed to simplify the overall patch. ::: Koha/SearchEngine/Elasticsearch/Search.pm @@ +93,5 @@
my $elasticsearch = $self->get_elasticsearch(); + + # XXX investigate where empty query_string is coming from + delete $query->{query}->{query_string} if + $query->{query}->{query_string} && !%{$query->{query}->{query_string}};
Is this a pre-existing problem or introduced in this patch? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@trust-box.at --- Comment #6 from Mark Hofstetter <koha@trust-box.at> --- This is/was a pre-existing problem -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #153720|0 |1 is obsolete| | --- Comment #7 from Thomas Klausner <domm@plix.at> --- Created attachment 154388 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=154388&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. We added a new SystemPreference `GeoSearchEnabled`. If it is set to `true`, we show a new advanced search field that allows for input of lat/lng and distance (radius). On the search result page, a we render a map using leaflet.js with pins for each found item. Therefor we also added a new api endpoint that returns lat/lng for each biblio in the selection. Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SearchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha: * Set system preference "GeoSearchEnabled" to "true" * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius, for example lat=48.2, lng=16.3, distance=100km * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ Authors: Mark Hofstetter <mark@hofstetter.at>, Thomas Klausner <domm@plix.at> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #8 from Thomas Klausner <domm@plix.at> --- The latest patch has some warnings/errors reported from koha-qa.pl. Plus I'm not totally happy with the way the search params are handled. So I guess it will need a bit more work. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Status|Needs Signoff |Failed QA --- Comment #9 from Lucas Gass <lucas@bywatersolutions.com> --- (In reply to Thomas Klausner from comment #8)
The latest patch has some warnings/errors reported from koha-qa.pl. Plus I'm not totally happy with the way the search params are handled. So I guess it will need a bit more work.
Based on this I am setting this to FQA, to get it out of the 'Needs signoff' queue -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Jessie Zairo <jzairo@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jzairo@bywatersolutions.com | |, | |kelly@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #10 from Thomas Klausner <domm@plix.at> --- Here's some old-ish work on improving ElasticSearch, which could be used/integrated here (as pointed out by Jonathan during PerlKohaCon) https://github.com/fredericd/Koha/compare/master...newelastic -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #11 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Thomas Klausner from comment #10)
Here's some old-ish work on improving ElasticSearch, which could be used/integrated here (as pointed out by Jonathan during PerlKohaCon)
https://github.com/fredericd/Koha/compare/master...newelastic
This should go onto a new bug - if we want to change searching at large let's do that, then base this on top of that work -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arthur.suzuki@biblibre.com See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25453 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #12 from Fridolin Somers <fridolin.somers@biblibre.com> --- Is is necessary to use external server for static files ? https://unpkg.com/leaflet@1.9.1/dist/leaflet.css https://unpkg.com/leaflet@1.9.1/dist/leaflet.js Can't we embed in sources ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #13 from Mark Hofstetter <koha@trust-box.at> --- I don't see a problem in embedding the files if that is "the prefered way of doing things" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #14 from Fridolin Somers <fridolin.somers@biblibre.com> --- Lets see what QA says -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #15 from David Cook <dcook@prosentient.com.au> --- Comment on attachment 154388 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=154388 Bug 31652: Add geo-search Review of attachment 154388: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=31652&attachment=154388) ----------------------------------------------------------------- ::: koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-results.tt @@ +592,5 @@
[% Asset.js("js/openlibrary.js") | $raw %] [% END %] + [% IF ( GeoSearchEnabled ) %] + <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.1/dist/leaflet.css" integrity="sha256-sA+zWATbFveLLNqWO2gtiw3HL/lh1giY/Inf1BJ0z14=" crossorigin=""/> + <script src="https://unpkg.com/leaflet@1.9.1/dist/leaflet.js" integrity="sha256-NDI0K41gVbWqfkkaHj15IzU7PtMoelkzyKp8TOaFQ3s=" crossorigin=""></script>
As Frido suggested, we don't link out to externally hosted static files. Lots of reasons for that. Koha already has leaflet.js and leaflet.css although a much older version: 1.0.3+ed36a04 I'd suggest opening a separate issue to upgrade Leaflet and then having that be a dependency of this issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18066 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18345 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I think we should definitely not link to external files but keep shipping our own. I believe updating the ones we ship would be a good first step here and hopefully not too hard. I think maybe they are used for Mana? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Heather <heather_hernandez@nps.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |heather_hernandez@nps.gov -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |36151 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36151 [Bug 36151] Update leaflet.js to current version -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #17 from Mark Hofstetter <koha@trust-box.at> --- git@github.com:HKS3/koha-branches.git it's in the branch topic-es-2312 for those who want to see it, please send me your github id -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |In Discussion --- Comment #18 from Mark Hofstetter <koha@trust-box.at> --- I now created a public git branch https://github.com/HKS3/koha-devel/tree/topic-es-2312 working on a cypress test -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|domm@plix.at |koha@trust-box.at -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #154388|0 |1 is obsolete| | --- Comment #19 from Mark Hofstetter <koha@trust-box.at> --- Created attachment 164666 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=164666&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SaerchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ Sponsored-by: Geosphere - https://www.geosphere.at/ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #164666|0 |1 is obsolete| | --- Comment #20 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 164689 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=164689&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SaerchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ Sponsored-by: Geosphere - https://www.geosphere.at/ Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #21 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 164690 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=164690&action=edit Bug 31652: (follow-up) Fix update and remove fixed fixme Update test count as well Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #22 from Nick Clemens <nick@bywatersolutions.com> --- Worked with Mark and Martin to move this forward - this version simply adds the ability to do a geo-search - it would need work for an interface to make the searching exposed, but I think this will provide a useful base. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #23 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Please check: 1) Are the changes to name / label in kohastructure intended? `id` int(11) NOT NULL AUTO_INCREMENT, - `name` varchar(255) NOT NULL COMMENT 'the name of the field as it will be stored in the search engine', - `label` varchar(255) NOT NULL COMMENT 'the human readable name of the field, for display', - `type` enum('','string','date','number','boolean','sum','isbn','stdno','year','callnumber') NOT NULL COMMENT 'what type of data this holds, relevant when storing it in the search engine', + `name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'the name of the field as it will be stored in the search engine', + `label` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'the human readable name of the field, for display', + `type` enum('','string','date','number','boolean','sum','isbn','stdno','year','callnumber','geo_point') COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'what type of data this holds, relevant when storing it in the search engine', 2) System preference GeoSearchEnabled I believe this should be removed and due to rewrites? There is no other mention of this pref. +++ b/koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/searching.pref @@ -99,6 +99,13 @@ Searching: - "List of search fields (separated by | or ,) that should not be autotruncated by Elasticsearch even if <a href='/cgi-bin/koha/admin/preferences.pl?op=search&searchfield=QueryAutoTruncate'>QueryAutoTruncate</a> is set to 'Yes':" - pref: ESPreventAutoTruncate class: long + - + - pref: GeoSearchEnabled + type: boolean + choices: + 1: Enable + 0: Disable + - 'GeoSearch via Elasticsearch' Search form: 3) Capitalization Usually I just fix these sneakily, but so we don't forget with the other points: + <option value="geo_point" selected="selected">Geo Point</option> We only capitalize the first word, so this should read: Geo Point (I know Call Number in the same spot is wrong as well, but no need to add more) Please fix! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_24_05_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #164689|0 |1 is obsolete| | --- Comment #24 from Thomas Klausner <domm@plix.at> --- Created attachment 165707 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165707&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * Set system preference "SaerchEngine" to "Elasticsearch" * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ Sponsored-by: Geosphere - https://www.geosphere.at/ Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Additional finetuning: - Fix update and remove fixed fixme - Update test count as well - fix last small issues raised in Comment 23 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165707|0 |1 is obsolete| | --- Comment #25 from Thomas Klausner <domm@plix.at> --- Created attachment 165708 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165708&action=edit Bug 31652: Add geo-search This patch adds geosearch to Koha (using Elasticsearch 7). ElasticSearch search_mappings get new types to store lat/lon, which can be indexed from MARC 034$s and 034$t. There is a small change to the DB to allow a new value in search_field.type ENUM. The QueryBuilder is extended to allow for building advanced ElasticSearch Querys (eg geo_distance) that cannot be represented in a simple string query. The UI for searching (including showing the results on a OSM/Leaflet map) is implemented in a separate plugin (https://github.com/HKS3/HKS3GeoSearch) Test Plan: * make sure you're running ElasticSearch 7 (eg via `curl http://es:9200?pretty | grep number`) * apply patch * got to a Framework, check Editor for 034$s and 034$t and save * got to some books (in the correct framework) and enter some lat and lon into 034$s and 034$t (for example lat=48.216, lon=16.395) * Run the elasticsearch indexer, maybe limited on the books you edited (-bn 123 -bn 456): misc/search_tools/rebuild_elasticsearch.pl -b -v * You can check if the indexing worked by inspecting the document in elasticsearch: * get the biblionumber (eg 123) * curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation * You should get back a JSON fragment containing the lat/lon you stored * You can query elasticsearch directly: * Run the following curl command, but adapt the value for lat/lng and/or the distance (in meters) * curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' * To run the search via Koha, you need to either install and use https://github.com/HKS3/HKS3GeoSearch or create a handcrafted query string: * handcrafted query string: * /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search * HKS3GeoSearch * install the plugin and enable it * got to OPAC / Advanced Search * There is a new input box "Geographic Search" where you can enter lat/long/radius * On the search result page a map is shown with pins for each found biblioitem Sponsored-by: ZAMG - Zentralanstalt für Meterologie und Geodynamik, Austria - https://www.zamg.ac.at/ Sponsored-by: Geosphere - https://www.geosphere.at/ Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Additional finetuning: - Fix update and remove fixed fixme - Update test count as well - fix last small issues raised in Comment 23 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #26 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Please switch back to PQA when you are done! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #27 from Thomas Klausner <domm@plix.at> --- I assume you mean status = "Passed QA"? I've now set it to this :-) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Patch doesn't apply --- Comment #28 from David Nind <david@davidnind.com> --- I thought I would have a go at testing, even though it has passed QA. Unfortunately, the patches don't apply - sha1/fake ancestor error: git bz apply 31652 Bug 31652 - Add geo-search 164690 - Bug 31652: (follow-up) Fix update and remove fixed fixme 165708 - Bug 31652: Add geo-search Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 31652: (follow-up) Fix update and remove fixed fixme error: sha1 information is lacking or useless (Koha/SearchEngine/Elasticsearch/Search.pm). error: could not build fake ancestor Patch failed at 0001 Bug 31652: (follow-up) Fix update and remove fixed fixme -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #29 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I think the patch sequence is wrong now. I was going to do 2 git bz apply -i here to fix on applying. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #30 from David Nind <david@davidnind.com> --- (In reply to Katrin Fischer from comment #29)
I think the patch sequence is wrong now. I was going to do 2 git bz apply -i here to fix on applying.
I tried that, but it didn't work for me 8-) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #31 from Thomas Klausner <domm@plix.at> --- When creating the current patch, I did a rebase on the fresh main and squashed the three commits (original, "(follow-up)" and my fixes according to Katrins comment) into one commit. So there should only be one patch: Bug 31652: Add geo-search (14.16 KB, patch) 2024-04-27 16:58 UTC, Thomas Klausner Not sure how/if I can obsolte this one: Bug 31652: (follow-up) Fix update and remove fixed fixme (2.72 KB, patch) 2024-04-11 13:19 UTC, Nick Clemens Or if it is possible (or how to do it) to tell git bz to only use my rebased/squashed patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #32 from David Nind <david@davidnind.com> --- (In reply to Thomas Klausner from comment #31)
When creating the current patch, I did a rebase on the fresh main and squashed the three commits (original, "(follow-up)" and my fixes according to Katrins comment) into one commit.
So there should only be one patch: Bug 31652: Add geo-search (14.16 KB, patch) 2024-04-27 16:58 UTC, Thomas Klausner
Not sure how/if I can obsolte this one: Bug 31652: (follow-up) Fix update and remove fixed fixme (2.72 KB, patch) 2024-04-11 13:19 UTC, Nick Clemens
Or if it is possible (or how to do it) to tell git bz to only use my rebased/squashed patch
You can obsolete the patch from Bugzilla: 1. Under the attachments section, click the 'Details' link beside the patch to obsolete. 2. In the title/header for the patch, click 'Edit details' again. 3. Click the tick box beside 'obsolete'. 4. Add a comment. 5. Click 'Submit'. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #164690|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Passed QA --- Comment #33 from Thomas Klausner <domm@plix.at> --- I obsolted the old patch, now git bz works again: kohadev-koha@kohadevbox:koha(another_31652_try)$ git bz apply 31652 Bug 31652 - Add geo-search 165708 - Bug 31652: Add geo-search Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 31652: Add geo-search Please try again -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #34 from David Nind <david@davidnind.com> --- Hi Thomas. I can confirm the bug now applies! I had a go at testing (not that this is required, as it has already passed QA). Things I noted: - database update required after applying the patch - also needed to reset the Elasticsearch mappings - I didn't get any search results in the OPAC - wasn't sure how to install the HKS3GeoSearch plugin Here are my testing notes (using KTD): 1. Start up KTD with Elasticsearch: ktd --es7 up 2. Apply the patch 3. Update the database: updatedatabase 4. Restart everything 5. Edit the books framework and make 034$s and 034$t visible in the editor 6. Add example coordinates for a couple of records (I added lat=48.216 and lon=16.395 to records 262 and 126) 7. Reset the Elasticsearch mappings (Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings) 8. Rebuild the search index: koha-elasticsearch --rebuild -d -b -a kohadev 9. Check if indexing working: curl http://es:9200/koha_kohadev_biblios/_doc/123?pretty | grep -A5 geolocation - The result I got: curl http://es:9200/koha_kohadev_biblios/_doc/262?pretty | grep -A5 geolocation % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1078 "geolocation" : [ 0 0 0 --:--:-- --:--:-- --:--:-- 0 3 { "lon" : "16.395", "lat" : "48.216" 1 } 0 ], 0 10783 0 0 2164k 0 --:--:-- --:--:-- --:--:-- 2632k 10. Query Elasticsearch directly: - Result: curl -X GET "http://es:9200/koha_kohadev_biblios/_search?pretty" -H 'Content-Type: application/json' -d '{"query": {"bool":{"must":{"match_all":{}},"filter":{"geo_distance":{"distance":100000,"geolocation":{"lat":48.2,"lon":16.4}}}}}}' { .... A lot of information returned - basically the details for the two records that I added the coordinates to. ... } 11. OPAC query: http://127.0.0.1:8080/cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search - Puts details in the search field: lat:48.25 lng:18.35 distance:100km - Results displayed: No results found! No results found for that in catalog. 12. HKS3GeoSearch Koha Plugin: - No release available. Does this mean I have to clone the repository into the plugins folder? (I tried cloning the repository into /var/lib/koha/kohadev/plugins, but nothing appears in the list of plugins.) David -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |24.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #35 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_24_05_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Needs documenting --- Comment #36 from Fridolin Somers <fridolin.somers@biblibre.com> --- Not backported to 23.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This lays the ground work release notes| |for geographical searching | |with Elasticsearch 7. This | |includes new search types | |for the Elasticsearch | |search mappings to store | |lat/lon and index them. The | |QueryBuilder is extended to | |allow for building advanced | |ElasticSearch querys (eg | |geo_distance) that cannot | |be represented in a simple | |string query. Keywords|release-notes-needed | --- Comment #37 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I've tried to add some release notes - please correct/extend as you see fit. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This lays the ground work |This lays the ground work release notes|for geographical searching |for geographical searching |with Elasticsearch 7. This |with Elasticsearch 7. This |includes new search types |includes new search types |for the Elasticsearch |for the Elasticsearch |search mappings to store |search mappings to store |lat/lon and index them. The |lat/lon and index them. The |QueryBuilder is extended to |QueryBuilder is extended to |allow for building advanced |allow for building advanced |ElasticSearch querys (eg |ElasticSearch querys (eg |geo_distance) that cannot |geo_distance) that cannot |be represented in a simple |be represented in a simple |string query. |string query. | | | |To use the | |feature via the UI, a | |plugin is needed that makes | |use of the special queries. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This lays the ground work |This lays the groundwork release notes|for geographical searching |for geographical searching |with Elasticsearch 7. This |using Elasticsearch 7+. |includes new search types |This includes: |for the Elasticsearch |- New search |search mappings to store |types for Elasticsearch |lat/lon and index them. The |search mappings to store |QueryBuilder is extended to |latitude and longitude |allow for building advanced |values and index them |ElasticSearch querys (eg |(using values from 034$s |geo_distance) that cannot |and 034$t). |be represented in a simple |- Extending the |string query. |QueryBuilder to allow for | |building advanced |To use the |Elasticsearch queries (for |feature via the UI, a |example, geo_distance) that |plugin is needed that makes |cannot be represented in a |use of the special queries. |simple string query. | | | |To | |use this new feature now in | |the OPAC, install and | |enable the HKS3GeoSearch | |plugin | |(https://github.com/HKS3/HK | |S3GeoSearch). --- Comment #38 from David Nind <david@davidnind.com> --- I've made some edits to the release notes. I was able to install the plugin, having the kpz available made that much easier. Results from searching: 1. I used the latitude and longitude from the test plan and added these to some records. 2. The curl queries return the results as in the test plan (the lat and lon values are in the Elasticsearch details). 3. If I search in the OPAC using the example query, the map appears, but it says no search results found. 4. If I search for a word in the records, the map appears with search results, but there are no pins showing on the map. Is there something else I need to do? Here is what I did (using KTD on the latest main, tested using ktd --es7 up and jktd --es8 up): 1. Installed the plugin, enabled, and restarted everything. 2. Made 034$s and 034$t visible in the editor for the BKS framework. 3. Added some latitude and longitude values to two records (I used the values from the test plan - lat=48.216, lon=16.395). 4. Checked that the mappings exist under Search engine configuration (Elasticsearch) for geolocation_lat and geolocation_lon. They did, so I didn't reset the mappings (I think that is right). 5. I reindexed, even though I'm not sure that this is required (koha-elasticsearch --rebuild -d -b -a kohadev). 6. OPAC keyword search - searched with a keyword that returns the records I added values to: - Result ==> Map appears (without pins), with the search results including the records with the latitude and longitude values 7. Direct query (from the test plan): <opac-url>:8080/cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48.25+lng:18.35+distance:100km&do=Search . Result: Map appears, but no results 8. If I change the 100km to 1000km in the query, then the results are listed and there is a pin for one of the records (but not the other). 9. Something else I noted: These details appear in the search box - lat:48.25 lng:18.35 distance:1000km. If I click the search button again, the URL is changed so that it has %3A instead of a +, and there are no results (<opac-url>:8080/cgi-bin/koha/opac-search.pl?idx=&q=lat%3A48.25+lng%3A18.35+distance%3A1000km&weight_search=1). Apart from a very basic knowledge about latitude and longitude, I'm guessing the query is saying give me any records that are X km from the latitude and longitude. Are you able to provide some latitude and longitude values that could be added to some records, and the queries to use to display some pins on the map? (For queries, either the values to enter in the advanced search form field, or the direct query.) This would make it easy for people to try it out with something that works. (And it would be great to demonstrate all your hard work!) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #39 from David Nind <david@davidnind.com> --- Just noting this presentation, will be of great help when updating the documentation (even though some things have changed since it was shared): https://domm.plix.at/talks/2023_helsinki_koha_geosearch/ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #40 from Mark Hofstetter <koha@trust-box.at> --- hi, I added https://github.com/HKS3/HKS3GeoSearch/blob/dev/TO_TEST.md https://github.com/HKS3/HKS3GeoSearch/tree/dev/scripts https://github.com/HKS3/HKS3GeoSearch/blob/dev/scripts/insert_geo_data.pl which adds 3 geo data point to 3 books (hardcoded for the dev install) so if now do a search around http://kohadev.mydnsname.org:8080/cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&q=lat:48+lng:16+distance:50km&do=Search 3 books should be found we are working on the a point and click interface to search -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #41 from David Nind <david@davidnind.com> --- Thanks Mark, that really helps! I added the example co-ordinates to the example records, and used the query - the pins were nicely displayed in the OPAC. Feel free to cc me in on any future bugs, as I'm happy to help with testing and sign off. David -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #42 from Heather <heather_hernandez@nps.gov> --- Created attachment 167223 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167223&action=edit MARC21 cartographic bib record with coordinates Bib record 446 for a nautical chart, from our catalog, keys.bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #43 from Heather <heather_hernandez@nps.gov> --- Created attachment 167224 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167224&action=edit Another MARC21 catalog record with coordinates Bib record 447 for a nautical chart with coordinates, from our catalog, keys.bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #44 from Heather <heather_hernandez@nps.gov> --- Created attachment 167225 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167225&action=edit A third MARC21 bib record with coordinates Bib record no.41753 for a nautical chart, with coordinates, from our catalog, keys.bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #45 from Heather <heather_hernandez@nps.gov> --- Hi! I've uploaded three fully cataloged cartographic records with correctly entered coordinates for nautical charts from our catalog, in case they will help! And for extra fun, they link to onlin images of the charts in case anyone would like to take a look.:) I am super excited about this development--I've been taking the time to enter coordinates into cartographic records for years and years and years with hopes that they'd be useful in the future! Cheerio! h2, A Real Cataloger:) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Bug 31652 depends on bug 36151, which changed state. Bug 36151 Summary: Update leaflet.js to current version https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36151 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Documentation| |https://gitlab.com/koha-com submission| |munity/koha-manual/-/merge_ | |requests/938 Documentation| |Caroline Cyr La Rose contact| | CC| |caroline.cyr-la-rose@inlibr | |o.com Status|Needs documenting |RESOLVED --- Comment #46 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- This bug seems to change things mostly in the "backend" of Koha, paving the way for further changes. But right now, nothing much to change in the manual as far as I could see. I added the new mappings, but that's it. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31652 --- Comment #47 from David Nind <david@davidnind.com> --- (In reply to Caroline Cyr La Rose from comment #46)
This bug seems to change things mostly in the "backend" of Koha, paving the way for further changes. But right now, nothing much to change in the manual as far as I could see. I added the new mappings, but that's it.
That's the case (see comments 37 to 40 for how I tested, after it was pushed). It is usable now using the plugin mentioned in the release notes. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org