[Bug 37985] New: Extend geo-search to support areas, not just points (MARC 034 subfields defg)
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Bug ID: 37985 Summary: Extend geo-search to support areas, not just points (MARC 034 subfields defg) Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: tadeusz@sosnierz.com QA Contact: testopia@bugs.koha-community.org Bug 31652 ("Add geo-search") implements indexing geographic coordinates (034s,t) in Elasticsearch to enable searching for records referring to specific coordinates. Some records (or authorities) will refer to whole areas, which MARC encodes as north/south/east/westmost points. Since elasticsearch supports indexing shapes, not just points, it would be a nice improvement to be able to index this as well. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #1 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 171877 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171877&action=edit Bug 37985: Extend geo-search to support areas, not just points (MARC 034 subfields defg) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171877|0 |1 is obsolete| | --- Comment #2 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 171878 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171878&action=edit Bug 37985: Extend geo-search to support areas, not just points (MARC 034 subfields defg) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff --- Comment #3 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Test plan: 1. Run the following code to add geolocation data to a book: ``` use C4::Biblio; use Koha::Biblios; # Add the bounding box of Austria to a biblio { my $biblionumber = 139; my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } ``` 2. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, observe no results 3. Apply the patch 4. Perform the required DB migration: `alter table search_field MODIFY COLUMN type enum('','string','date','number','boolean','sum','isbn','stdno','year','callnumber','geo_point', 'geo_shape');` 5. Rebuild the index: `koha-elasticsearch --rebuild -r kohadev` 6. Restart Koha: `koha-plack --restart kohadev` 7. Retry /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, see a result pop up. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |phil@chetcolibrary.org Assignee|koha-bugs@lists.koha-commun |tadeusz@sosnierz.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Brendan Gallagher <brendan@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |brendan@bywatersolutions.co | |m --- Comment #4 from Brendan Gallagher <brendan@bywatersolutions.com> --- Needs a rebase -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171878|0 |1 is obsolete| | --- Comment #5 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 183158 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183158&action=edit Bug 37985: Extend geo-search to support areas, not just points (MARC 034 subfields defg) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |lucas@bywatersolutions.com --- Comment #6 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Some failures noticed by the QA tool: [FAIL] Koha/SearchEngine/Elasticsearch/QueryBuilder.pm FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl Koha/SearchEngine/Elasticsearch/QueryBuilder.pm` [PASS] admin/searchengine/elasticsearch/field_config.yaml [PASS] admin/searchengine/elasticsearch/mappings.yaml [FAIL] installer/data/mysql/kohastructure.sql FAIL tinyint_has_boolean_flag The new column vendor_edi_accounts.po_is_basketname is using TINYINT(1) as type but is not defined as boolean in the schema file (Koha/Schema/Result/VendorEdiAccount.pm), see the SQL12 coding guideline -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183158|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #7 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 195318 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=195318&action=edit Bug 37985: Extend geo-search to support areas, not just points (MARC 034 subfields defg) Patch from commit 0f49758 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off --- Comment #8 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Fixed the tidiness, the other QA issue looks unrelated to this patch - perhaps it's been fixed since elsewhere. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #195318|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #9 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 203278 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=203278&action=edit Bug 37985: Enhance geosearch functionality - Support areas, not just points (MARC 034 subfields defg) - Embed geographic coordinates from GEOGR_NAME authority when indexing records - Allow bounding-box searches in geolocation queries - Allow sorting ES queries by geolocation. Default to closest-first for geosearches - Include ES geographic area coordinates in search results -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #10 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- *** Bug 37821 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #11 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- *** Bug 37537 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #12 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Squashed the other two bugs into this one (and included some extra changes), hoping that maybe this way it'd be easier to push it through the queue – the other patches have spent >2 years bitrotting, conflicting, bitrotting again, conflicting again, and keeping them all up to date and in sync just wasn't worth the effort. This is a fairly complete patch extending indexing, querybuilding, adding a relevant geolocation query variant and adding the coordinates from the search index onto the OPAC page so that plugins like https://github.com/HKS3/HKS3GeoSearch can make use of it more easily. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Extend geo-search to |Enhance geosearch |support areas, not just |functionality |points (MARC 034 subfields | |defg) | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #203278|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #13 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 208103 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208103&action=edit Bug 37985: Enhance geosearch functionality - Support areas, not just points (MARC 034 subfields defg) - Embed geographic coordinates from GEOGR_NAME authority when indexing records - Allow bounding-box searches in geolocation queries - Allow sorting ES queries by geolocation. Default to closest-first for geosearches - Include ES geographic area coordinates in search results Test plan: 1. Start ktd with --search-engine es8 2. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, 2b. Observe no results 3. Run the following fixture to add some geographic data to biblios and auths: ``` use strict; use warnings; use C4::Biblio; use Koha::Biblios; { my $biblionumber = 139; # "Effective Perl programming" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } { my $biblionumber = 262; # "Programming Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', s => 48.143618, t => 17.108417, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } my $authid; { my $record = MARC::Record->new; $record->append_fields(MARC::Field->new('151', '', '', a => 'Austria')); $record->append_fields(MARC::Field->new('034', '', '', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, )); $authid = C4::AuthoritiesMarc::AddAuthority($record, undef, 'GEOGR_NAME'); } { my $biblionumber = 165; # "Higher-order Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('651','','0', a => 'Austria', 9 => $authid, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } ``` 4. Rebuild the elasticsearch index just in case: koha-elasticsearch --rebuild -r kohadev 5. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km 5b. Observe results showing up 6. Try a bounding box search: /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=boundingbox%3A49.710272582105695%2C14.710709564388294%2C46.78501604269256%2C19.182145111263296 6b. Observe results again Sponsored-by: Geosphere Austria -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #208103|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #14 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- Created attachment 208234 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208234&action=edit Bug 37985: Enhance geosearch functionality - Support areas, not just points (MARC 034 subfields defg) - Embed geographic coordinates from GEOGR_NAME authority when indexing records - Allow bounding-box searches in geolocation queries - Allow sorting ES queries by geolocation. Default to closest-first for geosearches - Include ES geographic area coordinates in search results Test plan: 1. Start ktd with --search-engine es8 2. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, 2b. Observe no results 3. Run the following fixture to add some geographic data to biblios and auths: ``` use strict; use warnings; use C4::Biblio; use Koha::Biblios; { my $biblionumber = 139; # "Effective Perl programming" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } { my $biblionumber = 262; # "Programming Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', s => 48.143618, t => 17.108417, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } my $authid; { my $record = MARC::Record->new; $record->append_fields(MARC::Field->new('151', '', '', a => 'Austria')); $record->append_fields(MARC::Field->new('034', '', '', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, )); $authid = C4::AuthoritiesMarc::AddAuthority($record, undef, 'GEOGR_NAME'); } { my $biblionumber = 165; # "Higher-order Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('651','','0', a => 'Austria', 9 => $authid, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } ``` 4. Rebuild the elasticsearch index just in case: koha-elasticsearch --rebuild -r kohadev 5. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km 5b. Observe results showing up 6. Try a bounding box search: /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=boundingbox%3A49.710272582105695%2C14.710709564388294%2C46.78501604269256%2C19.182145111263296 6b. Observe results again Sponsored-by: Geosphere Austria Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |arthur.suzuki@biblibre.com --- Comment #15 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- Great enhancement, can't wait to see it in master :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #208234|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #16 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 208452 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208452&action=edit Bug 37985: Enhance geosearch functionality - Support areas, not just points (MARC 034 subfields defg) - Embed geographic coordinates from GEOGR_NAME authority when indexing records - Allow bounding-box searches in geolocation queries - Allow sorting ES queries by geolocation. Default to closest-first for geosearches - Include ES geographic area coordinates in search results Test plan: 1. Start ktd with --search-engine es8 2. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, 2b. Observe no results 3. Run the following fixture to add some geographic data to biblios and auths: ``` use strict; use warnings; use C4::Biblio; use Koha::Biblios; { my $biblionumber = 139; # "Effective Perl programming" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } { my $biblionumber = 262; # "Programming Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', s => 48.143618, t => 17.108417, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } my $authid; { my $record = MARC::Record->new; $record->append_fields(MARC::Field->new('151', '', '', a => 'Austria')); $record->append_fields(MARC::Field->new('034', '', '', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, )); $authid = C4::AuthoritiesMarc::AddAuthority($record, undef, 'GEOGR_NAME'); } { my $biblionumber = 165; # "Higher-order Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('651','','0', a => 'Austria', 9 => $authid, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } ``` 4. Rebuild the elasticsearch index just in case: koha-elasticsearch --rebuild -r kohadev 5. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km 5b. Observe results showing up 6. Try a bounding box search: /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=boundingbox%3A49.710272582105695%2C14.710709564388294%2C46.78501604269256%2C19.182145111263296 6b. Observe results again Sponsored-by: Geosphere Austria Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #17 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 208453 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208453&action=edit Bug 37985: Don't overwrite facet choices when building geosearch queries -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #18 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Pushed a bugfix (the bug seems to have been preexisting, not introduced here), so needs a new signoff. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #208452|0 |1 is obsolete| | Attachment #208453|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #19 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 208473 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208473&action=edit Bug 37985: Enhance geosearch functionality - Support areas, not just points (MARC 034 subfields defg) - Embed geographic coordinates from GEOGR_NAME authority when indexing records - Allow bounding-box searches in geolocation queries - Allow sorting ES queries by geolocation. Default to closest-first for geosearches - Include ES geographic area coordinates in search results Test plan: 1. Start ktd with --search-engine es8 2. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km, 2b. Observe no results 3. Run the following fixture to add some geographic data to biblios and auths: ``` use strict; use warnings; use C4::Biblio; use Koha::Biblios; { my $biblionumber = 139; # "Effective Perl programming" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } { my $biblionumber = 262; # "Programming Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('034','','', s => 48.143618, t => 17.108417, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } my $authid; { my $record = MARC::Record->new; $record->append_fields(MARC::Field->new('151', '', '', a => 'Austria')); $record->append_fields(MARC::Field->new('034', '', '', 'd' => 9.530833, 'e' => 17.160556, 'f' => 49.020556, 'g' => 46.3725, )); $authid = C4::AuthoritiesMarc::AddAuthority($record, undef, 'GEOGR_NAME'); } { my $biblionumber = 165; # "Higher-order Perl" my $biblio = Koha::Biblios->find($biblionumber); my $framework = $biblio->frameworkcode; my $record = $biblio->metadata->record; my @fields; $fields[0] = MARC::Field->new('651','','0', a => 'Austria', 9 => $authid, ); $record->append_fields(@fields); C4::Biblio::ModBiblio($record, $biblionumber, $framework); } ``` 4. Rebuild the elasticsearch index just in case: koha-elasticsearch --rebuild -r kohadev 5. Go to OPAC /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=lat%3A48.19538740833338+lng%3A16.30711931230611+distance%3A120km 5b. Observe results showing up 6. Try a bounding box search: /cgi-bin/koha/opac-search.pl?advsearch=1&idx=geolocation&do=Search&q=boundingbox%3A49.710272582105695%2C14.710709564388294%2C46.78501604269256%2C19.182145111263296 6b. Observe results again Sponsored-by: Geosphere Austria Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 --- Comment #20 from HKS3 Tadeusz Sośnierz <tadeusz@sosnierz.com> --- Created attachment 208474 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208474&action=edit Bug 37985: Don't overwrite facet choices when building geosearch queries Updated test plan: after appling the patch and performing searches, make sure that clicking facets produces expected results (narrowing down results appropriately). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37985 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com Status|Needs Signoff |Failed QA --- Comment #21 from David Nind <david@davidnind.com> --- I've Failed QA when attempting to test because of the test plan and qa script failures. Can the test plan be updated please? It seems to be missing some steps and the sequence is unclear to me, after spending some time attempting to test: 1. When do I apply the patches? 2. What should I see before and after the patches: - no results before? I assume that us the case with step 2b - X results after? I didn't get any results after 3. No mention of database update required, and restart 4. Doing a reindex is mentioned, however what is the difference between: - koha-elasticsearch --rebuild -r kohadev and - koha-elasticsearch --rebuild -d -b -a kohadev (what I normally do for a full reindex) 5. For the second patch, what should I see or is just facets in general? That is are they broken somehow before or just with the first patch? There are several qa script failures - tidiness, tests fail, filters, spelling and so on. See the list at the end. Testing notes - what I attempted (using KTD): 1. Run through steps 1 to 3 - Step 2: there is a comma at the end of the URL, I removed that - Step 3: save script in /kohadevbox/koha/record-updates.pl and then run: perl record-updates.pl 2. Apply the patches 3. Run the database update: databaseupdate 4. Restart everything: restart_all 5. Reindex everything - not sure what the difference is between: - how I normally do a reindex: koha-elasticsearch --rebuild -d -b -a kohadev - what is in the test plan: koha-elasticsearch --rebuild -r kohadev 6. Run through steps 5 to 6: I don't get any results Test failures ============= qa ... Result: [FAIL] C4/Search.pm FAIL pod_coverage sub _build_stemmed_operand has no pod coverage. sub _get_facets_from_records has no pod coverage. sub _detect_truncation has no pod coverage. sub _build_weighted_query has no pod coverage. [FAIL] Koha/Indexer/Utils.pm FAIL pod_coverage sub _append_grs1_defs_for_control_field has no pod coverage. sub _can_ignore_grs1_cfg_line has no pod coverage. sub _parse_xelm_melm has no pod coverage. sub _append_target_indexes has no pod coverage. sub _parse_grs1_index_def has no pod coverage. sub _parse_grs1_cfg_line has no pod coverage. sub _append_grs1_defs_for_leader has no pod coverage. sub _append_one_target_index has no pod coverage. sub _append_grs1_def_to_dom_cfg has no pod coverage. [SKIP] Koha/Schema/Result/SearchField.pm SKIP pod_coverage No POD exists [WARN] Koha/SearchEngine/Elasticsearch.pm WARN spelling refering ==> referring [PASS] Koha/SearchEngine/Elasticsearch/QueryBuilder.pm [PASS] Koha/SearchEngine/Elasticsearch/Search.pm [PASS] admin/searchengine/elasticsearch/field_config.yaml [PASS] admin/searchengine/elasticsearch/mappings.yaml [FAIL] installer/data/mysql/atomicupdate/bug_37985.pl FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl installer/data/mysql/atomicupdate/bug_37985.pl` [PASS] installer/data/mysql/kohastructure.sql [FAIL] koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt` [PASS] koha-tmpl/opac-tmpl/bootstrap/en/includes/resort_form.inc [FAIL] koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-results.tt FAIL filters missing_filter at line 341 ( data-biblionumber="[% SEARCH_RESULT.biblionumber %]") missing_filter at line 342 ( data-title="[% SEARCH_RESULT.title %]") missing_filter at line 343 ( data-geolocation="[% To.json(SEARCH_RESULT.geolocation) %]") missing_filter at line 350 ( data-biblionumber="[% SEARCH_RESULT.biblionumber %]") missing_filter at line 351 ( data-title="[% SEARCH_RESULT.title %]") missing_filter at line 352 ( data-geolocation-area="[% To.json(SEARCH_RESULT.geolocation_area) %]") [FAIL] opac/opac-search.pl FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl opac/opac-search.pl` [FAIL] t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t` Processing additional checks PASS! Running tests (1) * Proving /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t FAIL # Failed test 'should be able to create "distance from a point" queries' # at /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t line 432. # Structures begin differing at: # $got = undef # $expected = HASH(0x55673b3136e0) # Failed test 'should be able to create "within a bounding box" queries' # at /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t line 444. # Structures begin differing at: # $got = undef # $expected = HASH(0x55673b30ad40) # Looks like you failed 2 tests of 2. # Failed test 'geolocation queries' # at /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t line 451. # Looks like you planned 9 tests but ran 10. # Looks like you failed 1 test of 10 run. /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/9 subtests Test Summary Report ------------------- /kohadevbox/koha/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t (Wstat: 256 (exited 1) Tests: 10 Failed: 2) Failed tests: 9-10 Non-zero exit status: 1 Parse errors: Bad plan. You planned 9 tests but ran 10. Files=1, Tests=10, 2 wallclock secs ( 0.01 usr 0.01 sys + 1.43 cusr 0.19 csys = 1.64 CPU) Result: FAIL -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org