[koha-commits] main Koha release repository branch new__12478_elasticsearch created. v3.22.00-1041-gcbc9add
Git repo owner
gitmaster at git.koha-community.org
Wed Mar 23 23:40:58 CET 2016
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "main Koha release repository".
The branch, new__12478_elasticsearch has been created
at cbc9add8ce6b38f61816660b44f4771712058e90 (commit)
- Log -----------------------------------------------------------------
commit cbc9add8ce6b38f61816660b44f4771712058e90
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Tue Oct 13 10:45:32 2015 +0100
Bug 12478: Display facet terms ordered by number of occurrences
By default ES returns the facet terms ordered by most used, which makes
sense.
This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.
For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).
On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.
To test easily this change, turn on the displayFacetCount pref.
commit c2f9b605dafb39079fbef1d2b968ba9770486739
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Oct 13 17:21:36 2015 +1300
Revert "Bug 12478: Display facet terms ordered by number of occurrences"
This reverts commit cd4905c2969b067476881016d0b03271f0bcc7c8.
This commit caused an error in C4::Search::GetFacets when running in
zebra mode.
Conflicts:
Koha/SearchEngine/Elasticsearch/Search.pm
commit e978d54fb0b0b4dcb7d67c4990933cd31aee98ac
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 12 17:03:38 2015 +0100
Bug 12478: Remove previous mappings file (sql)
commit 6a07c766c9dbc7aa7224165414199c5418263f95
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 12 16:54:12 2015 +0100
Bug 12478: Manually add the many_to_many relationships
This will create the add_to_* accessors for us.
commit 8d9fffed910ae4800a2f3ee4be84e60b766e519d
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 12 16:52:11 2015 +0100
Bug 12478: update the schema
These files have been generated with the
misc/devel/update_dbix_class_files.pl script.
commit 36abdfb1b39a998223510636d06837ea4883dc1e
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 12 16:46:16 2015 +0100
Bug 12478: Move mapping attributes to the join table
To make things consistent, the attributes of the mappings (sort,
suggestible, facet) should be moved to the join tables
(search_marc_to_field).
Indeed the search_marc_map could represent a mapping for several search
fields with different attributes. Before this change it was not
possible.
commit cbd5c46890c2df843b780f59c38b1840ac75096d
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Sep 25 15:48:50 2015 +0100
Bug 12478: Add Koha::SearchField[s] and Koha::SearchMarcMap[s] classes
This was initially submited on bug 14899, but we will need it here.
commit 5fb4cc016ed1daaf4b78951bb6c4572f3ba700a5
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 12 17:11:28 2015 +0100
Bug 12478: Add the yaml mappings file
This should facilitate the modification of the mappings.
The yaml format is more readable than sql.
commit 41b1fc971977c4a1f5e1205dcc00bfe81e7cc70f
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Oct 8 17:02:21 2015 +1300
Bug 12478: "fix" notforloan
commit 8ea9dd3671679d7380ba32a1a1ec757a8589e2f6
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 17:15:01 2015 +0100
Bug 12478: Take the FacetMaxCount pref into account
The system preference FacetMaxCount should work as expected with ES.
commit 7bc7b7853f8e41b24312e24a4a22ab73bc87d492
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 11:34:54 2015 +0100
Bug 12478: Remove empty limit parameter
If no limit are passed, the url will contain '&limit=' anyway.
It is not necessary and can be avoided easily
Test plan:
1/ Search for a term in your catalogue
2/ Hover over a link in the facet area
3/ The link is
cgi-bin/koha/opac-search.pl?idx=kw&q=your_term&limit=&limit=[...]
With this patch, the empty limit parameter does not appear.
commit 635bf48f9b49823e4fd58f5ffe8a1ec61bbb0bf3
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 15:29:52 2015 +0100
Bug 12478: Display facet terms ordered by number of occurrences
By default ES returns the facet terms ordered by most used, which makes
sense.
This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.
For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).
On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.
To test easily this change, turn on the displayFacetCount pref.
commit b4a931c97e5c892f90b01ce6415e6ed0ee0fb721
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 12:50:52 2015 +0100
Bug 12478: facets - Display description instead of code for locations
commit 4f40f4faf6054d3736c7c10a923416426961d52d
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 12:51:37 2015 +0100
Bug 12478: Add Koha::AuthorisedValue[s] class
These 2 classes come from bug 10363.
commit 2fd23189f32050f4ef9bdcb041407841e4b2cfe6
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 12:17:09 2015 +0100
Bug 12478: Use the new Koha::ItemTypes to retrieve itypes descriptions
commit ab73a7a5c6952f07caa816640117e8dda4a346fd
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Tue Sep 15 17:02:28 2015 +0100
Bug 12478: Add Koha::ItemType[s] classes
These 2 classes come from bug 14828.
Tests in t/db_dependent/Koha/ItemTypes.t should be updated.
commit 092cbbb9612a73baf3e1963a49badbe54d746ac1
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Mon Oct 5 10:58:50 2015 +0100
Bug 12478: (follow-up) Display the correct number of facets
This patch fixes a side-effect of the previous "Display the correct
number of facets (5 instead of 6)" patch.
If you ask for more facets ('show more' link) and that the number of
facets is > 5 but < 10, 1 facet will be missing.
commit 4e78bfce3519f41f24e41909a98f83b95d10a10a
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Oct 5 17:44:14 2015 +1300
Bug 12478: abort early if there's no elasticserch definition
commit 71aacc265247c2f05871d416aec2017e5cba16db
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Oct 5 16:21:42 2015 +1300
Bug 14899: schema changes to make the database better
commit 3ba813247532c97bbb7fbd22ff1614c55d051622
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Sep 25 16:06:07 2015 +0100
Bug 14899: DB changes
This patch applies several changes to the DB structure introduced by
the previous ES patch set:
1/ Add a search_field.label column
=> It will be easier to understand what the search_field does.
Indeed, some are not user friendly: acqdate, an, pl, ff8-23, ln, etc.
TODO later: Update the labels with correct values (at the moment
label=name)
2/ Update the foreign key search_marc_to_field.search_marc_map_id with
on delete cascase. This will permit to automatically remove the entries
in search_marc_to_field when they are deleted in search_marc_map.
3/ Remove the index_name_2 unique constraint on the search_marc_map table.
I don't understand how this could be useful, it was defined like (index_name,
marc_type, marc_field), so it means a mapping cannot be defined twice with
different values for facet, suggestible and sort.
This limitation does not seem to make sense.
Robin, please correct me if I am wrong :)
commit 228d1a83e00e193d04b9bb01cd7892092166a336
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Oct 2 15:23:07 2015 +1300
Bug 12478: add location, su-geo, se to facets
commit 297236399afac68f830a736ce1e0f2b03d6908ff
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Oct 1 15:53:44 2015 +1300
Bug 12478: fix errors from rebase and new upstream version
commit 332163e292ef8a2ee3a8a576a8d1aca0cffe346d
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Sep 30 15:46:18 2015 +1300
Bug 12478: fix some issues on rebase
There were rebase conflicts that it was just easier to postpone until
afterwards.
commit 4b8c04430d393204ffc6eb43344be67a1fcfc7b8
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Sep 4 13:00:48 2015 +0100
Bug 12478: Fix error on indexing a specific record
% perl misc/search_tools/rebuild_elastic_search.pl -bn 42
Can't locate object method "idnumber" via package "MARC::Record" at
misc/search_tools/rebuild_elastic_search.pl line 171.
commit 7406aa1913bdcc253043c4c25c96cc787200ae00
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:59:26 2015 +0100
Bug 12478: Fix encoding issue on facets
Filer on "Ślez, Ts." => Can't escape \x{015A}, try uri_escape_utf8()
instead at
/home/koha/src/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 221.
commit d18d792a507719fcad9dfb8b4efb77528e253739
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:29:23 2015 +0100
Bug 12478: Do not display the 'Show more' links if no more facet available
commit e738ecb74ced1f35e0567987acb43a814b4fde6a
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:24:24 2015 +0100
Bug 12478: Display the correct number of facets (5 instead of 6)
The loop on facet entries was wrong, it should stop after 5
iterations.
commit 35bc072ee3bce52bf9ca4414d8eb87ebb0d10b2c
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:13:45 2015 +0100
Bug 12478: Remove Solr occurrences reintroduced by a previous patch
commit f092daaa7bee7a9d34eefd160326c797ef746c81
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:13:07 2015 +0100
Bug 12478: Fix pod in the rebuild_ES.pl script
commit 726e8542a8233d62b4176eb8f0e0fd3dcfd040ac
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:08:27 2015 +0100
Bug 12478: Fix the verbose flag on reindexing
commit 4da1c443da96f9c6a5fa7c3f64e9bcd22f70f440
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:07:51 2015 +0100
Bug 12478: Change the commit count to 5k
It will improve the indexing time.
commit d5b09e33325f4281e6912855242038564d1fda7c
Author: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
Date: Fri Aug 28 11:07:04 2015 +0100
Bug 12478: Fix the UNIMARC and NORMARC indexing
1/ This patch fixes a major issue on UNIMARC and NORMARC installations
(Compulsive c/p on insert into insert into search_marc_to_field for unimarc and normarc)
2/ After fixing that, I got
invalid marc path at /usr/share/perl5/Catmandu/Fix/marc_map.pm line 35.
on reindexing.
It was caused by some "" values instead of NULL.
3/ Then, a "duplicate entry" error was caused by 3 lines for Local-number.
It seems to be an error, 1 is enough (I suppose 0909 was an error too).
Note that the following is not fixed by this patch:
MariaDB [koha_es_unimarc]> insert into search_field (name, type) select distinct mapping, type from elasticsearch_mapping;
Query OK, 73 rows affected, 57 warnings (0.05 sec)
Records: 73 Duplicates: 0 Warnings: 57
MariaDB [koha_es_unimarc]> show warnings;
+---------+------+--------------------------------------------+
| Level | Code | Message |
+---------+------+--------------------------------------------+
| Warning | 1265 | Data truncated for column 'type' at row 1 |
commit 88c0022908a48f06eb60005673ae21764952e059
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Sep 25 15:11:29 2015 +1200
Bug 12478: allow more granular sorting configuration
This allows sorting to be configured within a field. For example, while
many values are included for search on author, sorting should only be
done on the main entry values. This permits that by have a sort value,
which can be true, false, or null. true and null are pretty much the
same, but false means that a field isn't available for sorting on. By
default (null), fields can be sorted on.
commit 14d6b216c4bedd902103113f01b1d6947d57f2cb
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Aug 27 16:13:16 2015 +1200
Bug 12478: pull the SQL updates from the browse interface
Hopefully will reduce the rate of conflicts down the road. We'll see.
commit 75931667f5d4d269513bc50fb49e192e43a90999
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Aug 26 17:29:17 2015 +1200
Bug 12478: pull the SQL updates from the browse interface
commit fab8839657bf5a8b80fd37766b8739471cb2e52e
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Aug 25 17:03:08 2015 +1200
Bug 12478: the facet "show more" links work properly now
commit f787a5cadfa36949ffcd1772d3fac93d9db36f03
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Aug 25 15:13:51 2015 +1200
Bug 12478: add the lenient option to the query_string query
It doesn't really help.
commit bec54b03588e2a96fcba48f1d534543d8b36c207
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Aug 20 18:12:42 2015 +1200
Bug 12478: improve error reporting a bit
commit 9a42fb8855000c968651c757cbb85b88c2b92d77
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Aug 3 17:55:54 2015 +1200
Bug 12478: update to support new Cat::Store::ES
commit 70525daa74457ba184c079cab1688c9e0435643a
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jul 17 14:38:19 2015 +1200
Bug 12478: ES is now updated when records are updated/deleted
commit 735be3f5986187be1f39336ccfe91402e3be8e1a
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jul 10 15:19:21 2015 +1200
Bug 12478: fix capitalisation issue with index names
commit 01a139fd4bfd03f355513f9408de316e55c40043
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Jul 8 16:34:12 2015 +1200
Bug 12478: make things using SimpleSearch use the new version
commit d5f2064cca8d9c9ef2ec3445b2030bc3daf5cd3f
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jun 19 15:55:46 2015 +1200
Bug 12478: a replacement to the SimpleSearch interface implemented
commit 57a3710206a55105e4a089ca9c6ee5810d781f41
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Jun 17 15:37:29 2015 +1200
Bug 12478: fix issue with authority sort order being weird
commit 27401d263d3747bbdc93a5c5a0f4f55571cca27d
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Jun 15 17:35:41 2015 +1200
Bug 12478: auth search works in the staff client
Some issues with sort order though.
commit 56528f4c3facf6deb8e8a3ee20602bc6e91ee59b
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Jun 15 15:12:39 2015 +1200
Bug 12478: starting authority search in staff client
commit 5f0c706f2c0a7b861bf891889c0c3b08a68efd60
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jun 12 15:19:39 2015 +1200
Bug 12478: fix issues with the conversion SQL
commit 528e7de069ab945dc698e2002aca54e7376bdf42
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Jun 10 14:35:33 2015 +1200
Bug 12478: change the schema for storing mappings
This makes the mapping schema a proper many-to-many relational
structure, which is a little more fiddly to work with, but will make
things like editors for it easier to write.
commit 1324ffd874fe9930a901e71b7bc693496e89d2f8
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jun 5 14:29:31 2015 +1200
Bug 12478: fix multi-choice stuff in advanced search
This means that things like itype get "OR"ed together, rather than
"AND"ed like other things.
commit 5dc1a018d1061ff6cdeef6c9584aa1dc9559e362
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu May 14 15:15:04 2015 +1200
Bug 12478: starting to add search to staff client
commit 8e6bc0fa8ac77ea07d119d558b654becd1e8e3cf
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Mar 31 17:26:49 2015 +1300
Bug 12478: fix wrong counts/results for authority searches
commit b448656f4c03ba09884facd904ac04f48f1a82c0
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Mar 25 17:29:14 2015 +1300
Bug 12478: fix issues caused by rebasing
commit 1fb368ca7bbb4e91a5741cc6c94dacc50b1a2091
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Mar 17 17:23:56 2015 +1300
Bug 12478: reinstate zebra compatibility for auth search
commit df434d2ff43a7a45c918c458bbbc98435d930f85
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Mar 13 15:27:24 2015 +1300
Bug 12478: paging is fixed now too
commit be348c7016dfbff0426e3c5024b58a36d4ff826d
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Mar 13 15:22:53 2015 +1300
Bug 12478: sort is sorted
commit 2f6eafc21366307317775ac2571e29b614214cfb
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Mar 12 18:11:30 2015 +1300
Bug 12478: phrase searching for authorities is happier
commit 7cb9e562aed54565995bba3d51af7814658cf785
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Mar 11 16:14:14 2015 +1300
Bug 12478: fix the way counting is done
commit c81e6b8dd8d2810ec3953bb6a94ce1ad8a3c8576
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Mar 10 18:12:51 2015 +1300
Bug 12478: authority paging works
commit 4c8888d772500623221371260a1a33037c36c032
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Mar 10 17:32:08 2015 +1300
Bug 12478: working on authority results
commit d68fd6175858d3db126d9cf4daf62f4bec33ce05
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Mar 4 18:19:25 2015 +1300
Bug 12478: make authid get to the templates properly
commit c90e0103cbbd8e6c42f64426c75cab8b3f5da960
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Mar 4 17:38:18 2015 +1300
Bug 12478: basic auth search and display is working
commit e4a3c713714d9486c4ebd226b8516b7d01c65662
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Mar 3 14:02:19 2015 +1300
Bug 12478: more authorities
commit 584bf298d599174b09f71f5e8c06c4dc742e5f79
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Feb 24 14:43:01 2015 +1300
Bug 12478 - more authorites
commit a8f1ed9d9a687bbffab03e78d7ea47ead33fd411
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Feb 17 18:38:07 2015 +1300
Bug 12478 - more authority searching
Queries are being built, but they seem to be wrong as no results are
returned.
commit 6f3d37d6164b8022b6867de8ca1877fa034f4401
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Feb 4 16:42:12 2015 +1300
Bug 12478 - more authority query building
commit a8d23da6af7055e0d0ed82b54035c95dec0c53a9
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Feb 3 14:40:50 2015 +1300
Bug 12478 - fix syntax errors so that stuff runs
commit 916d52ce2e18f921d43b1f1d44ab0b7b79e7cb72
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Fri Jan 30 16:06:46 2015 +1300
Bug 12478 - authority work in progress
commit 91aadf800a86e6bbf861f6f5406be5de4d7b958e
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Nov 19 15:05:12 2014 +1300
Bug 12478 - fix issue with class loading
commit 806a2004117944f9db2a9eaf5052ab7bf24f5bdd
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Nov 19 14:07:20 2014 +1300
Bug 12478 - fix error in configuration loading
commit 1592cb7ad8c00ba5e0ef592bfb0d3ef31d329468
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Wed Nov 12 17:27:14 2014 +1300
Bug 12478 - fix error in SQL
commit 42bfd0e574b78e9ce1a793d2e7d8745d781a03ad
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Sep 23 17:32:57 2014 +1200
Bug 12478 - add test cases
commit bf1204e3dccbb7232986b0c66439ddab19ca4765
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Thu Sep 4 13:42:26 2014 +1200
Bug 12478 - authorities can now be stored in ES
(Not fetched yet though.)
commit ff51bde371ac00c542d2ded8e2085593f0b08331
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Mon Aug 25 15:55:50 2014 +1200
Bug 12478 - set up database tables for elasticsearch
This sets up the database changes needed to run ES. This isn't intended
to be ready for upstreaming, just to make testing easier.
commit f4767425996b296a2abfc2f46c440babadd7adbd
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Sep 16 17:38:43 2014 +1200
Bug 12478 - pile of elasticsearch code
commit 88ea3deb5b81dfe802f3f69db4338d55e7db6de9
Author: Robin Sheat <robin at catalyst.net.nz>
Date: Tue Sep 16 17:34:25 2014 +1200
Bug 12478 - add some base objects that the ES code will depend on
-----------------------------------------------------------------------
hooks/post-receive
--
main Koha release repository
More information about the koha-commits
mailing list