Default indexing and search options for 3.14
Hi, I would like to have a discussion regarding making certain options the default for new installations of Koha 3.14 and deprecating other options. Here's the list of proposals: Enable DOM mode by default for both bibs and authorities ----------------------------------------------------------------------------------- Now that MARC21, NORMARC, and UNIMARC have DOM indexing configurations available, this is now possible. Deprecating the GRS-1 filter ----------------------------------------- Giving that DOM has significant advantages of GRS-1 in its flexibility, and as the DOM filter would be required for non-MARC metadata support by Zebra, I propose that we announce a deprecation of the GRS-1 configuration with it to be removed by 3.16. I foresee several features coming down the pike for 3.16 that would require a reindexing during upgrade anyway, so switching to DOM for 3.16 would not impose a burden that wouldn't be present anyway. Enable QueryParser by default --------------------------------------------- Bug 10542 might be a blocker for this, and doing something about bug 10831 would be nice in order to do this. Deprecating non-QueryParser mode --------------------------------------------------- Removing the non-QueryParser code would go a long way towards easing the rewrite of C4::Search. Enable ICU by default ------------------------------- This was discussed during hackfest. One precondition I see is that bug 10729 would have to pass QA Regards, Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
Hi. Going forward we expect to use DOM and ICU with MARC21. Broadly we are getting good behaviour. However at present searching authorities gives the same results for "starts with" and "contains". On some installations we have that are older versions of zebra and Koha and probably not using DOM or ICU the two authority searches do give results that are more-expected. Colin has been trying to track down why "starts with" is treated the same as "contains", but it would be good to have this resolved so we are improving searching for all and not breaking existing functions. If anyone can shine any light on this problem or a solution that would be most helpful. +1 for the roadmap. Ian On 24/10/2013 15:44, Galen Charlton wrote:
Hi,
I would like to have a discussion regarding making certain options the default for new installations of Koha 3.14 and deprecating other options. Here's the list of proposals:
Enable DOM mode by default for both bibs and authorities -----------------------------------------------------------------------------------
Now that MARC21, NORMARC, and UNIMARC have DOM indexing configurations available, this is now possible.
Deprecating the GRS-1 filter -----------------------------------------
Giving that DOM has significant advantages of GRS-1 in its flexibility, and as the DOM filter would be required for non-MARC metadata support by Zebra, I propose that we announce a deprecation of the GRS-1 configuration with it to be removed by 3.16. I foresee several features coming down the pike for 3.16 that would require a reindexing during upgrade anyway, so switching to DOM for 3.16 would not impose a burden that wouldn't be present anyway.
Enable QueryParser by default ---------------------------------------------
Bug 10542 might be a blocker for this, and doing something about bug 10831 would be nice in order to do this.
Deprecating non-QueryParser mode ---------------------------------------------------
Removing the non-QueryParser code would go a long way towards easing the rewrite of C4::Search.
Enable ICU by default -------------------------------
This was discussed during hackfest. One precondition I see is that bug 10729 would have to pass QA
Regards,
Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com <mailto:gmc@esilibrary.com> direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Bays Director of Projects, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7774 995297 (mobile) +44 (0) 800 756 6384 (fax) skype: ian.bays email: ian.bays@ptfs-europe.com
On Thu, Oct 24, 2013 at 04:32:10PM +0100, Ian Bays wrote:
Colin has been trying to track down why "starts with" is treated the same as "contains", but it would be good to have this resolved so we are improving searching for all and not breaking existing functions.
This problem exists irrespective of the ICU and DOM settings so is not relevant to the question Galen posed. I think the problem is that we assume starts with is searching headings, zebra is returning keywords that start with i.e. starts with O returns O'Reilly but also Duke of Argyll because both contain keywords starting with o. As I posted earlier I suspect a search tool which is keyword oriented is possibly not the best tool for authority headings. Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com
On Thu, Oct 24, 2013 at 07:44:09AM -0700, Galen Charlton wrote:
Deprecating non-QueryParser mode ---------------------------------------------------
Removing the non-QueryParser code would go a long way towards easing the rewrite of C4::Search.
I find after switching to use QueryParser searches return no hits. I've not searched very deeply for a cause. But I'm a bit sceptical that its ready for prime time yet Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com
Hi, Just a question on this topic: Some time ago I tested the SRU server on ports 9998/9999 with Zebra/DOM, and at that time it would not work (rightaway, or as quick as it did with Zebra/GRS). (But I did not spend much time on sorting it out btw.) Any chance that someone can confirm that this problem has been resolved? (Are you working with it?) I think we should not deprecate GRS if this still needs more attention.. Marcel ________________________________ Van: koha-devel-bounces@lists.koha-community.org [koha-devel-bounces@lists.koha-community.org] namens Galen Charlton [gmc@esilibrary.com] Verzonden: donderdag 24 oktober 2013 16:44 Aan: koha-devel@lists.koha-community.org Onderwerp: [Koha-devel] Default indexing and search options for 3.14 Hi, I would like to have a discussion regarding making certain options the default for new installations of Koha 3.14 and deprecating other options. Here's the list of proposals: Enable DOM mode by default for both bibs and authorities ----------------------------------------------------------------------------------- Now that MARC21, NORMARC, and UNIMARC have DOM indexing configurations available, this is now possible. Deprecating the GRS-1 filter ----------------------------------------- Giving that DOM has significant advantages of GRS-1 in its flexibility, and as the DOM filter would be required for non-MARC metadata support by Zebra, I propose that we announce a deprecation of the GRS-1 configuration with it to be removed by 3.16. I foresee several features coming down the pike for 3.16 that would require a reindexing during upgrade anyway, so switching to DOM for 3.16 would not impose a burden that wouldn't be present anyway. Enable QueryParser by default --------------------------------------------- Bug 10542 might be a blocker for this, and doing something about bug 10831 would be nice in order to do this. Deprecating non-QueryParser mode --------------------------------------------------- Removing the non-QueryParser code would go a long way towards easing the rewrite of C4::Search. Enable ICU by default ------------------------------- This was discussed during hackfest. One precondition I see is that bug 10729 would have to pass QA Regards, Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com<mailto:gmc@esilibrary.com> direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
DOM and ICU are definitely the newer technologies and have been built explicitly to fix a number of shortcomings in GRS-1 and chr. As Ian mentioned we have been slowly moving all our customers to using this combination, and admittedly it has been a relatively painful process whilst we iron out the various bugs. However, for MARC21 at least, most of these are now fixed, and trying to support both sets of configs is only going to be a burden in the future. Whilst we support the both DOM and GRS-1 we will inevitably keep getting the configurations are out of sync, and should we try to keep them in sync we will be limiting the DOM config with the limitations of GRS-1. My vote is to make the move to DOM with ICU, advocating that getting it right once is much easier than trying to maintain two disparate configurations. Martin Renvoize Software Engineer, PTFS Europe Ltd Content Management and Library Solutions Skype: Landline: 0203 286 8685 Mobile: 07725985636 http://www.ptfs-europe.com On 25 October 2013 09:10, Marcel de Rooy <M.de.Rooy@rijksmuseum.nl> wrote:
Hi,
Just a question on this topic:
Some time ago I tested the SRU server on ports 9998/9999 with Zebra/DOM, and at that time it would not work (rightaway, or as quick as it did with Zebra/GRS). (But I did not spend much time on sorting it out btw.) Any chance that someone can confirm that this problem has been resolved? (Are you working with it?) I think we should not deprecate GRS if this still needs more attention..
Marcel
------------------------------ *Van:* koha-devel-bounces@lists.koha-community.org [ koha-devel-bounces@lists.koha-community.org] namens Galen Charlton [ gmc@esilibrary.com] *Verzonden:* donderdag 24 oktober 2013 16:44 *Aan:* koha-devel@lists.koha-community.org *Onderwerp:* [Koha-devel] Default indexing and search options for 3.14
Hi,
I would like to have a discussion regarding making certain options the default for new installations of Koha 3.14 and deprecating other options. Here's the list of proposals:
Enable DOM mode by default for both bibs and authorities
-----------------------------------------------------------------------------------
Now that MARC21, NORMARC, and UNIMARC have DOM indexing configurations available, this is now possible.
Deprecating the GRS-1 filter -----------------------------------------
Giving that DOM has significant advantages of GRS-1 in its flexibility, and as the DOM filter would be required for non-MARC metadata support by Zebra, I propose that we announce a deprecation of the GRS-1 configuration with it to be removed by 3.16. I foresee several features coming down the pike for 3.16 that would require a reindexing during upgrade anyway, so switching to DOM for 3.16 would not impose a burden that wouldn't be present anyway.
Enable QueryParser by default ---------------------------------------------
Bug 10542 might be a blocker for this, and doing something about bug 10831 would be nice in order to do this.
Deprecating non-QueryParser mode ---------------------------------------------------
Removing the non-QueryParser code would go a long way towards easing the rewrite of C4::Search.
Enable ICU by default -------------------------------
This was discussed during hackfest. One precondition I see is that bug 10729 would have to pass QA
Regards,
Galen -- Galen Charlton Manager of Implementation Equinox Software, Inc. / The Open Source Experts email: gmc@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web: http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org & http://evergreen-ils.org
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
On Fri, Oct 25, 2013 at 08:10:11AM +0000, Marcel de Rooy wrote:
Some time ago I tested the SRU server on ports 9998/9999 with Zebra/DOM, and at that time it would not work
Anybody using SRU might want to take a look at indexdata's site. A new version of yaz has been released which supports a different (possibly incompatible) version of SRU. Not sure what implications this may have Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com
participants (5)
-
Colin Campbell -
Galen Charlton -
Ian Bays -
Marcel de Rooy -
Martin Renvoize