ElasticSearch for Koha - presenting the Koha Gruppo Italiano's project
The Koha Gruppo Italiano is proud to present a new project for the development of Koha. Recently the community has shown a big interest in substituting Zebra with a new search engine. How this is done -- whether Zebra is replaced or complemented, etc. -- will determine the workload and necessary financial support. The Koha Gruppo Italiano wants to promote the project through a crowd-funding campaign, sponsorships, events, endorsements etc. The Koha Gruppo Italiano would like to bring together the community of developers to determine the possibility of introducing the new search engine, to set up the work flow, and to collect the funding necessary for the project. On March 11, 2014 the project will be presented in Marseille at the Koha Hackfest for evaluation. Important software editors/producers working for Koha, and others interested in the project will be invited. The project will be presented by Stefano Bargioni, Deputy Director and System Librarian at the Library of the Pontificia Università della Santa Croce. The goal is to discuss solutions, and propose ideas in order to take initial decisions. The project and its progress can be followed through the website <http://kohagruppoitaliano.moonfruit.com>, on FB <http://www.facebook.com/KohaGruppoItaliano> and on Twitter <https://twitter.com/kohagruppoit>. We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example: - Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? - Should the effort focus on complementing Zebra or on its replacement? - Subdivide the workload. - Draft a road map. We are grateful to all developers who will be contributing to our project. We hope that this project is seen favorably within the community, and specifically wish that the Librarian community worldwide see the benefits of this project and would be willing to assist us with finding financial support. Kindest regards, Franziska Wallner, Stefano Bargioni, Sebastian Hierl, Juan Diego Ramirez
It's great to see such an interest in putting ElasticSearch into Koha. May I suggest you try to catch up with Chris Cormack.. he stated in the 3.16 Architectural Goals mail thread on this list that he was already working on integrating ElasticSearch (some Catalyst with Bywater collaboration I believe), visible here: http://git.catalyst.net.nz/gw?p=koha.git;a=shortlog;h=refs/heads/elastic_sea... I'm sure he'll pipe up by himself soon enough.. but it would be terrible to see duplicated effort.. on the flip side it would be great to see some more collaboration and funding for the developers to be able to spend more time on getting this right. Martin Renvoize Software Engineer, PTFS Europe Ltd Content Management and Library Solutions Skype: Landline: 0203 286 8685 Mobile: 07725985636 http://www.ptfs-europe.com On 5 February 2014 16:34, Stefano Bargioni <bargioni@pusc.it> wrote:
The Koha Gruppo Italiano is proud to present a new project for the development of Koha.
Recently the community has shown a big interest in substituting Zebra with a new search engine. How this is done -- whether Zebra is replaced or complemented, etc. -- will determine the workload and necessary financial support.
The Koha Gruppo Italiano wants to promote the project through a crowd-funding campaign, sponsorships, events, endorsements etc.
The Koha Gruppo Italiano would like to bring together the community of developers to determine the possibility of introducing the new search engine, to set up the work flow, and to collect the funding necessary for the project. On March 11, 2014 the project will be presented in Marseille at the Koha Hackfest for evaluation. Important software editors/producers working for Koha, and others interested in the project will be invited. The project will be presented by Stefano Bargioni, Deputy Director and System Librarian at the Library of the Pontificia Università della Santa Croce. The goal is to discuss solutions, and propose ideas in order to take initial decisions.
The project and its progress can be followed through the website < http://kohagruppoitaliano.moonfruit.com>, on FB < http://www.facebook.com/KohaGruppoItaliano> and on Twitter < https://twitter.com/kohagruppoit>.
We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example:
- Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? - Should the effort focus on complementing Zebra or on its replacement? - Subdivide the workload. - Draft a road map.
We are grateful to all developers who will be contributing to our project. We hope that this project is seen favorably within the community, and specifically wish that the Librarian community worldwide see the benefits of this project and would be willing to assist us with finding financial support.
Kindest regards, Franziska Wallner, Stefano Bargioni, Sebastian Hierl, Juan Diego Ramirez
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi Martin, thanks a lot for your message. ElasticSearch might not be the best search engine for Koha. However, we know that some developers prefer it to Solr. In order to determine this, the Hackfest in Marseille will be important. Otherwise we could conduct a survey. Of course, Chris' opinion is welcome...! Both on the project itself and on the complementation / substitution of Zebra. The Koha Gruppo Italiano hopes that Chris and other developers will participate in the Koha Hackfest. Bye Stefano On 05/feb/2014, at 17.59, Martin Renvoize wrote:
It's great to see such an interest in putting ElasticSearch into Koha.
May I suggest you try to catch up with Chris Cormack.. he stated in the 3.16 Architectural Goals mail thread on this list that he was already working on integrating ElasticSearch (some Catalyst with Bywater collaboration I believe), visible here: http://git.catalyst.net.nz/gw?p=koha.git;a=shortlog;h=refs/heads/elastic_sea...
I'm sure he'll pipe up by himself soon enough.. but it would be terrible to see duplicated effort.. on the flip side it would be great to see some more collaboration and funding for the developers to be able to spend more time on getting this right.
Martin Renvoize Software Engineer, PTFS Europe Ltd Content Management and Library Solutions Skype: Landline: 0203 286 8685 Mobile: 07725985636
On 5 February 2014 16:34, Stefano Bargioni <bargioni@pusc.it> wrote: The Koha Gruppo Italiano is proud to present a new project for the development of Koha.
Recently the community has shown a big interest in substituting Zebra with a new search engine. How this is done -- whether Zebra is replaced or complemented, etc. -- will determine the workload and necessary financial support.
The Koha Gruppo Italiano wants to promote the project through a crowd-funding campaign, sponsorships, events, endorsements etc.
The Koha Gruppo Italiano would like to bring together the community of developers to determine the possibility of introducing the new search engine, to set up the work flow, and to collect the funding necessary for the project. On March 11, 2014 the project will be presented in Marseille at the Koha Hackfest for evaluation. Important software editors/producers working for Koha, and others interested in the project will be invited. The project will be presented by Stefano Bargioni, Deputy Director and System Librarian at the Library of the Pontificia Università della Santa Croce. The goal is to discuss solutions, and propose ideas in order to take initial decisions.
The project and its progress can be followed through the website <http://kohagruppoitaliano.moonfruit.com>, on FB <http://www.facebook.com/KohaGruppoItaliano> and on Twitter <https://twitter.com/kohagruppoit>.
We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example:
- Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? - Should the effort focus on complementing Zebra or on its replacement? - Subdivide the workload. - Draft a road map.
We are grateful to all developers who will be contributing to our project. We hope that this project is seen favorably within the community, and specifically wish that the Librarian community worldwide see the benefits of this project and would be willing to assist us with finding financial support.
Kindest regards, Franziska Wallner, Stefano Bargioni, Sebastian Hierl, Juan Diego Ramirez
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
On 7/02/2014 12:26 am, "Stefano Bargioni" <bargioni@pusc.it> wrote:
Hi Martin, thanks a lot for your message. ElasticSearch might not be the best search
engine for Koha.
However, we know that some developers prefer it to Solr. In order to determine this, the Hackfest in Marseille will be important. Otherwise we could conduct a survey. Of course, Chris' opinion is welcome...! Both on the project itself and on the complementation / substitution of Zebra. The Koha Gruppo Italiano hopes that Chris and other developers will participate in the Koha Hackfest.
As Martin said, work is already well under way, we have Indexing and basic searching working now and working on browse. You can follow along on the branch Martin mentioned. We aren't removing Zebra, just adding Elasticsearch as an option. I'd love to participate in the hackfest, if you would like to fund my travel :-) Chris
Bye Stefano
On 05/feb/2014, at 17.59, Martin Renvoize wrote:
It's great to see such an interest in putting ElasticSearch into Koha.
May I suggest you try to catch up with Chris Cormack.. he stated in the 3.16 Architectural Goals mail thread on this list that he was already working on integrating ElasticSearch (some Catalyst with Bywater
collaboration I believe), visible here: http://git.catalyst.net.nz/gw?p=koha.git;a=shortlog;h=refs/heads/elastic_search >> >> I'm sure he'll pipe up by himself soon enough.. but it would be terrible to see duplicated effort.. on the flip side it would be great to see some more collaboration and funding for the developers to be able to spend more time on getting this right. >> >> Martin Renvoize >> Software Engineer, PTFS Europe Ltd >> Content Management and Library Solutions >> Skype: >> Landline: 0203 286 8685 >> Mobile: 07725985636 >> >> http://www.ptfs-europe.com >> >> >> On 5 February 2014 16:34, Stefano Bargioni <bargioni@pusc.it> wrote: >>> >>> The Koha Gruppo Italiano is proud to present a new project for the development of Koha. >>> >>> Recently the community has shown a big interest in substituting Zebra with a new search engine. How this is done -- whether Zebra is replaced or complemented, etc. -- will determine the workload and necessary financial support. >>> >>> The Koha Gruppo Italiano wants to promote the project through a crowd-funding campaign, sponsorships, events, endorsements etc. >>> >>> The Koha Gruppo Italiano would like to bring together the community of developers to determine the possibility of introducing the new search engine, to set up the work flow, and to collect the funding necessary for the project. On March 11, 2014 the project will be presented in Marseille at the Koha Hackfest for evaluation. Important software editors/producers working for Koha, and others interested in the project will be invited. The project will be presented by Stefano Bargioni, Deputy Director and System Librarian at the Library of the Pontificia Università della Santa Croce. The goal is to discuss solutions, and propose ideas in order to take initial decisions. >>> >>> The project and its progress can be followed through the website < http://kohagruppoitaliano.moonfruit.com>, on FB < http://www.facebook.com/KohaGruppoItaliano> and on Twitter < https://twitter.com/kohagruppoit>. >>> >>> We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example: >>> >>> - Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? >>> - Should the effort focus on complementing Zebra or on its replacement? >>> - Subdivide the workload. >>> - Draft a road map. >>> >>> We are grateful to all developers who will be contributing to our project. We hope that this project is seen favorably within the community, and specifically wish that the Librarian community worldwide see the benefits of this project and would be willing to assist us with finding financial support. >>> >>> Kindest regards, >>> Franziska Wallner, Stefano Bargioni, Sebastian Hierl, Juan Diego Ramirez >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel@lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >> >> > > > _______________________________________________ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/
Salvete! In general for your efforts: bravissimi! :D I feel that it's much needed.
We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example:
- Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? - Should the effort focus on complementing Zebra or on its replacement? - Subdivide the workload. - Draft a road map.
A long time ago, when animals could talk, it was hoped that whatever steps were taken in search would do so agnostically so that one could select ElasticSearch, Solr, Zebra, or whatever else they so chose. This hope was coupled by the realisation that this approach would probably take longer and involve more resources than just settling on one search to end them all, BUT would be offset in guaranteeing user freedom. This is olde, but might be useful for Solr discussions: http://wiki.koha-community.org/wiki/Switch_to_Solr_RFC#Concerns Realise that this improvement is in discussion, but quite high in difficulty in terms of help: http://wiki.koha-community.org/wiki/I_want_to_help And tooting me own horn and Jared's: http://wiki.koha-community.org/wiki/C_%26_P_Search_Rewrite_RFC Cheers, Brooke
I'd like to see some effort up front on defining what we want from search and how we evaluate what works. A couple of days ago I did a totally unscientific survey of a few uk public and academic libraries' opac searches. They'd all obviously spent money and resources on buying nice search products, but nearly all of them were mediocre at best in actually finding anything. Colin -- Colin Campbell Chief Software Engineer, PTFS Europe Limited Content Management and Library Solutions +44 (0) 800 756 6803 (phone) +44 (0) 7759 633626 (mobile) colin.campbell@ptfs-europe.com skype: colin_campbell2 http://www.ptfs-europe.com
Hi, Brooke, thanks for your reply. A long time ago, when zebras could talk... :-) We know that the most important decision is about adding a new SE or substituting Zebra. Anyway this decision belongs to the developers' community. Our task is to help the community, by trying to ensure a financial support for this project and then achieving this goal. We hope that the Koha Gruppo Italiano will start a new way to do big developments in Koha, by defining a group that leads the project in terms of communication and funding, and allowing the developers do their work as soon as possible. Additionally, we are aware that we will also spread the word of Koha in many other important environments. Best regards. Stefano, on behalf of the Koha Gruppo Italiano On 05/feb/2014, at 18.48, BWS Johnson wrote:
Salvete!
In general for your efforts: bravissimi! :D I feel that it's much needed.
We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example:
- Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? - Should the effort focus on complementing Zebra or on its replacement? - Subdivide the workload. - Draft a road map.
A long time ago, when animals could talk, it was hoped that whatever steps were taken in search would do so agnostically so that one could select ElasticSearch, Solr, Zebra, or whatever else they so chose. This hope was coupled by the realisation that this approach would probably take longer and involve more resources than just settling on one search to end them all, BUT would be offset in guaranteeing user freedom.
This is olde, but might be useful for Solr discussions: http://wiki.koha-community.org/wiki/Switch_to_Solr_RFC#Concerns
Realise that this improvement is in discussion, but quite high in difficulty in terms of help: http://wiki.koha-community.org/wiki/I_want_to_help
And tooting me own horn and Jared's: http://wiki.koha-community.org/wiki/C_%26_P_Search_Rewrite_RFC
Cheers, Brooke
Le 05/02/2014 18:48, BWS Johnson a écrit :
Salvete! In general for your efforts: bravissimi! :D I feel that it's much needed. agreed. A long time ago, when animals could talk, it was hoped that whatever steps were taken in search would do so agnostically so that one could select ElasticSearch, Solr, Zebra, or whatever else they so chose. This hope was coupled by the realisation that this approach would probably take longer and involve more resources than just settling on one search to end them all, BUT would be offset in guaranteeing user freedom.
This is olde, but might be useful for Solr discussions: http://wiki.koha-community.org/wiki/Switch_to_Solr_RFC#Concerns
Realise that this improvement is in discussion, but quite high in difficulty in terms of help: http://wiki.koha-community.org/wiki/I_want_to_help
And tooting me own horn and Jared's: http://wiki.koha-community.org/wiki/C_%26_P_Search_Rewrite_RFC You can also add this one, that was made during the KohaCon13 hackfest : http://wiki.koha-community.org/wiki/KohaCon13_searchengine_plan
-- Paul POULAIN - BibLibre http://www.biblibre.com Free & Open Source Softwares for libraries Koha, Drupal, Piwik, Jasper
We are very interested to hearing your first impressions, your suggestions etc. about this proposed project. Please respond to this e-mail if you have any comments. Probably the best contributions during this early phase will be the definition of the fundamentals. Example:
- Which search engine? - ElasticSearch, Solr or other (even if, at this time, the name of the initial project is ElasticSearch for Koha)? The best option / more long term is to add an abstraction layer, then we should be able to switch to any search engine. BibLibre made work around
Le 05/02/2014 17:34, Stefano Bargioni a écrit : that, before and during KohaCon12. We could not pursue the effort (lack of funding), but there's already a framework (that partially work with Solr)
- Should the effort focus on complementing Zebra or on its replacement? We should focus on letting libraries *choose*.
Why did the community decide not to *replace* zebra by Solr, 3 years ago ? Because Solr requires a lot of ressources, is hard to setup, require tomcat/java and all that stuff. OTOH, zebra (is a pain to change, but run out of the box with Koha and ) requires only a few hardware ressources. But for sure, zebra is a project not evolving anymore, that will be outdated sooner or later (and my opinion is that it is already outdated in terms of lemmatization, soundex, suggestions, facets, languages support,...) So let's rewrite Koha search layer, to have some abstraction, then let the user choose between zebra, Solr, ElasticSearch, Bing, or anything that will appear in the next years, ... [*] An important point = Koha *must* provide a z39.50 server. With zebra, it was built in. With Solr, we wrote something. What about ElasticSearch ?
- Subdivide the workload. - Draft a road map.
We are grateful to all developers who will be contributing to our project. We hope that this project is seen favorably within the community, and specifically wish that the Librarian community worldwide see the benefits of this project and would be willing to assist us with finding financial support. We had some discussion by mail when you asked if you could come to Marseille for the hackfest to talk about that. So you already know my opinion here = I'm *very* happy to see someone trying to take the lead on this matter !
[*] a troll is hidden in this list, can you find it ? -- Paul POULAIN - BibLibre http://www.biblibre.com Free & Open Source Softwares for libraries Koha, Drupal, Piwik, Jasper
participants (6)
-
BWS Johnson -
Chris Cormack -
Colin Campbell -
Martin Renvoize -
Paul Poulain -
Stefano Bargioni