[Bug 21210] New: Elasticsearch based SRU server
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Bug ID: 21210 Summary: Elasticsearch based SRU server Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: new feature Priority: P5 - low Component: Z39.50 / SRU / OpenSearch Servers Assignee: koha-bugs@lists.koha-community.org Reporter: joonas.kylmala@helsinki.fi QA Contact: testopia@bugs.koha-community.org CC: ere.maijala@helsinki.fi, m.de.rooy@rijksmuseum.nl There is no SRU server yet that works with Koha's Elasticsearch index. Without a Elasticsearch based SRU server one would have to run Zebra alongside Elasticsearch and that has it downsides like added complexity and non-realtime updates to the Zebra index (because the index will be only updated to Elasticsearch on biblio updates). There has been some work on the Elasticsearch SRU server: https://github.com/NatLibFi/Koha/blob/ks-rumble/misc/sru_server.pl. Maybe that could be adapted to the current version of Koha? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #1 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Created attachment 77787 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77787&action=edit Bug 21210: Implement Elasticsearch based SRU server I combined the previous work by Ere together and based it on the latest master. I would like to know if you have any opinions about storing the "cql_mapping" in /etc/sru_server/catmandu.yml. Should we just treat this file in a way that we know everybody must tweak it to their liking and just do a configuration that has most basic fields like author and title? General comments about the implementation are also welcome. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |ere.maijala@helsinki.fi |ity.org | Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #2 from Ere Maijala <ere.maijala@helsinki.fi> --- Thanks for doing this, Joonas. I'll be happy to do any changes that may be needed to make everything work. I suppose we may need to put the default config file somewhere else than /etc since that caused trouble with other ES files too. In a more perfect world I think we should have a search backend agnostic SRU server. That's a fair bit more work to implement, though. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Johanna Räisä <johanna.raisa@mikkeli.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |johanna.raisa@mikkeli.fi --- Comment #3 from Johanna Räisä <johanna.raisa@mikkeli.fi> --- This is missing few Perl dependecies. Please include Dancer and Dancer::Plugin::Catmandu::SRU. Otherwise first impression is good. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Johanna Räisä from comment #3)
This is missing few Perl dependecies. Please include Dancer and Dancer::Plugin::Catmandu::SRU. Otherwise first impression is good.
As Dancer is a whole framework I think we should probably discuss in a dev meeting. We've had similar discussions about including other frameworks in the past (I remember Moose and Moo) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |dependency -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no --- Comment #5 from Magnus Enger <magnus@libriotech.no> --- If there are site-specific config files those should probably go in /etc/koha/sites/<instance>/, at least for Debian installs? (In reply to Katrin Fischer from comment #4)
(In reply to Johanna Räisä from comment #3)
This is missing few Perl dependecies. Please include Dancer and Dancer::Plugin::Catmandu::SRU. Otherwise first impression is good.
As Dancer is a whole framework I think we should probably discuss in a dev meeting. We've had similar discussions about including other frameworks in the past (I remember Moose and Moo)
Dancer is a web-framework, more comparable to Mojolicious, I think. And we are already using Mojo for the REST API. Would rewriting the SRU server to use Mojo be doable? An SRU server for Elasticsearch would definitely be awesome, so thanks for working on this! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #6 from Ere Maijala <ere.maijala@helsinki.fi> ---
From what I gathered from the discussion in Kohacon, this would be only a stop-gap solution and we should rather aim for something more search backend agnostic and support both SRU and Z39.50. I may have some code that might be useful, but I need a bit of time to work on it.
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Supporting Z39.50 and SRU on Elastic side is something we really need imo to abandon Zebra completely. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #8 from Ere Maijala <ere.maijala@helsinki.fi> --- Note also that AFAIK we can't use Plack for the server since Z39.50 is not HTTP-based. I'm leaning towards something like Net::Z3950::SimpleServer where the daemon would work in a similar fashion to the SIP server in Koha. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #9 from Magnus Enger <magnus@libriotech.no> --- (In reply to Ere Maijala from comment #8)
Note also that AFAIK we can't use Plack for the server since Z39.50 is not HTTP-based. I'm leaning towards something like Net::Z3950::SimpleServer where the daemon would work in a similar fashion to the SIP server in Koha.
Do we need to have a Z39.50 server? At least in Norway that would not be a requirement, I think. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Magnus Enger from comment #9)
(In reply to Ere Maijala from comment #8)
Note also that AFAIK we can't use Plack for the server since Z39.50 is not HTTP-based. I'm leaning towards something like Net::Z3950::SimpleServer where the daemon would work in a similar fashion to the SIP server in Koha.
Do we need to have a Z39.50 server? At least in Norway that would not be a requirement, I think.
Yeah we still need it. Think about all copy cataloging that still goes thru Z39.50 and not SRU. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=13937 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |13937 --- Comment #11 from Ere Maijala <ere.maijala@helsinki.fi> --- Since there's already work done on a SimpleServer in bug 13937, it would probably make sense to use it as the basis. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13937 [Bug 13937] Add a Z39.50 daemon that can inject item status MARC subfields -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21210 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE Status|ASSIGNED |RESOLVED --- Comment #12 from Ere Maijala <ere.maijala@helsinki.fi> --- I'm marking this duplicate of bug 13937 since the implementation there will work with any search engine and doesn't need Dancer. I still need to make sure we have all the index definitions that were available here, but the current version in bug 13937 is already functional. *** This bug has been marked as a duplicate of bug 13937 *** -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org