Best practices for using Koha APIs server-side by Koha
Hi all, I mostly see Koha APIs being used client-side by Koha using cookie authentication. I know we have OAuth2 and Basic Auth that can be tied to a Koha user, and used by a third-party system, which is great. However, what about when we have server-side Koha components that want to use Koha APIs? (I suppose you could set up a user for that component, but it could always be deleted by an overzealous librarian.) Or should we only be using a Message Broker (like RabbitMQ) for passing messages among Koha server-side components? (That would require more of a service oriented architecture of course, which we don't have in place, while we do have a HTTP API in place.) What are your thoughts? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595
Maybe I'm lacking in imagination, but I can only think of two cases where a Koha to Koha api would be useful: scheduled tasks (cron, et. all), and command line scripts. I think in both of these cases it's feasible to avoid auth entirely since it is basically jumping right into Koha at the control level (assuming an MCV approach). As someone who has a deployment of Koha heavily dependent on IndependentBranches mode, I find I have to watch for %userenv{branch}, but that's for rare cases in which branch can't be taken from the data being operated on. The only other thing I can think of would be something like notices wanting to take a user's preferred language, and pull a language specific template (do we even do that at this point?). Please enlighten me on your use cases, I'm probably missing something. On Fri, Aug 7, 2020 at 5:09 AM <dcook@prosentient.com.au> wrote:
Hi all,
I mostly see Koha APIs being used client-side by Koha using cookie authentication.
I know we have OAuth2 and Basic Auth that can be tied to a Koha user, and used by a third-party system, which is great.
However, what about when we have server-side Koha components that want to use Koha APIs? (I suppose you could set up a user for that component, but it could always be deleted by an overzealous librarian.)
Or should we only be using a Message Broker (like RabbitMQ) for passing messages among Koha server-side components? (That would require more of a service oriented architecture of course, which we don’t have in place, while we do have a HTTP API in place.)
What are your thoughts?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Michael Hafen Washington County School District Technology Department Systems Analyst
Hi Michael, Another case where a Koha to Koha API would be useful is in the use of additional service (or microservices). At the moment, the SIP server is closely coupled with Koha, but it could just as easily be a separate component that used HTTP APIs (rather than internal Perl APIs) to work with Koha. You could argue that you could avoid auth (although I don’t understand your interpretation of MCV in this case), but that would be out of convenience rather than security. You could argue that you just protect the edges of your host (via a firewall for instance), but what about if that service isn’t running on the same host? Then you need to open a hole in that firewall. Then how do you protect that hole? Just using IP addresses isn’t going to work very well. You’ll want to have authentication and authorization, especially when your application is part of a larger multi-vendor cloud ecosystem for instance. The particular use case I have in mind is for a OAI-PMH harvester service. There could be 1 OAI-PMH harvester servicing many Koha instances. It could be running in its own local Docker container or its own local virtual machine or even as a remote Internet service controlled by a different organisation. It’s only relation to Koha is in terms of accepting JSON serialized harvesting requests and sending MARCXML records for Koha to import. For integrating the services, we could use point-to-point (P2P) communication via the HTTP APIs. That’s pretty much the only option we have at the moment. However, Koha will be adding RabbitMQ as a Message Broker (for a hub-and-spoke model), so Koha and the other services could have their own credentials to authenticate against the Message Broker, and use it for communication. Both models have their pros and cons. My current plan is to use https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26170 to provide more stable Koha users and https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25905 to provide an OAI-PMH ingestion endpoint to Koha. Then I’ll implement the OAI-PMH harvester separate from Koha (with a Koha plugin initially to send OAI-PMH harvesting requests to the OAI-PMH harvester service), which sends downloaded records to the Koha API for the Koha that requested the harvest. This approach will have some friction though. I’ll have to add a Koha user, share the Koha user credentials to the OAI-PMH service, register the Koha instance in the OAI-PMH service, and share the OAI-PMH credentials with Koha. That process could be unnecessary using a Message Broker. Another option would be to set up single sign on (SSO) with P2P where Koha and the OAI-PMH service delegate AuthN (and even potentially AuthZ) to an identity provider. That would make things smoother but also more complicated. If I wanted to more tightly integrate with Koha, I suppose I could create a koha-oaipmh CLI tool which automates the user creation and credential sharing I suppose. The other benefit of using the Message Broker is that you don’t have to manage IP addresses and ports. I should look at the koha-sip tool again, but I don’t know how it handles setting up different ports for each different Koha instance. In any case, I’m keen to get input from others on system design. On one hand, I’d like something that easily works for everyone, and on the other hand I’d like something that works really well. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 From: Michael Hafen (TECH) <michael.hafen@washk12.org> Sent: Saturday, 8 August 2020 2:46 AM To: David Cook <dcook@prosentient.com.au> Cc: Koha Devel <koha-devel@lists.koha-community.org>; Kyle Hall <kyle@bywatersolutions.com>; Tomas Cohen Arazi <tomascohen@theke.io> Subject: Re: [Koha-devel] Best practices for using Koha APIs server-side by Koha Maybe I'm lacking in imagination, but I can only think of two cases where a Koha to Koha api would be useful: scheduled tasks (cron, et. all), and command line scripts. I think in both of these cases it's feasible to avoid auth entirely since it is basically jumping right into Koha at the control level (assuming an MCV approach). As someone who has a deployment of Koha heavily dependent on IndependentBranches mode, I find I have to watch for %userenv{branch}, but that's for rare cases in which branch can't be taken from the data being operated on. The only other thing I can think of would be something like notices wanting to take a user's preferred language, and pull a language specific template (do we even do that at this point?). Please enlighten me on your use cases, I'm probably missing something. On Fri, Aug 7, 2020 at 5:09 AM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Hi all, I mostly see Koha APIs being used client-side by Koha using cookie authentication. I know we have OAuth2 and Basic Auth that can be tied to a Koha user, and used by a third-party system, which is great. However, what about when we have server-side Koha components that want to use Koha APIs? (I suppose you could set up a user for that component, but it could always be deleted by an overzealous librarian.) Or should we only be using a Message Broker (like RabbitMQ) for passing messages among Koha server-side components? (That would require more of a service oriented architecture of course, which we don’t have in place, while we do have a HTTP API in place.) What are your thoughts? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ -- Michael Hafen Washington County School District Technology Department Systems Analyst
You've got me there, I wasn't even thinking about third-party services like OAI-PMH. My comment about avoiding auth is only really feasible for first-party or locally developed cli scripts. (I could say second-party there, but that's just silly.) Third-party services definitely need to authen, and probably authiz too. A message broker would be handy for that, where you probably have one set of credentials that are shared by any services talking on that queue. oAuth can do that too, but you'd have to build a way for the client id and secret to be stored by both services in lieu of a username and password. Which seems basically the same to me as having a username and password. On the one hand we'd have to build a connector between the message broker and the Koha api's. On the other hand we'd have to build oAuth access token handling into the existing api's. (where the appropriate api's already exist.) On Sun, Aug 9, 2020 at 7:00 PM <dcook@prosentient.com.au> wrote:
Hi Michael,
Another case where a Koha to Koha API would be useful is in the use of additional service (or microservices). At the moment, the SIP server is closely coupled with Koha, but it could just as easily be a separate component that used HTTP APIs (rather than internal Perl APIs) to work with Koha.
You could argue that you could avoid auth (although I don’t understand your interpretation of MCV in this case), but that would be out of convenience rather than security. You could argue that you just protect the edges of your host (via a firewall for instance), but what about if that service isn’t running on the same host? Then you need to open a hole in that firewall. Then how do you protect that hole? Just using IP addresses isn’t going to work very well. You’ll want to have authentication and authorization, especially when your application is part of a larger multi-vendor cloud ecosystem for instance.
The particular use case I have in mind is for a OAI-PMH harvester service. There could be 1 OAI-PMH harvester servicing many Koha instances. It could be running in its own local Docker container or its own local virtual machine or even as a remote Internet service controlled by a different organisation. It’s only relation to Koha is in terms of accepting JSON serialized harvesting requests and sending MARCXML records for Koha to import.
For integrating the services, we could use point-to-point (P2P) communication via the HTTP APIs. That’s pretty much the only option we have at the moment. However, Koha will be adding RabbitMQ as a Message Broker (for a hub-and-spoke model), so Koha and the other services could have their own credentials to authenticate against the Message Broker, and use it for communication. Both models have their pros and cons.
My current plan is to use https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26170 to provide more stable Koha users and https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25905 to provide an OAI-PMH ingestion endpoint to Koha. Then I’ll implement the OAI-PMH harvester separate from Koha (with a Koha plugin initially to send OAI-PMH harvesting requests to the OAI-PMH harvester service), which sends downloaded records to the Koha API for the Koha that requested the harvest. This approach will have some friction though. I’ll have to add a Koha user, share the Koha user credentials to the OAI-PMH service, register the Koha instance in the OAI-PMH service, and share the OAI-PMH credentials with Koha. That process could be unnecessary using a Message Broker. Another option would be to set up single sign on (SSO) with P2P where Koha and the OAI-PMH service delegate AuthN (and even potentially AuthZ) to an identity provider. That would make things smoother but also more complicated.
If I wanted to more tightly integrate with Koha, I suppose I could create a koha-oaipmh CLI tool which automates the user creation and credential sharing I suppose.
The other benefit of using the Message Broker is that you don’t have to manage IP addresses and ports. I should look at the koha-sip tool again, but I don’t know how it handles setting up different ports for each different Koha instance.
In any case, I’m keen to get input from others on system design. On one hand, I’d like something that easily works for everyone, and on the other hand I’d like something that works really well.
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
*From:* Michael Hafen (TECH) <michael.hafen@washk12.org> *Sent:* Saturday, 8 August 2020 2:46 AM *To:* David Cook <dcook@prosentient.com.au> *Cc:* Koha Devel <koha-devel@lists.koha-community.org>; Kyle Hall < kyle@bywatersolutions.com>; Tomas Cohen Arazi <tomascohen@theke.io> *Subject:* Re: [Koha-devel] Best practices for using Koha APIs server-side by Koha
Maybe I'm lacking in imagination, but I can only think of two cases where a Koha to Koha api would be useful: scheduled tasks (cron, et. all), and command line scripts. I think in both of these cases it's feasible to avoid auth entirely since it is basically jumping right into Koha at the control level (assuming an MCV approach). As someone who has a deployment of Koha heavily dependent on IndependentBranches mode, I find I have to watch for %userenv{branch}, but that's for rare cases in which branch can't be taken from the data being operated on. The only other thing I can think of would be something like notices wanting to take a user's preferred language, and pull a language specific template (do we even do that at this point?).
Please enlighten me on your use cases, I'm probably missing something.
On Fri, Aug 7, 2020 at 5:09 AM <dcook@prosentient.com.au> wrote:
Hi all,
I mostly see Koha APIs being used client-side by Koha using cookie authentication.
I know we have OAuth2 and Basic Auth that can be tied to a Koha user, and used by a third-party system, which is great.
However, what about when we have server-side Koha components that want to use Koha APIs? (I suppose you could set up a user for that component, but it could always be deleted by an overzealous librarian.)
Or should we only be using a Message Broker (like RabbitMQ) for passing messages among Koha server-side components? (That would require more of a service oriented architecture of course, which we don’t have in place, while we do have a HTTP API in place.)
What are your thoughts?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
--
Michael Hafen
Washington County School District Technology Department
Systems Analyst
-- Michael Hafen Washington County School District Technology Department Systems Analyst
participants (2)
-
dcook＠prosentient.com.au -
Michael Hafen (TECH)