[Bug 42179] New: Use Microsoft Graph for Sending Messages
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 Bug ID: 42179 Summary: Use Microsoft Graph for Sending Messages Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: brian.pichman@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Created attachment 196176 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196176&action=edit patch This changes outbound email delivery via Microsoft Graph using a host-managed Entra application and tenant consent from the Koha staff interface. The goal is to reduce partner setup complexity. The host configures one Microsoft 365 app in sysprefs, and the library admin completes tenant connection by clicking “Connect Microsoft 365” from the SMTP server configuration screen. Main changes: - Add Microsoft Graph as a delivery transport (left room to be expanded on later -- IE one click AWS SES set up for partners could be accomplished this way using API calls that can add domains and share back verification needs) for smtp_servers - Add host-level sysprefs for Graph client ID/client secret - Add tenant connection storage and connection workflow - Add shared connection scope so one tenant connection can be used for all libraries using the server - Add dedicated callback endpoint for Microsoft consent return - Route queued email delivery through Graph when configured Known setup requirements: - Redirect URI must be registered in Entra - App must have Microsoft Graph Mail.Send application permission - Admin consent must be granted This shouldn't conflict with this bug: 38338 as I'm recommending an entirely new layer within smtp_servers.pl. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 Brian Pichman <brian.pichman@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #196176|0 |1 is obsolete| | --- Comment #1 from Brian Pichman <brian.pichman@bywatersolutions.com> --- Created attachment 196177 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196177&action=edit Using API as a transport method with Microsoft Graph -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lisette@bywatersolutions.co | |m Assignee|koha-bugs@lists.koha-commun |brian.pichman@bywatersoluti |ity.org |ons.com Status|NEW |ASSIGNED --- Comment #2 from Lisette Scheer <lisette@bywatersolutions.com> --- (Not sure if this is supposed to be at Needs Signoff? If it is, please provide a test plan) Your patch has the author listed as John Doe. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 --- Comment #3 from Owen Leonard <oleonard@myacpl.org> --- Working directly with a third-party commercial service seems to me like something that should be a plugin. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 --- Comment #4 from Brian Pichman <brian.pichman@bywatersolutions.com> --- I think, ultimately, this would need to be put within the SMTP space -- since Microsoft is no longer allowing SMTP connections to send email. I'm open to other ideas, but the recommended path is using Microsoft Graph or OAuth -- both of which require a decent lift on IT teams to set up and use to work with Koha. The method I proposed provides a one-click button for library staff to email. In other words, SMTP is dying from providers like Microsoft for sending, and they want to force people to use APIs (Microsoft Graph) or OAuth. I wouldn't be surprised if AWS follows the same path. Sending from the AWS SES API is much faster and more reliable than via the SMTP Route. https://stackoverflow.com/questions/30721611/amazon-simple-email-service-ses.... There's a decent discussion here In talking with others, this may be a larger discussion too -- as calling this SMTP Delivery is a bit incorrect and probably needs to be renamed as Message Delivery or something more standard ... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #5 from David Cook <dcook@prosentient.com.au> --- (In reply to Brian Pichman from comment #4)
I think, ultimately, this would need to be put within the SMTP space -- since Microsoft is no longer allowing SMTP connections to send email. I'm open to other ideas, but the recommended path is using Microsoft Graph or OAuth -- both of which require a decent lift on IT teams to set up and use to work with Koha.
I think that you mean SMTP Plain/Login auth. When using XOAuth2 for auth, it's still SMTP. Microsoft is definitely still using SMTP connections to send email.
The method I proposed provides a one-click button for library staff to email.
I'm curious to hear more on this one. It seems to me that both the API and SMTP XOAUTH2 would need the same app registration in Microsoft Entra ID in order to work. Indeed in the Koha::SMTP::Server change, using the Microsoft Graph API is also using the OAuth client credentials grant for the authentication step (just like Olivier's code in bug 38338).
In other words, SMTP is dying from providers like Microsoft for sending, and they want to force people to use APIs (Microsoft Graph) or OAuth. I wouldn't be surprised if AWS follows the same path. Sending from the AWS SES API is much faster and more reliable than via the SMTP Route. https://stackoverflow.com/questions/30721611/amazon-simple-email-service-ses... should-i-use-smtp-interface-or-ses-api. There's a decent discussion here
I'd say this is a mischaracterisation of the situation. SMTP isn't going anywhere. But... I think using APIs to send emails is another good option for sure.
In talking with others, this may be a larger discussion too -- as calling this SMTP Delivery is a bit incorrect and probably needs to be renamed as Message Delivery or something more standard ...
Yeah, I think definitely more thought and discussion required. At the moment, we link libraries to SMTP servers, but yeah it would probably need a rebrand. I like the overall idea though. I think Owen makes a good point about making it pluggable as well. When it comes to APIs, different providers (like Microsoft, AWS, others) are likely to have different APIs. Note also that someone with report permissions could easily lookup the Microsoft365GraphClientId and Microsoft365GraphClientSecret. I don't think we have a good way of storing secrets in system preferences at the moment. (While we do store secrets in them, I think moving forward we always want to be striving to do better.) Another note on that... I notice a new "svc/api_delivery/callback" file, but we're not adding any new files to the "svc" API. -- Going back to the AWS SES API... take a look at https://metacpan.org/pod/Email::Sender::Transport::SES. Tomas actually thought about doing the same thing with XOAUTH2: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38338#c52 Perhaps smtp_servers should be "email_transports". Hmm... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 Olivier Hubert <olivier.hubert@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |olivier.hubert@inlibro.com --- Comment #6 from Olivier Hubert <olivier.hubert@inlibro.com> --- (In reply to David Cook from comment #5)
Note also that someone with report permissions could easily lookup the Microsoft365GraphClientId and Microsoft365GraphClientSecret. I don't think we have a good way of storing secrets in system preferences at the moment. (While we do store secrets in them, I think moving forward we always want to be striving to do better.)
From what I understand of Report.pm' FORBIDDEN_COLUMN_MATCHES, any column with
the word "secret" (case-insensitive) should be intercepted and never disclosed by any report. It's why I did not change anything at that level after adding in patch on bz38338. I'm unsure about the ClientId though. Is it considered sensitive information? Should it also be filtered by Report.pm? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to Olivier Hubert from comment #6)
(In reply to David Cook from comment #5)
Note also that someone with report permissions could easily lookup the Microsoft365GraphClientId and Microsoft365GraphClientSecret. I don't think we have a good way of storing secrets in system preferences at the moment. (While we do store secrets in them, I think moving forward we always want to be striving to do better.)
From what I understand of Report.pm' FORBIDDEN_COLUMN_MATCHES, any column with the word "secret" (case-insensitive) should be intercepted and never disclosed by any report. It's why I did not change anything at that level after adding in patch on bz38338.
I'm unsure about the ClientId though. Is it considered sensitive information? Should it also be filtered by Report.pm?
So the column name would be "variable" in this case, so that forbidden feature wouldn't come into play. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38338 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 David Liddle <david_liddle@sil.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david_liddle@sil.org --- Comment #8 from David Liddle <david_liddle@sil.org> --- (In reply to Brian Pichman from comment #4)
I think, ultimately, this would need to be put within the SMTP space -- since Microsoft is no longer allowing SMTP connections to send email. I'm open to other ideas, but the recommended path is using Microsoft Graph or OAuth -- both of which require a decent lift on IT teams to set up and use to work with Koha.
The method I proposed provides a one-click button for library staff to email.
In other words, SMTP is dying from providers like Microsoft for sending, and they want to force people to use APIs (Microsoft Graph) or OAuth. I wouldn't be surprised if AWS follows the same path. Sending from the AWS SES API is much faster and more reliable than via the SMTP Route. https://stackoverflow.com/questions/30721611/amazon-simple-email-service-ses... should-i-use-smtp-interface-or-ses-api. There's a decent discussion here
In talking with others, this may be a larger discussion too -- as calling this SMTP Delivery is a bit incorrect and probably needs to be renamed as Message Delivery or something more standard ...
I just want to mention that Microsoft has not completely killed off SMTP authentication. It is still available through Azure Communications Services Email. See the following: Overview of Azure Communication Services email https://learn.microsoft.com/en-us/azure/communication-services/concepts/emai... Azure Communication Services pricing https://azure.microsoft.com/en-us/pricing/details/communication-services/ Before leaving a position in which I managed a M365 tenant, I converted all systems and services using SMTP AUTH from regular M365 accounts to ACS configurations, and it worked well. It was not difficult to set up, but one would need to set a reminder to rotate the client secrets. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42179 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=42870 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org