[Bug 33964] New: Use Email::Sender::Transport::SMTP::Persistent for sending email
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Bug ID: 33964 Summary: Use Email::Sender::Transport::SMTP::Persistent for sending email Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: kyle@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=30013 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |kyle@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #1 from Kyle M Hall <kyle@bywatersolutions.com> --- As described in bug 30013, some outgoing SMTP services ( such as Gmail ) do not like Koha's current behavior of initiating a new connection for each email sent. If we switch from Email::Sender::Transport::SMTP to Email::Sender::Transport::SMTP::Persistent and store the object for the duration of the message queue processing, this should solve that issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #2 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 152211 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152211&action=edit Bug 33964: Use Email::Sender::Transport::SMTP::Persistent for sending email As described in bug 30013, some outgoing SMTP services ( such as Gmail ) do not like Koha's current behavior of initiating a new connection for each email sent. If we switch from Email::Sender::Transport::SMTP to Email::Sender::Transport::SMTP::Persistent and store the object for the duration of the message queue processing, this should solve that issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Is this still in progress or should it be needs signoff? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff --- Comment #4 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to David Cook from comment #3)
Is this still in progress or should it be needs signoff?
Yes please! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Eric Balluto <eric.balluto@univ-lyon2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eric.balluto@univ-lyon2.fr Status|Needs Signoff |Failed QA --- Comment #5 from Eric Balluto <eric.balluto@univ-lyon2.fr> --- Needs test plan -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #6 from Kyle M Hall <kyle@bywatersolutions.com> --- Test Plan: 1) Verify your server can successfully send email via SMTP 2) Install Email::Sender::Transport::SMTP::Persistent ( sudo cpanm Email::Sender::Transport::SMTP::Persistent ) 3) Apply this patch 4) Restart all the things! 5) Enqueue some email message 6) Run process_message_queue.pl 7) Verify the emails were sent! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- I'm keen to QA this one, so I'm just waiting for someone else to sign it off. We've been setting up a lot more SMTP lately. I'm curious to compare the speed of Email::Sender::Transport::SMTP vs Email::Sender::Transport::SMTP::Persistent as well. I suspect that the latter will be much nicer for performance. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Sam Lau <samalau@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Sam Lau <samalau@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152211|0 |1 is obsolete| | --- Comment #8 from Sam Lau <samalau@gmail.com> --- Created attachment 153101 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153101&action=edit Bug 33964: Use Email::Sender::Transport::SMTP::Persistent for sending email As described in bug 30013, some outgoing SMTP services ( such as Gmail ) do not like Koha's current behavior of initiating a new connection for each email sent. If we switch from Email::Sender::Transport::SMTP to Email::Sender::Transport::SMTP::Persistent and store the object for the duration of the message queue processing, this should solve that issue. Signed-off-by: Sam Lau <samalau@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Sam Lau <samalau@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |samalau@gmail.com --- Comment #9 from Sam Lau <samalau@gmail.com> --- (In reply to David Cook from comment #7)
I'm keen to QA this one, so I'm just waiting for someone else to sign it off.
All yours! :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Sam Lau from comment #9)
(In reply to David Cook from comment #7)
I'm keen to QA this one, so I'm just waiting for someone else to sign it off.
All yours! :)
Thanks, Sam. I probably won't get to this until next week, so happy for someone else to swoop in, but failing that I'll take a look then. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- *** Bug 30013 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Just noting that libemail-sender-perl contains Email::Sender::Transport::SMTP::Persistent. So thats fine already. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #153101|0 |1 is obsolete| | --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 153723 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153723&action=edit Bug 33964: Use Email::Sender::Transport::SMTP::Persistent for sending email As described in bug 30013, some outgoing SMTP services ( such as Gmail ) do not like Koha's current behavior of initiating a new connection for each email sent. If we switch from Email::Sender::Transport::SMTP to Email::Sender::Transport::SMTP::Persistent and store the object for the duration of the message queue processing, this should solve that issue. Signed-off-by: Sam Lau <samalau@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 153724 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153724&action=edit Bug 33964: (QA follow-up) Handle absence of smtp server Resolve: Use of uninitialized value in hash element at /usr/share/koha/C4/Letters.pm line 1472. Use of uninitialized value in hash element at /usr/share/koha/C4/Letters.pm line 1473. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I was still looking for some kind of confirmation that it really works as advertised using one connection. Cannot easily read it from the logs: Here a short impression for two notices sent in one run: WITH PATCHES 2023-07-20 14:13:06 1qMUOv-0013tQ-UW <= noreply@DOMAIN H=(localhost.localdomain) [127.0.0.1] P=esmtp S=647 id=16898623860.4c95DE.253291@master 2023-07-20 14:13:06 1qMUOv-0013tQ-Vl <= noreply@DOMAIN H=(localhost.localdomain) [127.0.0.1] P=esmtp S=621 id=16898623861.2FAcE0.253291@master WITHOUT PATCHES 2023-07-20 14:14:43 1qMUQV-0013w2-Lp <= noreply@DOMAIN H=(localhost.localdomain) [127.0.0.1] P=esmtp S=650 id=16898624830.dE16C1Fba.253453@master 2023-07-20 14:14:44 1qMUQV-0013w5-N3 <= noreply@DOMAIN H=(localhost.localdomain) [127.0.0.1] P=esmtp S=624 id=16898624831.B6fAaA9BB.253453@master I only see that in the part with the patches the identifiers are closer to each other: both start with 1qMUOv-0013tQ While without patches the second part is already different. 1qMUQV-0013w2-Lp 1qMUQV-0013w5-N3 Not sure what this proves? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #16 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Exim told me this: When two or more messages are delivered down a single SMTP connection, an asterisk follows the remote IP address (and port if enabled) in the log lines for the second and subsequent messages. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #15)
I was still looking for some kind of confirmation that it really works as advertised using one connection.
You can use tcpdump to watch the SMTP port. I've set messaging preferences for check in and check out and generated 2 pending emails. I then manually run ./misc/cronjobs/process_message_queue.pl Without the patch, I'm seeing 2 different TCP connections (one for each email). With the patch, I'm seeing only 1 TCP connection and both emails go out. Looks good to me. Looking forward to this one going in. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #17)
(In reply to Marcel de Rooy from comment #15) You can use tcpdump to watch the SMTP port.
I was expecting an answer like that somehow :) When I look at the dump, I see SMTP traffic on the same port at the sending side while seeing two different ports without these patches. And I see Reset with the patches, not without. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #19 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #18)
(In reply to David Cook from comment #17)
(In reply to Marcel de Rooy from comment #15) You can use tcpdump to watch the SMTP port.
I was expecting an answer like that somehow :)
Hehe. I'm nothing if not predictable.
When I look at the dump, I see SMTP traffic on the same port at the sending side while seeing two different ports without these patches. And I see Reset with the patches, not without.
I did notice the resets but neglected to look into them further. It does suggest a TCP connection hangup rather than a graceful close. Maybe there's a follow-up necessary to gracefully close the persistent connections at the end of the calling function. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #20 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Module does this: if (my $client = $self->_cached_client) { return $client if eval { $client->reset; $client->ok; }; -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- Friday 5:05pm for me so I'll have to leave it with you folk for now. Happy Friday folks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #22 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- RM: Reset is expected. So go ahead. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |23.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #23 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #24 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 153857 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153857&action=edit Bug 33964: (QA follow-up) Syntax error in cpanfile Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #25 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 153869 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153869&action=edit Bug 33964: (QA follow-up) Remove library from cpanfile Email::Sender::Transport::SMTP::Persistent is part of the Email::Sender distribution, and a git diff on the repository doesn't show any difference. The patch author just took the number from MetaCPAN. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #26 from Fridolin Somers <fridolin.somers@biblibre.com> --- More enhancement no ? Can this be backported ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 --- Comment #27 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to Fridolin Somers from comment #26)
More enhancement no ? Can this be backported ?
This fixes brokenness with using Gmail as an SMTP server for Koha, so I would consider it a bug fix. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.11.00 |23.11.00,23.05.03 released in| | Status|Pushed to master |Pushed to stable --- Comment #28 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 23.05.x for 23.05.03 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33964 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|Pushed to stable |RESOLVED CC| |pedro.amorim@ptfs-europe.co | |m --- Comment #29 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Missing dependencies for 22.11.x. Not pushing. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org