[Bug 33360] New: process_message_queue: Should limit parameter look at recipient domain?
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Bug ID: 33360 Summary: process_message_queue: Should limit parameter look at recipient domain? Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: m.de.rooy@rijksmuseum.nl QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz Some email providers look at rates per minute, or day, etc. for sending or receiving mail. E.g. Gmail does not like incidental bursts. Our current limit parameter does just look at the total volume but could perhaps be made smarter than that? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149256 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149256&action=edit Bug 33360: Add Koha::Notice::Util for mail domain limits Includes unit test. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149257 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149257&action=edit Bug 33360: Extend Koha::Notice::Util with domain groups -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149258 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149258&action=edit Bug 33360: Incorporate Koha::Notice::Util in Letters:SendQueuedMessages -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|process_message_queue: |SendQueuedMessages: Improve |Should limit parameter look |limit behavior and add |at recipient domain? |domain limits -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- This is still work in progess. I stumbled over the where parameter now while converting stuff to a Koha object search. Not sure if we resolved such a thing somewhere else already. Could we automatically convert an old school SQL WHERE condition to correct Koha object/DBIx conditions? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=31453 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- <Joubu> marcelr: 33360 - see misc/cronjobs/membership_expiry.pl:199 $upcoming_mem_expires = $upcoming_mem_expires->search( \$where_literal ) if @where; -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #5)
<Joubu> marcelr: 33360 - see misc/cronjobs/membership_expiry.pl:199 $upcoming_mem_expires = $upcoming_mem_expires->search( \$where_literal ) if @where;
Yes, that would be enough! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Still patches forthcoming. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149256|0 |1 is obsolete| | Attachment #149257|0 |1 is obsolete| | Attachment #149258|0 |1 is obsolete| | --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149546 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149546&action=edit Bug 33360: Add Koha::Notice::Util for mail domain limits Ground work in new module. Includes unit test. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149547 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149547&action=edit Bug 33360: Extend Koha::Notice::Util with domain groups The functionality is extended a bit here by: [1] Allowing to combine domains in a shared count. [2] Only counting sent messages for the specified domains when it is really needed. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149548 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149548&action=edit Bug 33360: Incorporate Koha::Notice::Util in Letters:SendQueuedMessages Minimal adjustment to get the feature working. Will still be improved in follow-up. Test plan: [1] Add a domain limit to your koha-conf: <message_domain_limits> <domain><name>gmail.com</name><limit>1</limit> <unit>1h</unit></domain> </message_domain_limits> Replace gmail.com by the domain you want to send to. Restart all. [2] Disable cron job for message queue. [3] Generate two pending notices in the queue. [4] Run process_message_queue.pl [5] Check that one is sent, one is still pending. [6] Run again. [7] Check: nothing is sent, last one still pending. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149549 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149549&action=edit Bug 33360: Improving limit behavior in SendQueuedMessages Includes: [1] Do no longer use the limit in the sql selection, but apply the limit as a maximum for the number of sent messages. This is more practical in terms of not flooding your mail server (and the receiving ones). [2] Replace call of _get_unsent_messages by Koha objects search. [3] Do no longer report the number of messages seen, but report the number actually 'sent'. [4] If we lookup the to_address but we need to delay a message, save the email address for the next run. Also optimizing patron lookup in _send_message_by_email. [5} Add support for $where parameter in SendQueuedMessages. Used by process_message_queue.pl. [6] Handle scalar/array for letter_code and type parameter too. Test plan: [1] Adjust your domain limit settings in koha-conf. Use notices to three domains. Group A and B. <message_domain_limits> <domain><name>A</name><limit>1</limit><unit>1h</unit></domain> <domain><name>B/name><belongs_to>A</belongs_to></domain> <domain><name>C</name><limit>1</limit><unit>1h</unit></domain> </message_domain_limits> Replace A, B and C with your choice. Do not forget the belongs_to. Restart all. [2] Disable cron job for message queue. [3] Generate two notices for each domain A, B and C (in that order). Make sure that borrowers involved have correct address. [4] Run process_message_queue.pl -limit 1 [5] Check that one is sent for A, 5 pending. [6] Run process_message_queue.pl -limit 2 [7] Check that one is sent for C, 4 pending. [8] Run process_message_queue.pl (without limit). [9] Check that nothing is sent, 4 pending. [10] Check that message_queue.to_address is filled for those 4. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149550 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149550&action=edit Bug 33360: Adjust Letters.t for limit parameter and domain limits Includes: [1] Add a counter in the send_or_die mock. [2] Correct from processed count to sent count. [3] More extensive testing for limit parameter and domain limits. Test plan: Run t/db_dependent/Letters.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149551 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149551&action=edit Bug 33360: Update koha-conf.xml Test plan: We already tested it. Just look at changes in this patch. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 149552 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149552&action=edit Bug 33360: (follow-up) Adjust wording in verbose mode Koha reports 'sending mail' while we actually are still processing mail. It could still be delayed, fail, etc. Test plan: Run process_message_queue in verbose mode. If a message get processed, you should see: Processing email message to patron: 51 at /usr/share/koha/C4/Letters.pm line 1008. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Note for testers/QA: An adjusted version already runs in production on 21.11 for me. And was 'put to the test' with a large mailing already. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=30013 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149546|0 |1 is obsolete| | --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149645 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149645&action=edit Bug 33360: Add Koha::Notice::Util for mail domain limits Ground work in new module. Includes unit test. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149547|0 |1 is obsolete| | --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149646 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149646&action=edit Bug 33360: Extend Koha::Notice::Util with domain groups The functionality is extended a bit here by: [1] Allowing to combine domains in a shared count. [2] Only counting sent messages for the specified domains when it is really needed. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149548|0 |1 is obsolete| | --- Comment #18 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149647 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149647&action=edit Bug 33360: Incorporate Koha::Notice::Util in Letters:SendQueuedMessages Minimal adjustment to get the feature working. Will still be improved in follow-up. Test plan: [1] Add a domain limit to your koha-conf: <message_domain_limits> <domain><name>gmail.com</name><limit>1</limit> <unit>1h</unit></domain> </message_domain_limits> Replace gmail.com by the domain you want to send to. Restart all. [2] Disable cron job for message queue. [3] Generate two pending notices in the queue. [4] Run process_message_queue.pl [5] Check that one is sent, one is still pending. [6] Run again. [7] Check: nothing is sent, last one still pending. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149549|0 |1 is obsolete| | --- Comment #19 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149648 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149648&action=edit Bug 33360: Improving limit behavior in SendQueuedMessages Includes: [1] Do no longer use the limit in the sql selection, but apply the limit as a maximum for the number of sent messages. This is more practical in terms of not flooding your mail server (and the receiving ones). [2] Replace call of _get_unsent_messages by Koha objects search. [3] Do no longer report the number of messages seen, but report the number actually 'sent'. [4] If we lookup the to_address but we need to delay a message, save the email address for the next run. Also optimizing patron lookup in _send_message_by_email. [5} Add support for $where parameter in SendQueuedMessages. Used by process_message_queue.pl. [6] Handle scalar/array for letter_code and type parameter too. Test plan: [1] Adjust your domain limit settings in koha-conf. Use notices to three domains. Group A and B. <message_domain_limits> <domain><name>A</name><limit>1</limit><unit>1h</unit></domain> <domain><name>B/name><belongs_to>A</belongs_to></domain> <domain><name>C</name><limit>1</limit><unit>1h</unit></domain> </message_domain_limits> Replace A, B and C with your choice. Do not forget the belongs_to. Restart all. [2] Disable cron job for message queue. [3] Generate two notices for each domain A, B and C (in that order). Make sure that borrowers involved have correct address. [4] Run process_message_queue.pl -limit 1 [5] Check that one is sent for A, 5 pending. [6] Run process_message_queue.pl -limit 2 [7] Check that one is sent for C, 4 pending. [8] Run process_message_queue.pl (without limit). [9] Check that nothing is sent, 4 pending. [10] Check that message_queue.to_address is filled for those 4. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149550|0 |1 is obsolete| | --- Comment #20 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149649 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149649&action=edit Bug 33360: Adjust Letters.t for limit parameter and domain limits Includes: [1] Add a counter in the send_or_die mock. [2] Correct from processed count to sent count. [3] More extensive testing for limit parameter and domain limits. Test plan: Run t/db_dependent/Letters.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149551|0 |1 is obsolete| | --- Comment #21 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149650 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149650&action=edit Bug 33360: Update koha-conf.xml Test plan: We already tested it. Just look at changes in this patch. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149552|0 |1 is obsolete| | --- Comment #22 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 149651 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149651&action=edit Bug 33360: (follow-up) Adjust wording in verbose mode Koha reports 'sending mail' while we actually are still processing mail. It could still be delayed, fail, etc. Test plan: Run process_message_queue in verbose mode. If a message get processed, you should see: Processing email message to patron: 51 at /usr/share/koha/C4/Letters.pm line 1008. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no --- Comment #23 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- *** Bug 30013 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Needs Signoff |Signed Off --- Comment #24 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- All working well in testing, not obvious code issues. Signing Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149645|0 |1 is obsolete| | Attachment #149646|0 |1 is obsolete| | Attachment #149647|0 |1 is obsolete| | Attachment #149648|0 |1 is obsolete| | Attachment #149649|0 |1 is obsolete| | Attachment #149650|0 |1 is obsolete| | Attachment #149651|0 |1 is obsolete| | --- Comment #25 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149662 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149662&action=edit Bug 33360: Add Koha::Notice::Util for mail domain limits Ground work in new module. Includes unit test. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #26 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149663 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149663&action=edit Bug 33360: Extend Koha::Notice::Util with domain groups The functionality is extended a bit here by: [1] Allowing to combine domains in a shared count. [2] Only counting sent messages for the specified domains when it is really needed. Test plan: Run t/db_dependent/Koha/Notice_Util.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #27 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149664 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149664&action=edit Bug 33360: Incorporate Koha::Notice::Util in Letters:SendQueuedMessages Minimal adjustment to get the feature working. Will still be improved in follow-up. Test plan: [1] Add a domain limit to your koha-conf: <message_domain_limits> <domain><name>gmail.com</name><limit>1</limit> <unit>1h</unit></domain> </message_domain_limits> Replace gmail.com by the domain you want to send to. Restart all. [2] Disable cron job for message queue. [3] Generate two pending notices in the queue. [4] Run process_message_queue.pl [5] Check that one is sent, one is still pending. [6] Run again. [7] Check: nothing is sent, last one still pending. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #28 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149665 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149665&action=edit Bug 33360: Improving limit behavior in SendQueuedMessages Includes: [1] Do no longer use the limit in the sql selection, but apply the limit as a maximum for the number of sent messages. This is more practical in terms of not flooding your mail server (and the receiving ones). [2] Replace call of _get_unsent_messages by Koha objects search. [3] Do no longer report the number of messages seen, but report the number actually 'sent'. [4] If we lookup the to_address but we need to delay a message, save the email address for the next run. Also optimizing patron lookup in _send_message_by_email. [5} Add support for $where parameter in SendQueuedMessages. Used by process_message_queue.pl. [6] Handle scalar/array for letter_code and type parameter too. Test plan: [1] Adjust your domain limit settings in koha-conf. Use notices to three domains. Group A and B. <message_domain_limits> <domain><name>A</name><limit>1</limit><unit>1h</unit></domain> <domain><name>B/name><belongs_to>A</belongs_to></domain> <domain><name>C</name><limit>1</limit><unit>1h</unit></domain> </message_domain_limits> Replace A, B and C with your choice. Do not forget the belongs_to. Restart all. [2] Disable cron job for message queue. [3] Generate two notices for each domain A, B and C (in that order). Make sure that borrowers involved have correct address. [4] Run process_message_queue.pl -limit 1 [5] Check that one is sent for A, 5 pending. [6] Run process_message_queue.pl -limit 2 [7] Check that one is sent for C, 4 pending. [8] Run process_message_queue.pl (without limit). [9] Check that nothing is sent, 4 pending. [10] Check that message_queue.to_address is filled for those 4. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #29 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149666 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149666&action=edit Bug 33360: Adjust Letters.t for limit parameter and domain limits Includes: [1] Add a counter in the send_or_die mock. [2] Correct from processed count to sent count. [3] More extensive testing for limit parameter and domain limits. Test plan: Run t/db_dependent/Letters.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #30 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149667 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149667&action=edit Bug 33360: Update koha-conf.xml Test plan: We already tested it. Just look at changes in this patch. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #31 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 149668 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149668&action=edit Bug 33360: (follow-up) Adjust wording in verbose mode Koha reports 'sending mail' while we actually are still processing mail. It could still be delayed, fail, etc. Test plan: Run process_message_queue in verbose mode. If a message get processed, you should see: Processing email message to patron: 51 at /usr/share/koha/C4/Letters.pm line 1008. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com --- Comment #32 from Kyle M Hall <kyle@bywatersolutions.com> --- QA thoughts: Considering the SMTP configuration is web based, it would be nice if the controls for this were integrated into it. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #33 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Kyle M Hall from comment #32)
QA thoughts: Considering the SMTP configuration is web based, it would be nice if the controls for this were integrated into it.
Thanks Kyle for QA. Happy to put some further improvements on a follow-up. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Keywords| |release-notes-needed --- Comment #34 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Kyle M Hall from comment #32)
QA thoughts: Considering the SMTP configuration is web based, it would be nice if the controls for this were integrated into it.
Agreed! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |23.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #35 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.05. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #36 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Tomás Cohen Arazi from comment #35)
Pushed to master for 23.05.
Nice work everyone, thanks!
Top ! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |In order to release notes| |control/throttle the volume | |of mail messages sent by | |Koha, this report makes the | |limit parameter of | |process_message_queue (and | |associated routine in | |Letters) look at the number | |of messages actually sent, | |not processed. It also adds | |the possibility of delaying | |messages to specified | |domains according to limits | |defined in koha-conf.xml | |(see example section added | |in this patch set). For | |instance, you may define | |that each minute only 30 | |messages are sent to | |outlook.com or 50 messages | |per hour to gmail.com, etc. | |This will help in reducing | |deferred or rejected mails | |due to exceeding limits of | |various email providers. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |33537 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33537 [Bug 33537] Move domain limits from koha-conf to staff SMTP configuration -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 --- Comment #37 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Tomás Cohen Arazi from comment #34)
(In reply to Kyle M Hall from comment #32)
QA thoughts: Considering the SMTP configuration is web based, it would be nice if the controls for this were integrated into it.
Agreed!
Bug 33537 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |matt.blenkinsop@ptfs-europe | |.com Status|Pushed to master |Needs documenting --- Comment #38 from Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> --- Enhancement - not backporting to 22.11.x Nice work everyone! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wizzyrea@gmail.com --- Comment #39 from Liz Rea <wizzyrea@gmail.com> --- (In reply to Kyle M Hall from comment #32)
QA thoughts: Considering the SMTP configuration is web based, it would be nice if the controls for this were integrated into it.
100% this yes please -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33360 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC|martin.renvoize@ptfs-europe | |.com | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org