- Koha-devel - lists.koha-community.org

Re: [Koha-devel] Feb. 2024 prospects of Koha lists delivery problems for Gmail, etc.
by Thomas Dukleth 28 Nov '23

28 Nov '23

[Resent without the accidental attachment from an unrelated message of David Cook which produced peculiar mailing list behaviour assigning the wrong thread and no copy of my own message from the mailing list. I do not generally use HTML mail attachments and did not notice it at the bottom of my compose window from the previous message rewriting from the forward function instead of new list message.] Beginning in February 2024, Koha mailing lists may have delivery problems for recipients using some large email service providers such as Gmail if not fixed. The problem is more likely to affect such recipients of the Koha general mailing list because more needs fixing even if relatively easy to fix and it might additionally be considered a large volume sender if there are enough subscribers reading via Gmail including on popular mobile devices despite fewer otherwise evident Gmail users. Please give attention to new party needed for hosting the Koha general mailing list in section 2.2 further below. 1. Diminishing Time for Implementing Fixes. By February 2024, fixing DKIM signing and re-signing may be needed for lists.koha-community.org lists such as koha-devel and more for the Koha general mailing list when Gmail and Yahoo mail [with AOL] may start blocking messages for not being DKIM re-signed when sent from the mailing lists. It is uncertain whether Gmail and Yahoo mail will continue to give a false pass for messages using the original author's DKIM signature which does not match the mailing list sending server, nor the From header with DMARC support enabled. A false DKIM pass may be more likely than not in February for the low volume mailing lists.koha-community.org, but we have been warned, see the announcements linked below. [The Koha general mailing list may be more of a problem with more configuration needed and a greater prospect of being recognised as a large volume sender.] Gmail, Yahoo mail, AOL mail, Microsoft [Exchange and various names] etc. have all rejected messages for bad DKIM signature from mailing lists during past periods of extra DKIM strictness. 2. Details of Changes Needed. See the bug report for some implementation details, "Adding DMARC compatibility to mailing lists" - https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34927 . The bug issue was slowly being advanced over the past year before any knowledge of any particular forthcoming changes at large email providers such as Google. 2.1. BibLibre Hosted Mailing Lists. I have given precise technical details of changes which may be made to Laurent Ducos and Fridolin Somers at BibLibre for resolving the issue of OpenDKIM signing and resigning for lists.koha-community.org . 2.2. New Party Needed to Host the Koha General Mailing List. I have also communicated with Rachel Hamilton-Williams about adding DMARC support for the Koha general mailing list. In addition to DKIM support, lists.katipo.co.nz needs a DMARC record in DNS and then activating DMARC in Mailman, and possibly an SPF record for the subdomain lists.katipo.co.nz . Rachel informed me that she would like to hand over hosting of the Koha mailing list for more attentive management than her partner and system administrator, Simon Blake, may be able to provide currently. Without the very simple task of adding DMARC support to the Koha general list, delivery problems are more likely than not for Gmail, Yahoo, etc. subscribers from Feb. A false pass would need at least the pretence of DMARC support. ARC support is required for all mailing lists irrespective of size, for which DMARC might be a workaround as possibly avoiding the need for ARC support. 3. February 2024 Changes for Gmail and Yahoo Mail, etc. At the end of last week, a radio broadcast brought my attention to changes coming in February 2024 which affect everyone in some manner. Everyone includes lists.koha-community.org. Large volume senders have additional requirements. [For large volume senders, there additional requirements beyond those affecting everyone, which might affect people subscribing to the Koha general mailing list if the number of mailing list subscribers is enough and enough people route mail through the popular choice of Gmail even on mobile despite having some other apparent domain of some subscriber wherever that may be hosted initially.] Details about more stringent SPF, DKIM, DMARC, ARC, and one-click unsubscribe link requirements are available from Google. Mailing lists may be able to substitute DMARC support for lack of ARC support when rewriting the From header and thus re-originating and not merely forwarding messages but adding ARC is best addressed second. "Email sender guidelines : Requirements for all senders" - https://support.google.com/mail/answer/81126#zippy=%2Crequirements-for-all-… . Another part of the same document has the requirements which may affect the Koha general mailing list "Requirements for sending 5,000 or more messages per day" - https://support.google.com/mail/answer/81126#requirements-5k&zippy=%2Crequi… . The Yahoo guide which I found has fewer details and does not refer to the coming February 2024 policy change: "Sender Best Practices" - https://senders.yahooinc.com/best-practices/ . There is no shortage of secondary sources such as from the support provider Proofpoint, "Google and Yahoo Set a Short Timeline to Meet New DMARC Policy & Setup Requirements. Are You Ready?" - https://www.proofpoint.com/us/blog/email-and-cloud-threats/google-and-yahoo… . 4. Note on ARC Support. ARC is intended for authenticating the email chain when forwarding messages which is the basic function of mailing lists. Adding DMARC support should make the issue of ARC support for acceptable authentication for mailing lists might be moot because the mailing list is more clearly shown as re-originating email and not merely forwarding. However, the announcements for February 2024 do not state that case with explicit clarity and Gmail adds ARC headers to all mail on their system and people at Google may presume that everyone else should to especially when messages may retain headers showing that the message has been forwarded over the mailing list despite having been re-originated from mailing list with DMARC authentication. While Mailman 3 has functionality for ARC support which was added essentially experimentally a few years ago, the proper place for ARC support is in the MTA not in the mailing list software. When using ARC via Mailman 3 the mail envelope is sealed before DKIM re-signing which is the wrong order and has caused ARC authentication failure. OpenARC, like OpenDKIM, functions in the MTA for Postfix or Sendmail, https://github.com/trusteddomainproject/OpenARC . Mailing lists at https://openarc.org/ . OpenARC is not as fully developed as OpenDKIM and support for some nice things such as multiple sending domains on the system seems to have been abandoned. OpenARC has better support for BSD Unix and Red Hat than Debian based systems but is not as well developed, and although not robustly maintained for Debian based systems, there are openarc packages based on the OpenARC development branch for Debian 9 to 11, https://download.opensuse.org/repositories/home%3A/andreasschulze/ . [Mailman 2 which we are using for the mailing lists does not go past Debian 10 for lack of Python 2. Upgrading to Mailman 3 is non-trivial because of configuration changes, etc. and should not be the most immediate priority.] There is a very brief blog post about using the Andreas Schulze Debian packages, "OpenARC with Postfix on Debian 10 (buster)" / Matthieu - https://weber.fi.eu.org/blog/Informatique/openarc_with_postfix_on_debian_10… . You can also build your own packages from source as I have. [In current testing of my source build, Postfix has a socket permissions error for OpenARC which may be from a mistake I had made with umask settings long ago on the system which runs my mailserver.] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

1 0

Feb. 2024 prospects of Koha lists delivery problems for Gmail, etc.
by Thomas Dukleth 28 Nov '23

28 Nov '23

Beginning in February 2024, Koha mailing lists may have delivery problems for recipients using some large email service providers such as Gmail if not fixed. The problem is more likely to affect such recipients of the Koha general mailing list because more needs fixing even if relatively easy to fix and it might additionally be considered a large volume sender if there are enough subscribers reading via Gmail including on popular mobile devices despite fewer otherwise evident Gmail users. Please give attention to new party needed for hosting the Koha general mailing list in section 2.2 further below. 1. Diminishing Time for Implementing Fixes. By February 2024, fixing DKIM signing and re-signing may be needed for lists.koha-community.org lists such as koha-devel and more for the Koha general mailing list when Gmail and Yahoo mail [with AOL] may start blocking messages for not being DKIM re-signed when sent from the mailing lists. It is uncertain whether Gmail and Yahoo mail will continue to give a false pass for messages using the original author's DKIM signature which does not match the mailing list sending server, nor the From header with DMARC support enabled. A false DKIM pass may be more likely than not in February for the low volume mailing lists.koha-community.org, but we have been warned, see the announcements linked below. [The Koha general mailing list may be more of a problem with more configuration needed and a greater prospect of being recognised as a large volume sender.] Gmail, Yahoo mail, AOL mail, Microsoft [Exchange and various names] etc. have all rejected messages for bad DKIM signature from mailing lists during past periods of extra DKIM strictness. 2. Details of Changes Needed. See the bug report for some implementation details, "Adding DMARC compatibility to mailing lists" - https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34927 . The bug issue was slowly being advanced over the past year before any knowledge of any particular forthcoming changes at large email providers such as Google. 2.1. BibLibre Hosted Mailing Lists. I have given precise technical details of changes which may be made to Laurent Ducos and Fridolin Somers at BibLibre for resolving the issue of OpenDKIM signing and resigning for lists.koha-community.org . 2.2. New Party Needed to Host the Koha General Mailing List. I have also communicated with Rachel Hamilton-Williams about adding DMARC support for the Koha general mailing list. In addition to DKIM support, lists.katipo.co.nz needs a DMARC record in DNS and then activating DMARC in Mailman, and possibly an SPF record for the subdomain lists.katipo.co.nz . Rachel informed me that she would like to hand over hosting of the Koha mailing list for more attentive management than her partner and system administrator, Simon Blake, may be able to provide currently. Without the very simple task of adding DMARC support to the Koha general list, delivery problems are more likely than not for Gmail, Yahoo, etc. subscribers from Feb. A false pass would need at least the pretence of DMARC support. ARC support is required for all mailing lists irrespective of size, for which DMARC might be a workaround as possibly avoiding the need for ARC support. 3. February 2024 Changes for Gmail and Yahoo Mail, etc. At the end of last week, a radio broadcast brought my attention to changes coming in February 2024 which affect everyone in some manner. Everyone includes lists.koha-community.org. Large volume senders have additional requirements. [For large volume senders, there additional requirements beyond those affecting everyone, which might affect people subscribing to the Koha general mailing list if the number of mailing list subscribers is enough and enough people route mail through the popular choice of Gmail even on mobile despite having some other apparent domain of some subscriber wherever that may be hosted initially.] Details about more stringent SPF, DKIM, DMARC, ARC, and one-click unsubscribe link requirements are available from Google. Mailing lists may be able to substitute DMARC support for lack of ARC support when rewriting the From header and thus re-originating and not merely forwarding messages but adding ARC is best addressed second. "Email sender guidelines : Requirements for all senders" - https://support.google.com/mail/answer/81126#zippy=%2Crequirements-for-all-… . Another part of the same document has the requirements which may affect the Koha general mailing list "Requirements for sending 5,000 or more messages per day" - https://support.google.com/mail/answer/81126#requirements-5k&zippy=%2Crequi… . The Yahoo guide which I found has fewer details and does not refer to the coming February 2024 policy change: "Sender Best Practices" - https://senders.yahooinc.com/best-practices/ . There is no shortage of secondary sources such as from the support provider Proofpoint, "Google and Yahoo Set a Short Timeline to Meet New DMARC Policy & Setup Requirements. Are You Ready?" - https://www.proofpoint.com/us/blog/email-and-cloud-threats/google-and-yahoo… . 4. Note on ARC Support. ARC is intended for authenticating the email chain when forwarding messages which is the basic function of mailing lists. Adding DMARC support should make the issue of ARC support for acceptable authentication for mailing lists might be moot because the mailing list is more clearly shown as re-originating email and not merely forwarding. However, the announcements for February 2024 do not state that case with explicit clarity and Gmail adds ARC headers to all mail on their system and people at Google may presume that everyone else should to especially when messages may retain headers showing that the message has been forwarded over the mailing list despite having been re-originated from mailing list with DMARC authentication. While Mailman 3 has functionality for ARC support which was added essentially experimentally a few years ago, the proper place for ARC support is in the MTA not in the mailing list software. When using ARC via Mailman 3 the mail envelope is sealed before DKIM re-signing which is the wrong order and has caused ARC authentication failure. OpenARC, like OpenDKIM, functions in the MTA for Postfix or Sendmail, https://github.com/trusteddomainproject/OpenARC . Mailing lists at https://openarc.org/ . OpenARC is not as fully developed as OpenDKIM and support for some nice things such as multiple sending domains on the system seems to have been abandoned. OpenARC has better support for BSD Unix and Red Hat than Debian based systems but is not as well developed, and although not robustly maintained for Debian based systems, there are openarc packages based on the OpenARC development branch for Debian 9 to 11, https://download.opensuse.org/repositories/home%3A/andreasschulze/ . [Mailman 2 which we are using for the mailing lists does not go past Debian 10 for lack of Python 2. Upgrading to Mailman 3 is non-trivial because of configuration changes, etc. and should not be the most immediate priority.] There is a very brief blog post about using the Andreas Schulze Debian packages, "OpenARC with Postfix on Debian 10 (buster)" / Matthieu - https://weber.fi.eu.org/blog/Informatique/openarc_with_postfix_on_debian_10… . You can also build your own packages from source as I have. [In current testing of my source build, Postfix has a socket permissions error for OpenARC which may be from a mistake I had made with umask settings long ago on the system which runs my mailserver.] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783

1 0

"Online access" vs "Online resources"
by David Cook 27 Nov '23

27 Nov '23

Hi all, Detail pages say "Online resources" but search results say "Online access". I don't know if I've never noticed before or what but I had a librarian raise it today. I think it makes sense to harmonise so that they're both the same, but how do we choose which one? I've raised a ticket for this: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35410 David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Call for news - Newsletter November 2023
by Koha Newsletter 13 Nov '23

13 Nov '23

Hi I'm collecting news for the November 2023 Koha Community Newsletter. Please send anything noteworthy to: kohanews (at) gmail (dot) com News criteria: * News items can be of any length. * Images are fine. * Anything and everything Koha. * Submit by the 26th of the month. Text format criteria: * Just structured plain text, or * HTML text to include tables or similar For events: * Consider adding your event to the Koha Community calendar at https://koha-community.org/calendar/ Thank you! Michael Kuhn Editor, Koha Community Newsletter

1 0

Translation server migrated
by Jonathan Druart 11 Nov '23

11 Nov '23

Hello, The translation server has been migrated from Pootle to Weblate. It's a whole new server with a complete new workflow. The goal was to have it ready to translate 23.11, but everything is not set up yet (downtimes may appear). You can login with your Pootle credentials, and you should have the same privileges (part of a language team/administrator of the team). Note that no SMTP server (email) is configured yet. There are some heavy tasks that will be run from 1:00 to 2:00 UTC and from 3:00 to 5:00 (or 6:00? I don't know yet!), expect a slow response time during these slots (can be adjusted later if needed). More information (about what has been done and what still need to be done) can be found on the wiki page https://wiki.koha-community.org/wiki/Translation_server_migration_weblate Please report any problems you encounter! Happy translating! Cheers, Jonathan PS: email to the general mailing list about this migration will be sent next week.

1 0

Anybody still using tarballs?
by Jonathan Druart 10 Nov '23

10 Nov '23

Hi, I am suggesting removing the tarball from our release process. I don't think anybody is using it anyway. It's extra work for RMaints, and it does not seem necessary. I am going to suggest removing the po files from the Koha src code starting from November releases, and that will require yet an extra (unnecessary?) step to embed the po files to generate the tarball. What do you think? Cheers, Jonathan

5 10

Koha 23.05.05 released
by Fridolin SOMERS 08 Nov '23

08 Nov '23

Hello everyone 🤗 The Koha community is proud to announce the release of Koha 23.05.05. It is a bugfix/maintenance release including 16 enhancements, 85 bugfixes. The full release notes are available here: https://koha-community.org/koha-23-05-05-released/ Debian packages should be available soon 🎁 Note we are at the end of cycle. For next cycle I will continue as maintainer of 23.11 🛠️ Best regards 🤓 -- Fridolin SOMERS <fridolin.somers(a)biblibre.com> Software and system maintainer 🦄 BibLibre, France

1 0

Using Elasticsearch in koha-testing-docker
by David Cook 07 Nov '23

07 Nov '23

Hi all, I'm trying to use Elasticsearch in koha-testing-docker, and I would appreciate some assistance. Here's the steps I'm taking: 1. Launch koha-testing-docker 2. Update Koha's sysprefs a. Change SearchEngine from "Zebra" to "Elasticsearch" at http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search <http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search&searchfie ld=SearchEngine> &searchfield=SearchEngine 3. (Re)build Elasticsearch indexes: a. koha-elasticsearch --rebuild kohadev 4. Start Elasticsearch indexer: a. koha-es-indexer --start kohadev Is there anything else to do? In a prod install, do I just need to update the "elasticsearch" config in koha-conf.xml? Do I leave all the Zebra things running or can I turn those off? I recall reading somewhere that zebra_queue might fill up if I turn it off? Thanks all! David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

2 2

Last month before the release
by Tomas Cohen Arazi 06 Nov '23

06 Nov '23

Hi all. We are almost done with the release, a couple medium/big things to push today/tomorrow and then bug fixing. I wanted to ask y'all to please view the Jenkins tasks status anytime after your stuff has been pushed. I know I said this many times, but some devs clearly break some tests that are trivially fixable and the team needs you to take care of your breakages. I'm doing my best to track things and fix, but you should all be worried about that. -- Tomás Cohen Arazi Theke Solutions (https://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F

3 10

Download a record with curl and Koha API
by Zeno Tajoli 03 Nov '23

03 Nov '23

Hi, I'm trying to download a record using curl and GET /biblios/{biblio_id} With curl I try: curl -u user:Pwd --request GET 'http://127.0.0.1:8080/api/v1/biblios/xxxx' --header "Content-Type: application/json" But the answer is: ["application\/json","application\/marcxml+xml","application\/marc-in-json","application\/marc","text\/plain"] not a json with the MARC21 data. What is the error ? Cheers Zeno Tajoli

2 1