https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Alex Buckley <alexbuckley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |alexbuckley@catalyst.net.nz |ity.org | Status|NEW |ASSIGNED Change sponsored?|--- |Sponsored -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #1 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Created attachment 166085 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166085&action=edit Bug 36766: WIP Command-line utility to SFTP file to remote server -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #2 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Created attachment 166089 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166089&action=edit Bug 36766: Add a command-line utility to SFTP a file to a remote server To test you need to have a remote server you are authorised to SFTP files to. Test plan: 1. Apply patch and restart services 2. Generate a file on your Koha server. Place it in /tmp/test.mrc 3. Manually connect to the SFTP remote host and accept the hostkey 4. Enter Koha shell 5. Run the sftp_file.pl command-line script ./sftp_file.pl --host <your remote hostname> --user <your user> --pass <your password> --upload_dir <your upload directory> --file /tmp/test.mrc 6. Confirm the SFTP was successful Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #3 from David Cook <dcook@prosentient.com.au> --- It looks like the email subject and body are hardcoded for OCLC. You'll want to make that more configurable I reckon. In theory, I'm not sure why you wouldn't just use a utility like lftp, but if using "koha-foreach", it could be nice to pick out things like the database name and admin email address I suppose. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #5 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Created attachment 166471 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166471&action=edit Bug 36766: Make SFTP notice more customisable SFTP_FAILURE and SFTP_SUCCESSFUL notices can be configured from Notices and slip tool. Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #7 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Created attachment 166859 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166859&action=edit Bug 36766: Only list the file basename as the SFTP put destination Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- (In reply to David Nind from comment #8)

I'd like to test, any tips on setting up configuring an SFTP server to run in KTD?

(I looked for some tutorials, but couldn't find anything that made sense to me.)

Take a look at the Docker hub entry for "atmoz/sftp". You should be able to add something like the following to your docker-compose.yml: sftp: image: atmoz/sftp command: USERNAME:PASSWORD:::upload networks: - kohanet I've used this image and config for other non-Koha projects before when I was testing SFTP clients, and it worked well. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #12 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 172152 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172152&action=edit Bug 36766: Make SFTP notice more customisable SFTP_FAILURE and SFTP_SUCCESSFUL notices can be configured from Notices and slip tool. Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #14 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 172154 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172154&action=edit Bug 36766: (QA follow-up) Tidy and use die_on_error over return Tidy and add exec to atomic update Fix POD Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #16 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- It would be nice to add failures for things like changing directory or bad email to the email sent by this - so that user's can be notified, but as this is new I think it is enough to move forward

In theory, I'm not sure why you wouldn't just use a utility like lftp, but if >using "koha-foreach", it could be nice to pick out things like the database >name and admin email address I suppose.

PTFS is looking into adding SFTP configs to the interface (more generalized than the config in EDI) so this script would be nice to have and could then read configs from the DB given an ID or a name, and then we can encrypt passwords etc. I think this is a good base If David (either one) signs off, then mine can be considered a QA stamp -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172152|0 |1 is obsolete| | --- Comment #18 from David Nind <david@davidnind.com> --- Created attachment 172162 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172162&action=edit Bug 36766: Make SFTP notice more customisable SFTP_FAILURE and SFTP_SUCCESSFUL notices can be configured from Notices and slip tool. Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172154|0 |1 is obsolete| | --- Comment #20 from David Nind <david@davidnind.com> --- Created attachment 172164 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172164&action=edit Bug 36766: (QA follow-up) Tidy and use die_on_error over return Tidy and add exec to atomic update Fix POD Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=35761 CC| |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wizzyrea@gmail.com --- Comment #22 from Liz Rea <wizzyrea@gmail.com> --- YES OMG THANK YOU. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Kyle M Hall (khall) <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #172161|0 |1 is obsolete| | Attachment #172162|0 |1 is obsolete| | Attachment #172163|0 |1 is obsolete| | Attachment #172164|0 |1 is obsolete| | --- Comment #23 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172176 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172176&action=edit Bug 36766: Add a command-line utility to SFTP a file to a remote server To test you need to have a remote server you are authorised to SFTP files to. Test plan: 1. Apply patch and restart services 2. Generate a file on your Koha server. Place it in /tmp/test.mrc 3. Manually connect to the SFTP remote host and accept the hostkey 4. Enter Koha shell 5. Run the sftp_file.pl command-line script ./sftp_file.pl --host <your remote hostname> --user <your user> --pass <your password> --upload_dir <your upload directory> --file /tmp/test.mrc 6. Confirm the SFTP was successful Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #25 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172178 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172178&action=edit Bug 36766: Only list the file basename as the SFTP put destination Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #27 from Kyle M Hall (khall) <kyle@bywatersolutions.com> --- Created attachment 172180 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172180&action=edit Bug 36766: (QA follow-up) Add single dash paramter options Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35761 CC| |martin.renvoize@ptfs-europe | |.com Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35761 [Bug 35761] Add an administration editor for FTP and SFTP servers -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |additional_work_needed --- Comment #30 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Please provide a follow-up: 1) In some email setups this won't work as the KohaAdminEmailAddress will be a noreply address (for anti-spam reasons) +if ($email) { + if ( C4::Context->preference('KohaAdminEmailAddress') ) { + $admin_address = C4::Context->preference('KohaAdminEmailAddress'); + } Can you please make the address configurable or better prefer ReplytoDefault and fall-back to KohaAdminEmailAddress if it's not set. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #31 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.11! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #32 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Katrin Fischer from comment #30)

Please provide a follow-up:

1) In some email setups this won't work as the KohaAdminEmailAddress will be a noreply address (for anti-spam reasons)

+if ($email) { + if ( C4::Context->preference('KohaAdminEmailAddress') ) { + $admin_address = C4::Context->preference('KohaAdminEmailAddress'); + }

Can you please make the address configurable or better prefer ReplytoDefault and fall-back to KohaAdminEmailAddress if it's not set.

I need this as a follow-up before release please. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #34 from Alex Buckley <alexbuckley@catalyst.net.nz> --- (In reply to Katrin Fischer from comment #32)

(In reply to Katrin Fischer from comment #30)

...

Please provide a follow-up:

1) In some email setups this won't work as the KohaAdminEmailAddress will be a noreply address (for anti-spam reasons)

+if ($email) { + if ( C4::Context->preference('KohaAdminEmailAddress') ) { + $admin_address = C4::Context->preference('KohaAdminEmailAddress'); + }

Can you please make the address configurable or better prefer ReplytoDefault and fall-back to KohaAdminEmailAddress if it's not set.

I need this as a follow-up before release please.

Hey Katrin, sorry I put the follow-up on a separate bug report ( bug 38282 ). I've just attached that patch to this bug report. Does that give you what you need, or is there anything else I can do? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 --- Comment #36 from Alex Buckley <alexbuckley@catalyst.net.nz> --- (In reply to Katrin Fischer from comment #35)

No that's good, thanks so much! :)

Thanks Katrin! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement adds a new release notes| |command line utility | |misc/cronjobs/sftp_file.pl | |that allows libraries to | |securely transfer MARC data | |to an SFTP server. | | | |Example usage: | |misc/cronjobs/sftp_file.pl | |--host sftp --user koha | |--pass koha --upload_dir | |upload --file /tmp/test.mrc | | | |Additional information: | |1. | |Use | |misc/cronjobs/sftp_file.pl | |--help to list all the | |options available. | |2. Two | |new notices are available: | |SFTP_FAILURE and | |SFTP_SUCCESSFUL. | |3. The | |from address for any emails | |defaults to | |KohaAdminEmailAddress if | |ReplytoDefault is not set. | | | |Example of usage and | |workflow | |--------------------------- | |-- | | | |In New Zealand, | |libraries need to send MARC | |files to a remote server | |(using SFTP) to keep the | |national Te Puna union | |catalogue up to date. | | | |This | |enhancement allows NZ | |libraries to automate | |sending their MARC files | |by: | |1. Using the | |runreport.pl cronjob to | |generate a list of | |bibliographic numbers. | |2. | |Using the export_records.pl | |cronjob to generate the | |MARC file for those | |bibliographic numbers. | |3. | |Use a new sftp_file cronjob | |to transfer the MARC file | |to the remote SFTP server. Keywords|release-notes-needed | --- Comment #37 from David Nind <david@davidnind.com> --- I've left the additional_work_needed keyword. Is this still required? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This enhancement adds a new |This enhancement adds a new release notes|command line utility |command line utility |misc/cronjobs/sftp_file.pl |misc/cronjobs/sftp_file.pl |that allows libraries to |that allows libraries to |securely transfer MARC data |securely transfer MARC data |to an SFTP server. |to an SFTP server. | | |Example usage: |Example usage: |misc/cronjobs/sftp_file.pl |misc/cronjobs/sftp_file.pl |--host sftp --user koha |--host sftp --user koha |--pass koha --upload_dir |--pass koha --upload_dir |upload --file /tmp/test.mrc |upload --file /tmp/test.mrc | | |Additional information: |Additional information: |1. |1. |Use |Use |misc/cronjobs/sftp_file.pl |misc/cronjobs/sftp_file.pl |--help to list all the |--help to list all the |options available. |options available. |2. Two |2. Two |new notices are available: |new notices are available: |SFTP_FAILURE and |SFTP_FAILURE and |SFTP_SUCCESSFUL. |SFTP_SUCCESSFUL. |3. The |3. The |from address for any emails |from address for any emails |defaults to |defaults to |KohaAdminEmailAddress if |KohaAdminEmailAddress if |ReplytoDefault is not set. |ReplytoDefault is not set. | | |Example of usage and |##### Example of usage and |workflow |workflow |--------------------------- | |-- |In New Zealand, | |libraries need to send MARC |In New Zealand, |files to a remote server |libraries need to send MARC |(using SFTP) to keep the |files to a remote server |national Te Puna union |(using SFTP) to keep the |catalogue up to date. |national Te Puna union | |catalogue up to date. |This | |enhancement allows NZ |This |libraries to automate |enhancement allows NZ |sending their MARC files |libraries to automate |by: |sending their MARC files |1. Using the |by: |runreport.pl cronjob to |1. Using the |generate a list of |runreport.pl cronjob to |bibliographic numbers. |generate a list of |2. |bibliographic numbers. |Using the export_records.pl |2. |cronjob to generate the |Using the export_records.pl |MARC file for those |cronjob to generate the |bibliographic numbers. |MARC file for those |3. |bibliographic numbers. |Use a new sftp_file cronjob |3. |to transfer the MARC file |Use a new sftp_file cronjob |to the remote SFTP server. |to transfer the MARC file | |to the remote SFTP server. | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|additional_work_needed, | |rel_24_11_candidate | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36766 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |38629 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38629 [Bug 38629] Update sftp_file.pl to support the new SFTP configurations page -- You are receiving this mail because: You are watching all bug changes.