https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 201498 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201498&action=edit Bug 42992: FTP file transport: fix list_files() corrupting filenames Koha::File::Transport::FTP::_list_files parsed the LIST / "ls -l" output with a non-greedy date capture that only consumed the first of the three date tokens (the month), leaving the day and time glued to the front of the filename. Every filename returned for an FTP transport was therefore prefixed with the listing date, e.g. -rw-r--r-- 1 user group 1234 Jan 01 12:00 QUOTES_413514.CEQ => filename "01 12:00 QUOTES_413514.CEQ" Callers (Koha::Edifact::Transport, marc_ordering_process.pl) then asked the server for a file that does not exist, so no files were downloaded, renamed or deleted over FTP. SFTP and Local transports were unaffected. This regressed when the EDI transport was moved onto the file_transports system (bug 38489). The previous Koha::Edifact::Transport::ftp_download listed files with Net::FTP->ls (NLST, bare names); the refactor switched to Net::FTP->dir (LIST) so it could return the richer hashref shape used by the SFTP and Local transports, but the parse was faulty. The date field in "ls -l" output is a fixed run of three whitespace separated tokens (Mon DD HH:MM or Mon DD YYYY). Consume it explicitly so the filename is captured cleanly, preserving spaces and still skipping directories. The LIST-based approach and the filename/longname/size/perms return shape are kept so the structure stays consistent across transport classes (relied on by consumers, including out-of-tree ones). Test plan: 1) prove t/db_dependent/Koha/File/Transport/FTP.t 2) git checkout Koha/File/Transport/FTP.pm and re-run: the list_files() subtest fails on the filename/size/perms assertions 3) prove t/db_dependent/Koha/Edifact/Transport.t -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 201501 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=201501&action=edit Bug 42992: FTP file transport: fix list_files() corrupting filenames Koha::File::Transport::FTP::_list_files parsed the LIST / "ls -l" output with a non-greedy date capture that only consumed the first of the three date tokens (the month), leaving the day and time glued to the front of the filename. Every filename returned for an FTP transport was therefore prefixed with the listing date, e.g. -rw-r--r-- 1 user group 1234 Jan 01 12:00 QUOTES_413514.CEQ => filename "01 12:00 QUOTES_413514.CEQ" Callers (Koha::Edifact::Transport, marc_ordering_process.pl) then asked the server for a file that does not exist, so no files were downloaded, renamed or deleted over FTP. SFTP and Local transports were unaffected. This regressed when the EDI transport was moved onto the file_transports system (bug 38489). The previous Koha::Edifact::Transport::ftp_download listed files with Net::FTP->ls (NLST, bare names); the refactor switched to Net::FTP->dir (LIST) so it could return the richer hashref shape used by the SFTP and Local transports, but the parse was faulty. The date field in "ls -l" output is a fixed run of three whitespace separated tokens (Mon DD HH:MM or Mon DD YYYY). Consume it explicitly so the filename is captured cleanly, preserving spaces and still skipping directories. The LIST-based approach and the filename/longname/size/perms return shape are kept so the structure stays consistent across transport classes (relied on by consumers, including out-of-tree ones). Test plan: 1) prove t/db_dependent/Koha/File/Transport/FTP.t 2) git checkout Koha/File/Transport/FTP.pm and re-run: the list_files() subtest fails on the filename/size/perms assertions 3) prove t/db_dependent/Koha/Edifact/Transport.t Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Thanks for testing David.. this one is indeed a difficult one to get to grips with.. it hurt my brain ;). I sort of discovered the easiest way to 'see' the difference before and after the patches is to use https://github.com/openfifth/koha-plugin-transport-browser (I only though about that after submitted my test plan in the bug). But even then, you'll need a FTP server somewhere to test it with) What you would find is that the file listing before the patch would look something like: `00 01 MAY filename.md` Then after the patch it'll be: `filename.md` And the other columns will contain the properly split out date instead of it being squashed into the filename string. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43078 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43078 [Bug 43078] File transport SFTP backend returns inconsistent list() structure compared to FTP/Local backends -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 Bug 42992 depends on bug 39190, which changed state. Bug 39190 Summary: Rework new (S)FTP classes to be polymorphic classes https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39190 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #6 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 202819 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=202819&action=edit Bug 42992: FTP file transport: fix list_files() corrupting filenames Koha::File::Transport::FTP::_list_files parsed the LIST / "ls -l" output with a non-greedy date capture that only consumed the first of the three date tokens (the month), leaving the day and time glued to the front of the filename. Every filename returned for an FTP transport was therefore prefixed with the listing date, e.g. -rw-r--r-- 1 user group 1234 Jan 01 12:00 QUOTES_413514.CEQ => filename "01 12:00 QUOTES_413514.CEQ" Callers (Koha::Edifact::Transport, marc_ordering_process.pl) then asked the server for a file that does not exist, so no files were downloaded, renamed or deleted over FTP. SFTP and Local transports were unaffected. This regressed when the EDI transport was moved onto the file_transports system (bug 38489). The previous Koha::Edifact::Transport::ftp_download listed files with Net::FTP->ls (NLST, bare names); the refactor switched to Net::FTP->dir (LIST) so it could return the richer hashref shape used by the SFTP and Local transports, but the parse was faulty. The date field in "ls -l" output is a fixed run of three whitespace separated tokens (Mon DD HH:MM or Mon DD YYYY). Consume it explicitly so the filename is captured cleanly, preserving spaces and still skipping directories. The LIST-based approach and the filename/longname/size/perms return shape are kept so the structure stays consistent across transport classes (relied on by consumers, including out-of-tree ones). Test plan: 1) prove t/db_dependent/Koha/File/Transport/FTP.t 2) git checkout Koha/File/Transport/FTP.pm and re-run: the list_files() subtest fails on the filename/size/perms assertions 3) prove t/db_dependent/Koha/Edifact/Transport.t Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #7 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Thanks everyone! Pushed to main for 26.11! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42992 --- Comment #8 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to 26.05.x for the upcoming 26.05.03 release. -- You are receiving this mail because: You are watching all bug changes.