[Bug 20254] New: Forbid the download of duplicate EDI messages
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Bug ID: 20254 Summary: Forbid the download of duplicate EDI messages Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Acquisitions Assignee: koha-bugs@lists.koha-community.org Reporter: colin.campbell@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org In theory EDI messages should be downloaded only once, as we rename it according to Edifact conventions as soon as downloaded. However the case has occured when after an issue with a vendor's server duplicates of the days files have been uploaded to the download directory. As a result duplicate quotes are downloaded, creating new orders on the system, as Koha cannot now rename the file (it would entail overwriting a file on the vendor's system) this can continue infinitely. So do not allow duplicate filenames to be downloaded, add a unique constraint to edifact_messages to support this -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |colin.campbell@ptfs-europe. |ity.org |com Status|NEW |Needs Signoff --- Comment #1 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 74002 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=74002&action=edit Block ingest of duplicate filenames Protect against ingest of duplicate filenames -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |mtompset@hotmail.com --- Comment #2 from M. Tompsett <mtompset@hotmail.com> --- While the code eyeballs well, is it possible to write a test to trigger this case? We have no tests which trigger ingest, as far as I can tell. And this is a Koha library. Failing QA for lack of test(s). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #74002|0 |1 is obsolete| | --- Comment #3 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 83013 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83013&action=edit Patch forbidding ingest of duplicate filename -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #4 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 83014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83014&action=edit Patch working_directory method Was an unused method. Needs an update to set the correct member as is now required for tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #5 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Created attachment 83015 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83015&action=edit Add tests for patch 20254 Run tests, confirms that file is added to edifact_messages and that a second attempt to re-add the same filename is rejected -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Colin Campbell <colin.campbell@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #6 from Colin Campbell <colin.campbell@ptfs-europe.com> --- Tests added to confirm patch does stop duplicate messages are loaded, protecting the system from damage resulting from multiple ingests of duplicate files -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #7 from M. Tompsett <mtompset@hotmail.com> --- Comment on attachment 83014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83014 Patch working_directory method Review of attachment 83014: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=20254&attachment=83014) ----------------------------------------------------------------- technically beyond scope of this bug, I think, but hey, valid fix. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #83013|0 |1 is obsolete| | Attachment #83014|0 |1 is obsolete| | Attachment #83015|0 |1 is obsolete| | --- Comment #8 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 91412 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=91412&action=edit Bug 20254: Do not ingest files with duplicate filenames If the supplier delivers the same file twice we are unable to rename it on the second occurence causing us to download and process it infinitely. Check that downloaded filenames are unique against those on file and reject processing if a duplicate. This patch delays this check to the ingest stage so that the same code is processed irrespective of transport Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #9 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 91413 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=91413&action=edit Bug 20254: Set correct member in working_directory Koha::Edifact::Transport working_directory was setting the wrong member name correct that method is only used in testing Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #10 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 91414 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=91414&action=edit Bug 20254: Add tests for transport ingest Add test framework for Koha::Edifact::Transport Add specific test that ingest does not duplicate an existing filename Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #91412|0 |1 is obsolete| | Attachment #91413|0 |1 is obsolete| | Attachment #91414|0 |1 is obsolete| | --- Comment #11 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 92353 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92353&action=edit Bug 20254: Do not ingest files with duplicate filenames If the supplier delivers the same file twice we are unable to rename it on the second occurence causing us to download and process it infinitely. Check that downloaded filenames are unique against those on file and reject processing if a duplicate. This patch delays this check to the ingest stage so that the same code is processed irrespective of transport Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #12 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 92354 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92354&action=edit Bug 20254: Set correct member in working_directory Koha::Edifact::Transport working_directory was setting the wrong member name correct that method is only used in testing Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #13 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 92355 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92355&action=edit Bug 20254: Add tests for transport ingest Add test framework for Koha::Edifact::Transport Add specific test that ingest does not duplicate an existing filename Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |19.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 --- Comment #14 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work! Pushed to master for 19.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20254 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED CC| |fridolin.somers@biblibre.co | |m --- Comment #15 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Enhancement not pushed to 19.05.x -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org