http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Assignee|gmcharlt@gmail.com |m.de.rooy@rijksmuseum.nl Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 42866 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42866&action=edit Bug 14893: Separate temporary storage per instance in Upload.pm To make life easier when multiple instances are uploading temporary files to Koha, this patch adds the database name to the upload subfolder in your /tmp folder. Note: Although multiple instances could share the same subfolder for temporary storage (hashvalue is based on a timestamp too), it will be better to separate them for efficient housekeeping (removing older or partial files with a cronjob etc.) Since multiple instances come with separate permissions, keeping them in separate folders will be much simpler. Permanent storage is not affected by this patch. The location of permanent storage is ruled by the upload_path in each config file. Sharing that space is not recommended too. Although it may not be strictly necessary yet to remove files from the old temp storage folder (before the 3.22 release), the accompanying db rev performs that housekeeping task. Test plan: [1] Do not yet apply this patch. Upload a temporary file (use the tools/upload.pl script without selecting a category). [2] Check /tmp/koha_upload. [3] Apply this patch. Run the db rev with web installer. [4] Upload another temporary file. [5] Check /tmp for folder [your_database]_upload. [6] Check that /tmp/koha_upload is gone and the associated records too. [7] Run the adjusted t/db../Upload.t -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=14294 -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to M. Tompsett from comment #2)

(In reply to Marcel de Rooy from comment #1)

...

To make life easier when multiple instances are uploading temporary files to Koha, this patch adds the database name to the upload subfolder in your /tmp folder.

/tmp may empty after a reboot. I think I best do more testing to understand file flow.

Not necessarily. Depends on your configuration. On Debian, check /etc/default/rcS # delete files in /tmp during boot older than x days. # '0' means always, -1 or 'infinite' disables the feature #TMPTIME=0 Note that when the OS empties the /tmp folder, this is not a problem. The files that we stored there were temporary. A future cronjob will remove all entries in the upload table that are not found anymore. Of course, one should not set /tmp as the location for permanent storage :) -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 Mirko Tietgen <mirko@abunchofthings.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mirko@abunchofthings.net -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 Marc Véron <veron@veron.ch> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #42866|0 |1 is obsolete| | --- Comment #5 from Marc Véron <veron@veron.ch> --- Created attachment 46826 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=46826&action=edit Bug 14893: Separate temporary storage per instance in Upload.pm To make life easier when multiple instances are uploading temporary files to Koha, this patch adds the database name to the upload subfolder in your /tmp folder. Note: Although multiple instances could share the same subfolder for temporary storage (hashvalue is based on a timestamp too), it will be better to separate them for efficient housekeeping (removing older or partial files with a cronjob etc.) Since multiple instances come with separate permissions, keeping them in separate folders will be much simpler. Permanent storage is not affected by this patch. The location of permanent storage is ruled by the upload_path in each config file. Sharing that space is not recommended too. Although it may not be strictly necessary yet to remove files from the old temp storage folder (before the 3.22 release), the accompanying db rev performs that housekeeping task. Test plan: [1] Do not yet apply this patch. Upload a temporary file (use the tools/upload.pl script without selecting a category). [2] Check /tmp/koha_upload. [3] Apply this patch. Run the db rev with web installer. [4] Upload another temporary file. [5] Check /tmp for folder [your_database]_upload. [6] Check that /tmp/koha_upload is gone and the associated records too. [7] Run the adjusted t/db../Upload.t Followed test plan. Could not do steps before applying patch [1]-[2] (I suppose a local permission problem). After applying patch and updating db file appears in /tmp/[database] as expected. Signed-off-by: Marc Véron <veron@veron.ch> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Thanks Marc and Magnus. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #8)

Marcel, Why do you remove temporary uploads?

Because of their temporary nature :) When I wrote this, I had the hope to still include this in 3.22. So deleting these files was not an issue. But, since these files have been uploaded as temporary, I imagine that we can delete them as well at the time of an upgrade to the next release. A future cron job should do that too (e.g. after x days). For a server with multiple instances it would be more complex. Instead of picking the files for each instance and moving them to different tmp subfolders, it is much easier to remove them. Note also that we could have permission problems here (even with deleting the folder). -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14893 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #46826|0 |1 is obsolete| | --- Comment #10 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 47198 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=47198&action=edit [PASSED QA] Bug 14893: Separate temporary storage per instance in Upload.pm To make life easier when multiple instances are uploading temporary files to Koha, this patch adds the database name to the upload subfolder in your /tmp folder. Note: Although multiple instances could share the same subfolder for temporary storage (hashvalue is based on a timestamp too), it will be better to separate them for efficient housekeeping (removing older or partial files with a cronjob etc.) Since multiple instances come with separate permissions, keeping them in separate folders will be much simpler. Permanent storage is not affected by this patch. The location of permanent storage is ruled by the upload_path in each config file. Sharing that space is not recommended too. Although it may not be strictly necessary yet to remove files from the old temp storage folder (before the 3.22 release), the accompanying db rev performs that housekeeping task. Test plan: [1] Do not yet apply this patch. Upload a temporary file (use the tools/upload.pl script without selecting a category). [2] Check /tmp/koha_upload. [3] Apply this patch. Run the db rev with web installer. [4] Upload another temporary file. [5] Check /tmp for folder [your_database]_upload. [6] Check that /tmp/koha_upload is gone and the associated records too. [7] Run the adjusted t/db../Upload.t Followed test plan. Could not do steps before applying patch [1]-[2] (I suppose a local permission problem). After applying patch and updating db file appears in /tmp/[database] as expected. Signed-off-by: Marc Véron <veron@veron.ch> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.