[Bug 8437] New: Plack and database backup
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Priority: P2 Change sponsored?: --- Bug ID: 8437 Assignee: jcamins@cpbibliography.com Blocks: 8268 Summary: Plack and database backup Severity: major Classification: Unclassified OS: All Reporter: paul.poulain@biblibre.com Hardware: All Status: ASSIGNED Version: master Component: Tools Product: Koha I've found a problem with patch of bug 8268, that you can see when you run plack: Variable "$backupdir" is not available at /home/paul/koha.dev/koha-community/tools/export.pl line 334. => the backup features won't work under plack (problem of variable scoping with Plack that put the script in a sub, making main() variable unaccessible in other subs. Problem solved everywhere with a our instead of my. The best option, that I would prefer -from far- for this new feature, being that we don't use such a our variable, but define backupdir where it's needed (+ it's just reading of a syspref, so not something costly) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #1 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- Created attachment 11177 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11177&action=edit Bug 8437: variable scoping for tools/export.pl Variable scoping problems prevented database export from working under Plack. This patch fixes that. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dpavlin@rot13.org --- Comment #2 from Paul Poulain <paul.poulain@biblibre.com> --- Dobrica, could you sign-off this one pls ? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #11177|0 |1 is obsolete| | --- Comment #3 from Dobrica Pavlinusic <dpavlin@rot13.org> --- Created attachment 11353 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11353&action=edit Bug 8437: variable scoping for tools/export.pl Variable scoping problems prevented database export from working under Plack. This patch fixes that. Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #4 from Dobrica Pavlinusic <dpavlin@rot13.org> --- Signed off using test scenario from Bug 8268 - Koha should offer way to backup entire db -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #5 from Dobrica Pavlinusic <dpavlin@rot13.org> --- Created attachment 11354 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11354&action=edit Bug 8437 - fix koha-dump configuration tar permissions This allows backups created using Debian package to show up automatically in web interface and makes permissions of database files consistent. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #6 from Dobrica Pavlinusic <dpavlin@rot13.org> --- Created attachment 11355 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11355&action=edit Bug 8437 - fix koha-dump configuration tar permissions This allows backups created using Debian package to show up automatically in web interface and makes permissions of backup files consistent. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #11354|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #7 from Dobrica Pavlinusic <dpavlin@rot13.org> --- This patch fixes errors under plack but doesn't work for large backup files. The problem is in download_backup which tries to print data back to browser in 64K chunks. Under plack, that gets buffered in memory and with backup of our production database it dies after a while and returns zero length file. Correct way to fix this is to issue redirect under plack in download_backup to some plack handled URL so we don't read whole file in memory. I don't know how to detect plack if we are running under plack since script is inside CGI::Compile, but we could use environment variable to signal that we are under plack. That would, however, require us to have some kind of koha.psgi in which we can implement this in master. I also had to remove && not $filename =~ m#|# from download_backup to make download work at all, so I will have to remove signed-off on this patch, sorry. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #11177|1 |0 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #11353|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #8 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- (In reply to comment #7)
This patch fixes errors under plack but doesn't work for large backup files.
The problem is in download_backup which tries to print data back to browser in 64K chunks. Under plack, that gets buffered in memory and with backup of our production database it dies after a while and returns zero length file.
Correct way to fix this is to issue redirect under plack in download_backup to some plack handled URL so we don't read whole file in memory.
I don't know how to detect plack if we are running under plack since script is inside CGI::Compile, but we could use environment variable to signal that we are under plack.
That would, however, require us to have some kind of koha.psgi in which we can implement this in master.
I guess the thing to do is note that this is not available under Plack?
I also had to remove
&& not $filename =~ m#|#
from download_backup to make download work at all, so I will have to remove signed-off on this patch, sorry.
There's a signed off patch which makes that last change on bug 8268. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #9 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- (In reply to comment #7)
This patch fixes errors under plack but doesn't work for large backup files.
The problem is in download_backup which tries to print data back to browser in 64K chunks. Under plack, that gets buffered in memory and with backup of our production database it dies after a while and returns zero length file.
Correct way to fix this is to issue redirect under plack in download_backup to some plack handled URL so we don't read whole file in memory.
This can't be how exporting large blobs from a database would be handled under Plack. Do you have any idea how that would work? Maybe we can use the same technique.
I don't know how to detect plack if we are running under plack since script is inside CGI::Compile, but we could use environment variable to signal that we are under plack.
That would, however, require us to have some kind of koha.psgi in which we can implement this in master.
-- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #10 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- Would using this syntax (borrowed from bug 7952) fix the problem? + open(my $fh, '<', $filename) + print <$fh>; + close $fh; -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 --- Comment #11 from Paul Poulain <paul.poulain@biblibre.com> --- (In reply to comment #10)
Would using this syntax (borrowed from bug 7952) fix the problem?
+ open(my $fh, '<', $filename) + print <$fh>; + close $fh;
Bumping Jared's question ! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Robin Sheat <robin@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |robin@catalyst.net.nz --- Comment #12 from Robin Sheat <robin@catalyst.net.nz> --- (In reply to Jared Camins-Esakov from comment #10)
Would using this syntax (borrowed from bug 7952) fix the problem?
+ open(my $fh, '<', $filename) + print <$fh>; + close $fh;
If we're dealing with big binary files, this may consume a lot of memory before it sees a newline. Also, we possibly run the risk of changing newlines if we're ever different (or more lenient) from what we accept to what we output. Needless to say, this is bad for binary files. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437 Mike <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org