[Koha-bugs] [Bug 31744] Inventory tool can take a long time and never complete

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31744

--- Comment #15 from David Cook <dcook at prosentient.com.au> ---
Created attachment 152370
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152370&action=edit
Bug 31744: Run inventory.pl as CGI with keep alives to prevent timeouts

This patches causes inventory.pl to run as CGI instead of PSGI,
since Plack::App::CGIBin buffers the whole response before sending any
data to the Apache reverse proxy.

This patch also causes inventory.pl to quickly output HTTP headers,
and print NULL characters roughly every 5 seconds until the HTML
or CSV data is ready to be printed. This prevents HTTP forward and
reverse proxies from timing out waiting for data from the inventory
script, which can be very slow.

In the future, we will want to move inventory.pl to use background
jobs, but this change makes inventory.pl usable for staff users stuck
behind HTTP forward proxies with short timeouts.

Test plan:
0. Apply patch
1. cp debian/templates/apache-shared-intranet-plack.conf
/etc/koha/apache-shared-intranet-plack.conf
2. cp debian/templates/apache-shared.conf
/etc/koha/apache-shared.conf
3. service apache2 restart

4. Go to create a SQL report from SQL
/cgi-bin/koha/reports/guided_reports.pl?phase=Create%20report%20from%20SQL
5. Save a report with the following SQL:
SELECT barcode FROM items where barcode <> '';
6. Run the report
7. Download as CSV
8. Edit the CSV and remove the "barcode" heading

9. Open F12 dev tools in browser and open the "Network" tab
10. Go to http://localhost:8081/cgi-bin/koha/mainpage.pl
11. Note that mainpage.pl has a "Content-Encoding: gzip" header
12. Go to http://localhost:8081/cgi-bin/koha/tools/inventory.pl
13. Note that inventory.pl does not have a "Content-Encoding: gzip"
header, while the .js and .css files do have this header.

14. Click "Choose file" and choose your barcode CSV file
15. Uncheck "Compare barcodes list to results"
16. Check "Export to CSV file"
17. Click "Submit"

18. Note that the download initially only increases in size by 1B at a time
19. Open inventory.csv with a text editor that can show symbols like Notepad++
20. Note approximately 7 or 8 NULL characters at start of file
21. Note that CSV loads without problems in Excel or LibreOffice (ie the NULL
characters are ignored)

22. Return to your browser and uncheck "Export to CSV file"
23. Click "Submit"
24. On the "Response" tab, note approximately 7 or 8 NULL characters at start
of response

--

25. For bonus points, set up a reverse proxy in front of inventory.pl
with a proxy timeout of 10 seconds and run an inventory job. Note that
the proxy times out without the patch, and does *not* time out with the patch.

-- 
You are receiving this mail because:
You are watching all bug changes.