[Bug 12025] New: Inventory tool fails with "Could not reads headers", or sometimes a timeout
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Bug ID: 12025 Summary: Inventory tool fails with "Could not reads headers", or sometimes a timeout Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: critical Priority: P5 - low Component: Tools Assignee: gmcharlt@gmail.com Reporter: jason@esilibrary.com QA Contact: testopia@bugs.koha-community.org Created attachment 26759 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=26759&action=edit sample file This happens with small test files of one or two barcodes. jason-temp@lumen:~$ od BARCODES.TXT 0000000 030063 033065 030463 030060 030060 030060 033062 031412 0000020 032460 031466 030061 030060 030060 031460 000064 0000035 jason-temp@lumen:~$ cat BARCODES.TXT 30563100000026 30563100000034jason-temp@lumen:~$ On the server side, it looks inventory.pl spins out of control, and is looping on commands like this: write(5, "s\0\0\0\3select authorised_value fro"..., 119) = 119 read(5, "\1\0\0\1\1h\0\0\2\3def\4salt\27marc_subfield"..., 16384) = 144 poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) write(5, "Y\0\0\0\3select authorised_value, li"..., 93) = 93 read(5, "\1\0\0\1\3\\\0\0\2\3def\4salt\21authorised_va"..., 16384) = 337 poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) write(5, "\1\0\0\0\16", 5) = 5 read(5, "\7\0\0\1\0\0\0\2\0\0\0", 16384) = 11 poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) write(5, "\1\0\0\0\16", 5) = 5 read(5, "\7\0\0\1\0\0\0\2\0\0\0", 16384) = 11 poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) This is with Koha 3.14.03.000, but Chris Cormack confirmed that it happens with master, 3.14.1, and 3.14.5, but that it works fine on 3.12.12. He tracked it down to this commit for this bug: 0313856d9a1a9c0c002da0aead8c8516b51e1c11 http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7684 There is an email thread discussing this on the Koha list: http://lists.katipo.co.nz/public/koha/2014-March/039092.html http://lists.katipo.co.nz/public/koha/2014-April/thread.html#39127 With the Chrome development tools, I also see the following errors when I first load the interface: Uncaught SyntaxError: Unexpected token ILLEGAL inventory.pl:62 event.returnValue is deprecated. Please use the standard event.preventDefault() instead. jquery.js:3 Let me know if there's anything else I can do to help. Thanks! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m Depends on| |7684 --- Comment #1 from Jonathan Druart <jonathan.druart@biblibre.com> --- Hello Jason, I don't reproduce the issue against master (and even on the 3.14.x branch). On the inventory tools page (tools/inventory.pl), I filled the "Barcode file" with a barcodes file, I clicked on submit and I did not get any error. Could you try to reproduce on a sandbox please? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #2 from Galen Charlton <gmcharlt@gmail.com> --- This bug depends on the number of item records in the database. When you upload a file of barcodes, and do not specify any filters on the rest of the form, it effectively runs GetItemsForInventory() such that *every* item in the database is returned. On a large enough database, this takes so long that the script times out. The current documentation in the manual implies that if you upload a file of barcodes, the *only* thing it does is set the date-last-seen field. This is not entirely true; it also tries to compare the list of scanned items to what is expected to be seen in the list of items. The work-around for libraries with a large number of item records is to, when uploading a file of barcodes, to also set the filters (e.g., library, shelving location, call number range, etc.) to specify the range of items that the file of barcodes corresponds to. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #3 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 28401 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28401&action=edit Bug 12025 - Inventory tool fails with "Could not reads headers", or sometimes a timeout For libraries with large numbers of records, using a file of barcodes in the inventory tool without selecting filters to limit the catalog selection to compare against can result in an error of "could not read headers" or a page timeout. When you upload a file of barcodes, and do not specify any filters on the rest of the form, it effectively runs GetItemsForInventory() such that *every* item in the database is returned. On a large enough database, this takes so long that the script times out. The current documentation in the manual implies that if you upload a file of barcodes, the *only* thing it does is set the date-last-seen field. This is not entirely true; it also tries to compare the list of scanned items to what is expected to be seen in the list of items. The work-around for libraries with a large number of item records is to, when uploading a file of barcodes, to also set the filters (e.g., library, shelving location, call number range, etc.) to specify the range of items that the file of barcodes corresponds to. This patch adds a warning when a librarian attempts to upload a barcode list file for comparison without selecting any filters to limit the catalog selection. Test Plan: 1) Apply this patch 2) Browse to the inventory tool 3) Click the "Choose File" button, select a file 4) Click the "Submit" button 5) You should see a warning message 6) Click "Cancel" 7) Select one of more filters 8) Click the "Submit" button 9) This time the script should proceed without triggering the warning. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com Assignee|gmcharlt@gmail.com |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #4 from Jonathan Druart <jonathan.druart@biblibre.com> --- Comment on attachment 28401 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28401 Bug 12025 - Inventory tool fails with "Could not reads headers", or sometimes a timeout Review of attachment 28401: --> (http://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=12025&attachment=28401) ----------------------------------------------------------------- ::: koha-tmpl/intranet-tmpl/prog/en/modules/tools/inventory.tt @@ -99,5 @@
</fieldset>
- <!-- submit button for uploaded file --> - <fieldset class="action"><input type="submit" value="Submit" class="button" /></fieldset> -
This is a "feature". Why did you remove it? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28401|0 |1 is obsolete| | --- Comment #5 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 28404 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28404&action=edit [SIGNED-OFF] Bug 12025 - Inventory tool fails with "Could not reads headers", or sometimes a timeout For libraries with large numbers of records, using a file of barcodes in the inventory tool without selecting filters to limit the catalog selection to compare against can result in an error of "could not read headers" or a page timeout. When you upload a file of barcodes, and do not specify any filters on the rest of the form, it effectively runs GetItemsForInventory() such that *every* item in the database is returned. On a large enough database, this takes so long that the script times out. The current documentation in the manual implies that if you upload a file of barcodes, the *only* thing it does is set the date-last-seen field. This is not entirely true; it also tries to compare the list of scanned items to what is expected to be seen in the list of items. The work-around for libraries with a large number of item records is to, when uploading a file of barcodes, to also set the filters (e.g., library, shelving location, call number range, etc.) to specify the range of items that the file of barcodes corresponds to. This patch adds a warning when a librarian attempts to upload a barcode list file for comparison without selecting any filters to limit the catalog selection. Test Plan: 1) Apply this patch 2) Browse to the inventory tool 3) Click the "Choose File" button, select a file 4) Click the "Submit" button 5) You should see a warning message 6) Click "Cancel" 7) Select one of more filters 8) Click the "Submit" button 9) This time the script should proceed without triggering the warning. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Work as described following test plan. No errors -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |bgkriegel@gmail.com --- Comment #6 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- (In reply to Jonathan Druart from comment #4)
This is a "feature". Why did you remove it?
I think the pourpose of this patch is to force the selection of a filter, so removing this first submit makes sense, the one left is under filters. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nengard@gmail.com --- Comment #7 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #6)
(In reply to Jonathan Druart from comment #4)
This is a "feature". Why did you remove it?
I think the pourpose of this patch is to force the selection of a filter, so removing this first submit makes sense, the one left is under filters.
Yes but it causes a regression with bug 11272. Nicole could I get your opinion on this change please? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #8 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- (In reply to Jonathan Druart from comment #7)
(In reply to Bernardo Gonzalez Kriegel from comment #6)
(In reply to Jonathan Druart from comment #4)
This is a "feature". Why did you remove it?
I think the pourpose of this patch is to force the selection of a filter, so removing this first submit makes sense, the one left is under filters.
Yes but it causes a regression with bug 11272. Nicole could I get your opinion on this change please?
You are right. I tested re-adding it (first submit) and still works, generating the (scary) message Perhaps switch to in discussion? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gmcharlt@gmail.com --- Comment #9 from Galen Charlton <gmcharlt@gmail.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #8)
(In reply to Jonathan Druart from comment #7)
(In reply to Bernardo Gonzalez Kriegel from comment #6)
(In reply to Jonathan Druart from comment #4)
This is a "feature". Why did you remove it?
I think the pourpose of this patch is to force the selection of a filter, so removing this first submit makes sense, the one left is under filters.
Yes but it causes a regression with bug 11272. Nicole could I get your opinion on this change please?
You are right. I tested re-adding it (first submit) and still works, generating the (scary) message Perhaps switch to in discussion?
I think the regression on 11272 is acceptable. This is because the premise of that bug is that uploading a barcode file and generating a list of items to view are completely independent operations. They are not: if you upload a file of barcodes, the code doesn't just mark them seen, it also tries to compare them against a range of items specified by the filter fields to identify missing ones. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #10 from Galen Charlton <gmcharlt@gmail.com> --- Because of the string change, I will not be pushing it today, but unless I run into a problem during testing, I fully expect this to make it in for 3.16.1. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 --- Comment #11 from M. de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 28519 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28519&action=edit Bug 6254: (follow-up) work-around to fix a failing test case This patch ensures that a patron category that is present in the (English) sample data is used for the ILS-DI tests, avoiding a situation where one part of AddMember was trying to access a patron category via DBIC but failed because it was not visible to it yet. Note that this will become easier once C4::Context->dbh and DBIC are using the same base DBI database handles. Signed-off-by: Galen Charlton <gmc@esilibrary.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #12 from M. de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to M. de Rooy from comment #11)
Created attachment 28519 [details] [review]
Oops. Wrong command :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28519|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=11272 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #28404|0 |1 is obsolete| | --- Comment #13 from M. de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 28521 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=28521&action=edit Bug 12025 - Inventory tool fails with "Could not reads headers", or sometimes a timeout For libraries with large numbers of records, using a file of barcodes in the inventory tool without selecting filters to limit the catalog selection to compare against can result in an error of "could not read headers" or a page timeout. When you upload a file of barcodes, and do not specify any filters on the rest of the form, it effectively runs GetItemsForInventory() such that *every* item in the database is returned. On a large enough database, this takes so long that the script times out. The current documentation in the manual implies that if you upload a file of barcodes, the *only* thing it does is set the date-last-seen field. This is not entirely true; it also tries to compare the list of scanned items to what is expected to be seen in the list of items. The work-around for libraries with a large number of item records is to, when uploading a file of barcodes, to also set the filters (e.g., library, shelving location, call number range, etc.) to specify the range of items that the file of barcodes corresponds to. This patch adds a warning when a librarian attempts to upload a barcode list file for comparison without selecting any filters to limit the catalog selection. Test Plan: 1) Apply this patch 2) Browse to the inventory tool 3) Click the "Choose File" button, select a file 4) Click the "Submit" button 5) You should see a warning message 6) Click "Cancel" 7) Select one of more filters 8) Click the "Submit" button 9) This time the script should proceed without triggering the warning. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Work as described following test plan. No errors Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |tomascohen@gmail.com --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master. Thanks Kyle! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable --- Comment #15 from Galen Charlton <gmcharlt@gmail.com> --- Pushed to 3.16.x for inclusion in 3.16.1. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 Fridolin SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolyn.somers@biblibre.co | |m --- Comment #16 from Fridolin SOMERS <fridolyn.somers@biblibre.com> --- Pushed to 3.14.x, will be in 3.14.12. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12025 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtompset@hotmail.com See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=14057 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org