[Bug 41162] New: Warn that printing large barcode ranges can lead to running out of memory
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Bug ID: 41162 Summary: Warn that printing large barcode ranges can lead to running out of memory Initiative type: --- Sponsorship --- status: Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: philip.orr@lmscloud.de QA Contact: testopia@bugs.koha-community.org If you are using the barcode range option in the label creator, if your barcode range is too large, your Koha server may run out of memory before the process finishes. This causes out of memory errors. This could happen if a library had e.g. "100101223" as their last used barcode and they wanted to print the next 50 barcodes and they make a mistake while entering the "To:" field in the barcode range creator. From: 100101224 To: 1000101274 You could very easily accidentally cause the label-create.pl process to create insane amounts of barcodes. In this case 900 million barcodes would be created, if my math is right. :) The memory usage will slowly increase as barcodes are created and "eventually" there will be no more memory left, causing OOM killer to strike. We should at least warn in the barcode range creation tool that such problems can happen if you aren't careful while entering values. I don't know if there's a good way to stop users from entering too large ranges, maybe there is a valid use case for a large barcode range if you have enough memory. To reproduce (assuming you are on ktd): 1. Check your current memory usage, e.g. with "ps aux --sort -rss" (or "free -h" to see how much memory is still free) 2. Make sure you know how to kill a process in Debian, e.g. "killall -9 /kohadevbox/koha/labels/label-create-pdf.pl" - you will need it soon.. 3. Go to Cataloging -> Label Creator -> Print barcode range 4. Enter in a number in "From:" and a crazy high number in "To:"; see above for an example 5. Print range -> Export (doesn't matter which templates etc.) -> Click on "Download as PDF" 5. Check your memory usage again, it should start spiking over the next minutes. The guilty process should be /kohadevbox/koha/labels/label-create-pdf.pl (assuming you are on ktd). 6. Either kill the process or you will run out of memory -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Nice spot, Philip. This is a tricky one. I hoped that maybe checking the size of the array created by ($from .. $to) would be enough, but it turns out that ($from .. $to) is a very heavy CPU and RAM intensive operation itself when using the values 100101224 and 1000101274. On the face of it, we could minus $to from $from to get a difference to try to validate, but... I think barcodes are not necessarily positive integers. The ".." Perl range operator also has a magical auto-increment algorithm when applied to strings, so it might be difficult to take into account that magic... That being said... if you put $from = 'ax' and increment is with $from++, you'll get 'ay' same as when using the ".." range operator. So using a for loop with ++ increment operator and a max count might work. That should have a number of advantages in terms of CPU and RAM usage actually since the calculation should be quite stable. We'd also want another variable to test the last generated value, because something like: "from *x to az" will only yield *x, so you wouldn't want to increment 10000 times and get *x for each iteration. Anyway... perhaps not a perfect solution but hopefully it would be practical? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- I briefly wondered if Perl could automagically do a diff on BARCODE9999 minus BARCODE1000 but no... it did not like that haha. You can increment and decrement but there are no operations that can automagically determine the difference it seems. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #3 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- *** Bug 42601 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #4 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 199142 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199142&action=edit Bug 41162: Don't allow strings of different length We already have a validator function to prevent this, however, it just returns 'true' in most cases when I believe it is meant to return the variable 'result' This won't prevent the letter cases highlighted in the comments, but as library barcodes are generally a fixed length it seems reasonable to at least restore this intention Note: In the cases where the inputs start with '0' we do some handling, but it seems only to be clearing previous errors? To test: 1 - Browse to Cataloging->Label Creator->Print barcode range 2 - Enter From: 1 To: 10 3 - Form is submitted 4 - Apply patch 5 - Reload the 'Print barcode range' page: http://localhost:8081/cgi-bin/koha/labels/label-edit-range.pl 6 - Enter From: 1 To: 10 7 - You are prevented from submitting 8 - Try any different length values - you are prevented -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199142|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #5 from David Nind <david@davidnind.com> --- Created attachment 199231 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199231&action=edit Bug 41162: Don't allow strings of different length We already have a validator function to prevent this, however, it just returns 'true' in most cases when I believe it is meant to return the variable 'result' This won't prevent the letter cases highlighted in the comments, but as library barcodes are generally a fixed length it seems reasonable to at least restore this intention Note: In the cases where the inputs start with '0' we do some handling, but it seems only to be clearing previous errors? To test: 1 - Browse to Cataloging->Label Creator->Print barcode range 2 - Enter From: 1 To: 10 3 - Form is submitted 4 - Apply patch 5 - Reload the 'Print barcode range' page: http://localhost:8081/cgi-bin/koha/labels/label-edit-range.pl 6 - Enter From: 1 To: 10 7 - You are prevented from submitting 8 - Try any different length values - you are prevented Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Laura O'Neil <laura@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rschumann@utah.gov -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Philip Orr <philip.orr@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199231|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #6 from Philip Orr <philip.orr@lmscloud.de> --- Created attachment 207280 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207280&action=edit Bug 41162: Don't allow strings of different length We already have a validator function to prevent this, however, it just returns 'true' in most cases when I believe it is meant to return the variable 'result' This won't prevent the letter cases highlighted in the comments, but as library barcodes are generally a fixed length it seems reasonable to at least restore this intention Note: In the cases where the inputs start with '0' we do some handling, but it seems only to be clearing previous errors? To test: 1 - Browse to Cataloging->Label Creator->Print barcode range 2 - Enter From: 1 To: 10 3 - Form is submitted 4 - Apply patch 5 - Reload the 'Print barcode range' page: http://localhost:8081/cgi-bin/koha/labels/label-edit-range.pl 6 - Enter From: 1 To: 10 7 - You are prevented from submitting 8 - Try any different length values - you are prevented Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Philip Orr <philip.orr@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #7 from Philip Orr <philip.orr@lmscloud.de> --- Nice work Nick. I noticed the logic can be circumvented by starting with a negative number. Not a realistic test case though and probably not worth the time it would take to rework. :-) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA QA Contact|testopia@bugs.koha-communit |andrew@bywatersolutions.com |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #207280|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 --- Comment #8 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 207824 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=207824&action=edit Bug 41162: Don't allow strings of different length We already have a validator function to prevent this, however, it just returns 'true' in most cases when I believe it is meant to return the variable 'result' This won't prevent the letter cases highlighted in the comments, but as library barcodes are generally a fixed length it seems reasonable to at least restore this intention Note: In the cases where the inputs start with '0' we do some handling, but it seems only to be clearing previous errors? To test: 1 - Browse to Cataloging->Label Creator->Print barcode range 2 - Enter From: 1 To: 10 3 - Form is submitted 4 - Apply patch 5 - Reload the 'Print barcode range' page: http://localhost:8081/cgi-bin/koha/labels/label-edit-range.pl 6 - Enter From: 1 To: 10 7 - You are prevented from submitting 8 - Try any different length values - you are prevented Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Philip Orr <philip.orr@lmscloud.de> Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41162 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |RESOLVED Resolution|--- |DUPLICATE --- Comment #9 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Work on Bug 42627 makes this patch moot, and 42627 is a better answer. *** This bug has been marked as a duplicate of bug 42627 *** -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org