[Bug 41617] New: itemsearch.pl produces wrong CSV
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Bug ID: 41617 Summary: itemsearch.pl produces wrong CSV Initiative type: --- Sponsorship --- status: Product: Koha Version: 24.11 Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org I've been following the CSV generation code in Koha for security reasons, and didn't notice this particular case until now. This script generates CSV using... templates instead of Text::CSV. Besides the fact this is wrong, there's a bug in the generated CSV. Bug 40880 solved an issue with the headers, but the produced file contains things like this: ``` "First field", "Second, field", "Third" ``` the extra space after the comma makes it get interpreted like: ``` First field <space>"Second <space>field" <space>"Third" ``` -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Depends on| |40880 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40880 [Bug 40880] Exporting Item search results to csv, columns after Damaged Status are misaligned -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | Status|NEW |Needs Signoff Version|24.11 |Main -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #1 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 191439 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=191439&action=edit Bug 41617: Remove extra spaces breaking CSV formatting This patch removes extra spaces breaking CSV parsing in tools like Excel. To test: 1. Perform an item search selecting the CSV option 2. Open the CSV with a TEXT EDITOR of your choice => FAIL: Notice the extra spaces after the commas (those used as separators, no the ones within the double quotes) 3. Apply this patch 4. Repeat 1-2 => SUCCESS: No extra spaces! 5. Sign off :-D Note: On the first attempt you can open the file on Excel if you have it and notice things are not aligned. Note2: Using double quotes on all fields regardless of the context works for most parsers, but is not correct CSV. Will deal with it on a separate bug. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Katie Bliss <kebliss@dmpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kebliss@dmpl.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #191439|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 191582 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=191582&action=edit Bug 41617: Remove extra spaces breaking CSV formatting This patch removes extra spaces breaking CSV parsing in tools like Excel. To test: 1. Perform an item search selecting the CSV option 2. Open the CSV with a TEXT EDITOR of your choice => FAIL: Notice the extra spaces after the commas (those used as separators, no the ones within the double quotes) 3. Apply this patch 4. Repeat 1-2 => SUCCESS: No extra spaces! 5. Sign off :-D Note: On the first attempt you can open the file on Excel if you have it and notice things are not aligned. Note2: Using double quotes on all fields regardless of the context works for most parsers, but is not correct CSV. Will deal with it on a separate bug. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This fixes the CSV export release notes| |from item search results in | |the staff interface (Search | |> Item search> Export | |select results (X) to CSV). | | | |It removes extra spaces | |after the comma separator, | |which causes issues when | |using the CSV file with | |some applications (such as | |Microsoft Excel). CC| |david@davidnind.com Summary|itemsearch.pl produces |CSV export from item search |wrong CSV |results - incorrect spaces | |after comma separator | |causes issues -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- This does not improve the readability of this include. Could you add newlines and remove them again with the TT syntax in the output? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #4 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- (In reply to Marcel de Rooy from comment #3)
This does not improve the readability of this include. Could you add newlines and remove them again with the TT syntax in the output?
Even if I remove the file on bug 41620? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41620 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 [Bug 41620] Make itemsearch.pl use Koha::CSV for CSV generation -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #5 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Tomás Cohen Arazi (tcohen) from comment #4)
(In reply to Marcel de Rooy from comment #3)
This does not improve the readability of this include. Could you add newlines and remove them again with the TT syntax in the output?
Even if I remove the file on bug 41620?
Haha. There is nothing above improvement before deletion ;) Clean garbage can. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #6 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 192337 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192337&action=edit Bug 41617: Remove extra spaces breaking CSV formatting This patch removes extra spaces breaking CSV parsing in tools like Excel. To test: 1. Perform an item search selecting the CSV option 2. Open the CSV with a TEXT EDITOR of your choice => FAIL: Notice the extra spaces after the commas (those used as separators, no the ones within the double quotes) 3. Apply this patch 4. Repeat 1-2 => SUCCESS: No extra spaces! 5. Sign off :-D Note: On the first attempt you can open the file on Excel if you have it and notice things are not aligned. Note2: Using double quotes on all fields regardless of the context works for most parsers, but is not correct CSV. Will deal with it on a separate bug. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #191582|0 |1 is obsolete| | Attachment #192337|0 |1 is obsolete| | --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 192603 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192603&action=edit Bug 41617: Remove extra spaces breaking CSV formatting This patch removes extra spaces breaking CSV parsing in tools like Excel. To test: 1. Perform an item search selecting the CSV option 2. Open the CSV with a TEXT EDITOR of your choice => FAIL: Notice the extra spaces after the commas (those used as separators, no the ones within the double quotes) 3. Apply this patch 4. Repeat 1-2 => SUCCESS: No extra spaces! 5. Sign off :-D Note: On the first attempt you can open the file on Excel if you have it and notice things are not aligned. Note2: Using double quotes on all fields regardless of the context works for most parsers, but is not correct CSV. Will deal with it on a separate bug. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |26.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #8 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 26.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Jacob O'Mara <jacob.omara@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|26.05.00 |26.05.00,25.11.03 released in| | Status|Pushed to main |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #9 from Jacob O'Mara <jacob.omara@openfifth.co.uk> --- Thanks all, pushed to 25.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Laura Escamilla <Laura.escamilla@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|26.05.00,25.11.03 |26.05.00,25.11.03,25.05.09 released in| | Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 --- Comment #10 from Laura Escamilla <Laura.escamilla@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.05.09 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m Status|Pushed to oldstable |Needs documenting --- Comment #11 from Fridolin Somers <fridolin.somers@biblibre.com> --- Depends on Bug 40880 not in 24.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED --- Comment #12 from David Nind <david@davidnind.com> --- Bug fix, no changes to the manual required. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org