[Bug 41620] New: Make itemsearch.pl use Koha::CSV for CSV generation
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Bug ID: 41620 Summary: Make itemsearch.pl use Koha::CSV for CSV generation Initiative type: --- Sponsorship --- status: Product: Koha Version: 24.11 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org This will save us a lot of headaches for sure. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |41619, 40880 CC| |tomascohen@gmail.com Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | Version|24.11 |Main Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40880 [Bug 40880] Exporting Item search results to csv, columns after Damaged Status are misaligned https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41619 [Bug 41619] Add `Koha::CSV` -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Medium patch Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 --- Comment #1 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 191467 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=191467&action=edit Bug 41620: Introduce Koha::CSV::ItemSearch This patch adds Koha::CSV::ItemSearch, a specialized CSV generator for item search results with predefined headers and formatting. The class: - Extends Koha::CSV with always_quote enabled for Excel compatibility - Provides 21 predefined column headers for item search exports - Formats Koha::Item objects into CSV-ready field arrays - Handles authorized values, dates, and library names - Supports MARC21 and UNIMARC title formatting This replaces the template-based CSV generation in itemsearch.pl, eliminating spacing issues and providing a testable, maintainable approach to item search CSV exports. Test plan: 1. Apply patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/CSV/ItemSearch.t => SUCCESS: Tests pass! 3. Tests cover: - Object initialization with always_quote enabled - Header generation (21 columns) - Item formatting with all field types - Authorized value lookups - Date formatting - Library name resolution - CSV output to filehandle 4. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 --- Comment #2 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 191468 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=191468&action=edit Bug 41620: Use Koha::CSV::ItemSearch for CSV exports This patch replaces template-based CSV generation with Koha::CSV::ItemSearch in both itemsearch.pl and item-export.pl, eliminating spacing issues around delimiters that caused Excel to display quotes as literal characters. Changes: - Update itemsearch.pl to use Koha::CSV::ItemSearch - Update item-export.pl to use Koha::CSV::ItemSearch - Remove obsolete CSV templates - Stream output directly without template overhead This fixes the issue where extra spaces around commas in the templates caused Excel to treat quotes as part of the data rather than CSV delimiters. Test plan: 1. Apply patches 2. Test itemsearch.pl: a. Go to Cataloging > Item search b. Perform a search c. Click "Export results" > "CSV" d. Open items.csv in Excel => SUCCESS: Fields display without quotes 3. Test item-export.pl: a. Go to a bibliographic record b. Select multiple items c. Click "Export selected items" d. Open items.csv in Excel => SUCCESS: Fields display without quotes 4. Verify all 21 columns are present and correctly formatted 5. Verify dates, authorized values, and library names display correctly 6. Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@openfifth.c | |o.uk, | |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41621 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41621 [Bug 41621] Use Koha::CSV everywhere [UMBRELLA] -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply CC| |david@davidnind.com --- Comment #3 from David Nind <david@davidnind.com> --- Getting a sha1/fake ancestor error when applying the patch: git bz apply 41620 Bug 41620 Depends on bug 41619 (Signed Off) Follow? [(y)es, (n)o] y Bug 41619 - Add `Koha::CSV` 191491 - Bug 41619: Add Koha::CSV Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 41619: Add Koha::CSV Bug 41620 - Make itemsearch.pl use Koha::CSV for CSV generation 191467 - Bug 41620: Introduce Koha::CSV::ItemSearch 191468 - Bug 41620: Use Koha::CSV::ItemSearch for CSV exports Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 41620: Introduce Koha::CSV::ItemSearch Applying: Bug 41620: Use Koha::CSV::ItemSearch for CSV exports error: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/en/includes/catalogue/itemsearch_item.csv.inc). error: could not build fake ancestor Patch failed at 0001 Bug 41620: Use Koha::CSV::ItemSearch for CSV exports hint: Use 'git am --show-current-patch=diff' to see the failed patch When you have resolved this problem run "git bz apply --continue". If you would prefer to skip this patch, instead run "git bz apply --skip". To restore the original branch and stop patching run "git bz apply --abort". Patch left in /tmp/Bug-41620-Use-KohaCSVItemSearch-for-CSV-exports-4t4s7561.patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |41617 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 [Bug 41617] CSV export from item search results - incorrect spaces after comma separator causes issues -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff --- Comment #4 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Sorry, it was missing a dependency. It applies cleanly so far. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 --- Comment #5 from David Nind <david@davidnind.com> --- (In reply to Tomás Cohen Arazi (tcohen) from comment #4)
Sorry, it was missing a dependency. It applies cleanly so far.
Thanks Tomás! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 --- Comment #6 from David Nind <david@davidnind.com> --- My observations from exporting - not sure if this is another issue, or what is expected. Happy to sign off it this is okay. Testing notes (using KTD): 1. CSV export and opening with a text editor: - All 21 columns exported - Comma separated, quotes around values - Items with accent characters: content is okay in the text file - Items with other characters: doesn't export correctly, for example record 233 where the title is "МОДЕРНИЗАЦИЯ НАЦИОНАЛЬНЫХ СИСТЕМ РАСПРОСТРАНЕНИЯ ЗНАНИЙ ПО ВОПРОСАМ СЕЛЬСКОГО ХОЗЯЙСТВА": " ÐÐÐÐÐ ÐÐÐÐЊÐЯ ÐÐЊÐÐÐÐÐЬÐЫХ СÐСТÐÐ Ð ÐСÐÐ ÐСТРÐÐÐÐÐЯ ÐÐÐÐÐÐ ÐÐ ÐÐÐÐ ÐСÐРСÐÐЬСÐÐÐРХÐÐЯÐСТÐÐ Qamar, M.K.","2008","FAO","Reference","39999000010114","513","","","Fairview","Fairview","General stacks","Books","","0","","","","2014-09-04","0","","" 2. CSV export and opening with LibreOffice: - All 21 columns exported - Items with accent characters: content is okay in the text file, but in LibreOffice doesn't show correctly, for example record 192 where the title is "Páipéir bhána agus páipéir bhreaca.": "P�ip�ir bh�na agus p�ip�ir bhreaca." � Cadhain, M�irt�n. - The item in the CSV that didn't show correcty, is shown correctly in LibreOffice: МОДЕРНИЗАЦИЯ НАЦИОНАЛЬНЫХ СИСТЕМ РАСПРОСТРАНЕНИЯ ЗНАНИЙ ПО ВОПРОСАМ СЕЛЬСКОГО ХОЗЯЙСТВА Qamar, M.K. 2. CSV export and opening with Excel (used the web-based version of Excel): - All 21 columns exported - For the barcode column, shows as 4E+13 (from the item search results) or 3.9999E+13 (record holding table export) - changing the column to a number displays the number correctly - Items with accent characters: shows correctly (which is the opposite of LIbreOffice!) - Items with other characters: don't show correctly, for the one above: МОДЕРÐИЗÐЦИЯ ÐÐЦИОÐÐЛЬÐЫХ СИСТЕМ Ð ÐСПРОСТРÐÐЕÐИЯ ЗÐÐÐИЙ ПО ВОПРОСÐÐœ СЕЛЬСКОГО ХОЗЯЙСТВРQamar, M.K. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Bug 41620 depends on bug 41617, which changed state. Bug 41617 Summary: CSV export from item search results - incorrect spaces after comma separator causes issues https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41617 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Bug 41620 depends on bug 41619, which changed state. Bug 41619 Summary: Add `Koha::CSV` https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41619 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41620 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- QA tests pass, but there are several smaller issues: 1) Translatability + if ( C4::Context->preference('marcflavour') eq 'UNIMARC' && $author ) { + return "$title by $author"; + } 2) Formatting a) Without this patch the dates are formatted to DateFormat, but with this patch we see YYYY-MM-DD for date accessioned, last checkout date and due date. b) Not for loan status Without the patch the not for loan status is empty, like the other status columns, with this patch, a 0 displays. The previous behavior is more consistent. 3) Encoding Before this patch when I open the CSV file with LibreOffice in Linux, the diacritics do not always display correctly anymore. I am using all item of the sample ktd database for testing. Broken records include: C�line et Julie vont en bateau The tin drum / Grass, G�nter, Strangely enough, the Kyrillic and Arabic records appear unharmed. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org