[Bug 11997] New: Download / export lists of records coding and format errors
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 Bug ID: 11997 Summary: Download / export lists of records coding and format errors Change sponsored?: --- Product: Koha Version: 3.14 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Lists Assignee: gmcharlt@gmail.com Reporter: pablo.bianchi@gmail.com QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Either from OPAC and Intranet, while exporting lists with formats "MARC" (OPAC) / "Iso2709" (on Intranet; different name to same thing) the downloaded file named "shelf.iso2709" is on latin1 instead of utf8, never reporting this, and also it seems very buggy: $ yaz-marcdump -f latin1 -t utf8 -i marc -o line "shelf.iso2709" | head -n30 [...] 245 00 $a Introducción general a la informática : $b periféricos y redes locales / $c Mario Carlos Ginzburg; ilustrado por Rafael Ginzburg (Separator but not at end of field length=134) 250 a4 $ ed. (Separator but not at end of field length=11) 260 aB $e nos Aires : $b Mario Carlos Ginzburg, $c 2006 (Separator but not at end of field length=51) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #1 from Pablo AB <pablo.bianchi@gmail.com> --- Selecting 8859-1 (latin1) while staging records for import doesn't fix the issue. A workaround was to: yaz-iconv -f latin1 -t utf8 shelf.iso2709 > shelf.iso2709-utf This "fix" the file and then yaz-marcdump shelf.iso2709-utf gives the expected output. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@biblibre.co | |m --- Comment #2 from Jonathan Druart <jonathan.druart@biblibre.com> --- Pablo, could you please test it again against the master branch? Some recent fixes related to the export and encoding have been pushed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=11944 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #3 from Pablo AB <pablo.bianchi@gmail.com> --- Still there, Koha 3.18.06.000: $ yaz-marcdump shelf.iso2709 [ugly thing with � and «(Separator but not at end of field...»] $ yaz-iconv -f latin1 -t utf8 shelf.iso2709 > shelf.iso2709-utf $ yaz-marcdump shelf.iso2709-utf [Quod erat demonstrandum :P Expected result] -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|3.14 |3.18 --- Comment #4 from Jonathan Druart <jonathan.druart@biblibre.com> --- Pablo, Bug 11944 has been pushed to master and will be in Koha 3.20. I can confirm you this bug is fixed on the master branch (tested right now). I don't plan to provide patches for encoding related issues on the branches without patches from bug 11944. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 Pablo AB <pablo.bianchi@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|3.18 |master --- Comment #5 from Pablo AB <pablo.bianchi@gmail.com> --- Hi Jonathan! The bug is still there in master. Now I'm using 3.20.03.000 and the I can see the record looks good on Koha, but exporting the problem is still there. You can look at it with the procedure of comment 3. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #6 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- I don't understand how you export your records. I don't manage to get a file with a .iso2709 extension. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #7 from Pablo AB <pablo.bianchi@gmail.com> --- - Search for books on intranet. - Clic on Add to and save them on a list. - Go to Home › Lists › Your lists - Clic on the list you just create - Clic on Download list and select "iso2709" - Save the "shelf.iso2709" file. - On terminal, follow steps on comment 3 NOTE: An ISO2709 is *almost* the same as a "MARC file". A mrc is an iso2709 that also have some other properties, like always subfields on tags>010. To avoid confusions maybe here template should say "MARC file" and filename should be: «list-<valid_name_of_list_name_for_filename>.mrc». -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #8 from Pablo AB <pablo.bianchi@gmail.com> --- Just created bug 14786 about this terminological issues. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Looks good here: 200 1 $a Une Fêteກ ຂ ຄ ງ ຈ ຊ ຍ monstre <test2> test3 $f Stéphanie Laslett $g ill. Nigel MacMullen $b LIVR $é éééééééééééééééééééééééééé $ñ ññññññññññññññññññññññññññ -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- % yaz-iconv -f latin1 -t utf8 shelf.iso2709 > shelf.iso2709-utf % yaz-marcdump shelf.iso2709-utf | grep 200 200 1 $a Une Fêteຠຠຠຠຠຠຠmonstre <test2> test3 $f Stéphanie Laslett $g ill. Nigel MacMullen $b LIVR $à ©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã©Ã© -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- tested on master and 3.20.02 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #12 from Pablo AB <pablo.bianchi@gmail.com> --- Well, not in my case. I tried on different Kohas, all using MARC 21. Maybe (I don't think so) have something to do with the fact that my records have newlines in it. Bug 11388. More tests: First on MySQL directly: <datafield tag="245" ind1="0" ind2="0"> <subfield code="a">Química : </subfield> $ yaz-marcdump bib-2087.utf8 bib-13459.utf8 | grep "^245" 245 00 $a Química : $b una ciencia experimental / [...] 245 00 $a Estudio experimental de dispersión de [...] $ yaz-marcdump shelf.iso2709 | grep "^245" 245 00 $a Qu�mica : $b una ciencia experimental / [...] [Notice here second record is missing. If my list have only one record above command throw no output.] $ yaz-iconv -f latin1 -t utf8 shelf.iso2709 > shelf.iso2709-utf $ yaz-marcdump shelf.iso2709-utf | grep "^245" 245 00 $a Química : $b una ciencia experimental / [...] 245 00 $a Estudio experimental de dispersión de [...] -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #13 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Could you attach the file here? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #14 from Pablo AB <pablo.bianchi@gmail.com> --- Created attachment 42528 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42528&action=edit Exported list to buggy ISO2709 (~MARC file) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11997 --- Comment #15 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- I got several (Separator but not at end of field length=96) Not sure it's normal. % yaz-marcdump shelf.iso | grep "^245" 245 00 $a Física / $c Paul A. Tipler $n [volumen 2] Where does the record come from? Manually cataloged into Koha? z3950 import? How does look the record using the MARC preview (detail page, link MARC Preview: Show)? -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org