[Bug 12174] New: Zotero connection with Koha: problem with UTF-8
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Bug ID: 12174 Summary: Zotero connection with Koha: problem with UTF-8 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: minor Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: gmcharlt@gmail.com Reporter: z.tajoli@cineca.it QA Contact: testopia@bugs.koha-community.org If i donwload Koha records with Zotero from server with Koha 3.12/3.14, I have problems with UTF-8. I don't had this problem with an old 3.2 in the past. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Change sponsored?|--- |Sponsored Assignee|gmcharlt@gmail.com |z.tajoli@cineca.it -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 --- Comment #1 from Zeno Tajoli <z.tajoli@cineca.it> --- I find the problem. It is a mix of apache / new Zotero working. 1)The bug is not present in installation that use Ngix instead of apache. 2)Zotero doesn't use only COinS from Koha but if use also the record export (../cgi-bin/koha/opac-export.pl) 3)The cgi opac-export.pl it is ok to write a file on this but not if you try to use it like a stream in Zotero. Try: curl -i 'http://pust.urbe.it/cgi-bin/koha/opac-export.pl?format=utf8&op=export&bib=71000&save=Go' The result: HTTP/1.1 200 OK Date: Fri, 02 May 2014 09:41:22 GMT Server: Apache/2.2.22 (Debian) Content-Disposition: attachment; filename="bib-71000.utf8" Transfer-Encoding: chunked Content-Type: application/octet-stream; charset=ISO-8859-1 01403nam a2200265 i 4500001000600000003000900006005001700015 [the iso2709 output] 4)The header "charset=ISO-8859-1" is the default on Apache (Ngix doesn't inser anything instead). 5)Zotero read the header and it thinks that the stream in iso2709 format is written with ISO-8859-1 charset. So the fix is to be esplict on charset when we print the result. From 94: print $query->header( 95: -type => 'application/octet-stream', 96: -attachment=>"bib-$biblionumber.$format"); To 94: print $query->header( 95: -type => 'application/octet-stream', 96: -charset=>'utf-8', 97: -attachment=>"bib-$biblionumber.$format"); I will write and send the patch soon -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 --- Comment #2 from Zeno Tajoli <z.tajoli@cineca.it> --- Created attachment 27923 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=27923&action=edit file opac-export.pl is patched This patch solves the bug for UTF-8 output. MARC-8 works only if you save it as a file. In fact the standard Internet stack (browser, web server, etc.) doesn't know anything about MARC-8 or it's ISO base (ISO 2022). To test the bug: -- without the patch use Zotero on a recent Koha install with Apache. You can use PTFS sandboxes. You can't use Biblibre sandboxes, they use Ngix. -- Test a record with special chars. -- Install the patch -- Catch (using Zotero) the same record. -- Test all directly download from Opac. -- To check MARC-8, use a tool that support it. For example MarcEdit. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Zeno Tajoli <z.tajoli@cineca.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #27923|0 |1 is obsolete| | --- Comment #3 from Zeno Tajoli <z.tajoli@cineca.it> --- Created attachment 27924 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=27924&action=edit file opac-export.pl is patched -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 --- Comment #4 from Zeno Tajoli <z.tajoli@cineca.it> --- Created attachment 27925 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=27925&action=edit a follow-up to fix a missing print -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 --- Comment #5 from Zeno Tajoli <z.tajoli@cineca.it> --- To test the bug: -- without the patch use Zotero on a recent Koha install with Apache. You can use PTFS sandboxes. You can't use Biblibre sandboxes, they use Ngix. -- Test a record with special chars. -- Install the patch -- Catch (using Zotero) the same record. -- Test all directly download from Opac. -- To check MARC-8, use a tool that support it. For example MarcEdit. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #27924|0 |1 is obsolete| | --- Comment #6 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 29137 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=29137&action=edit [SIGNED-OFF] Bug 12174 - Zotero connection with Koha: problem with UTF-8 This patch solves the bug for UTF-8 output. MARC-8 works only if you save it as a file. In fact the standard Internet stack (browser, web server, etc.) doesn't know anything about MARC-8 or it's ISO base (ISO 2022). To test the bug: -- without the patch use Zotero on a recent Koha install with Apache. You can use PTFS sandboxes. You can't use Biblibre sandboxes, they use Ngix. -- Test a record with special chars. -- Install the patch -- Catch (using Zotero) the same record. -- Test all directly download from Opac. -- To check MARC-8, use a tool that support it. For example MarcEdit. http://bugs.koha-community.org/show_bug.cgi?id=12174 Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Work as described Tested with zotero add-on for firefox. Before patch bad diacritics, after patch ok Record exported as MARC-8, transformed using yaz-iconv -f marc8 -t utf-8 bib.marc8 > bib.utf8 yaz-marcdump -i marc -o line bib.utf8, checking accents, also comparing with direct utf-8 export No koha-qa errors -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #27925|0 |1 is obsolete| | --- Comment #7 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 29138 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=29138&action=edit [SIGNED-OFF] Bug 12174 follow-up of the patch (a missing print statement) http://bugs.koha-community.org/show_bug.cgi?id=12174 Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |bgkriegel@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #29137|0 |1 is obsolete| | Attachment #29138|0 |1 is obsolete| | --- Comment #8 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 30359 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30359&action=edit [PASSED QA] Bug 12174 - Zotero connection with Koha: problem with UTF-8 This patch solves the bug for UTF-8 output. MARC-8 works only if you save it as a file. In fact the standard Internet stack (browser, web server, etc.) doesn't know anything about MARC-8 or it's ISO base (ISO 2022). To test the bug: -- without the patch use Zotero on a recent Koha install with Apache. You can use PTFS sandboxes. You can't use Biblibre sandboxes, they use Ngix. -- Test a record with special chars. -- Install the patch -- Catch (using Zotero) the same record. -- Test all directly download from Opac. -- To check MARC-8, use a tool that support it. For example MarcEdit. http://bugs.koha-community.org/show_bug.cgi?id=12174 Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Work as described Tested with zotero add-on for firefox. Before patch bad diacritics, after patch ok Record exported as MARC-8, transformed using yaz-iconv -f marc8 -t utf-8 bib.marc8 > bib.utf8 yaz-marcdump -i marc -o line bib.utf8, checking accents, also comparing with direct utf-8 export No koha-qa errors Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Tested with Zotero in Firefox and with yaz-iconv as suggested by Bernardo, no problems found. Passes tests and QA script. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 30360 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30360&action=edit [PASSED QA] Bug 12174 follow-up of the patch (a missing print statement) http://bugs.koha-community.org/show_bug.cgi?id=12174 Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |tomascohen@gmail.com --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patches pushed to master. Thanks Zeno! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12174 Marjorie Barry-Vila <marjorie.barry-vila@ccsr.qc.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marjorie.barry-vila@ccsr.qc | |.ca -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org