[Bug 32336] New: MARCXML output of REST API may be badly encoded (UNIMARC)
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Bug ID: 32336 Summary: MARCXML output of REST API may be badly encoded (UNIMARC) Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: normal Priority: P5 - low Component: REST API Assignee: julian.maurice@biblibre.com Reporter: julian.maurice@biblibre.com CC: tomascohen@gmail.com In Koha/REST/V1/Biblios.pm (methods get and get_public), MARC::Record::as_xml_record is called without a $format parameter, which means it relies on the package variable $MARC::File::XML::RecordFormat to be correctly set, which may or may not be the case depending on what the current starman/mojo worker has done before. We should pass the format explicitly to avoid problems. This is only a problem for UNIMARC, because $MARC::File::XML::RecordFormat defaults to 'USMARC'. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #1 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 144207 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144207&action=edit Bug 32336: Fix encoding of MARCXML output for REST API (UNIMARC) Test plan: 1. Be sure to test on a UNIMARC instance 2. Enable system preferences RESTPublicAPI and RESTPublicAnonymousRequests 3. Create a biblio with some diacritics 4. Restart koha and do not interact with OPAC or the staff interface (this may change $MARC::File::XML::RecordFormat) 5. Query the public API to retrieve the created biblio in MARCXML. For instance with cURL: curl -H 'Accept: application/marcxml+xml' \ http://koha.local/api/v1/public/biblios/<biblionumber> You should see encoding issues. 6. Apply the patch and restart Koha. 7. Repeat step 5. You should see no encoding issues. 8. You should also test /api/v1/biblios/<biblionumber> (this one requires authentication, so cURL might not be the most practical tool; use your favorite tool for this) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Please add tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #3 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 146846 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=146846&action=edit Bug 32336: Add unit test for UNIMARC encoding problems in REST API -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #144207|0 |1 is obsolete| | --- Comment #4 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 146847 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=146847&action=edit Bug 32336: Fix encoding of MARCXML output for REST API (UNIMARC) Test plan: 1. Be sure to test on a UNIMARC instance 2. Enable system preferences RESTPublicAPI and RESTPublicAnonymousRequests 3. Create a biblio with some diacritics 4. Restart koha and do not interact with OPAC or the staff interface (this may change $MARC::File::XML::RecordFormat) 5. Query the public API to retrieve the created biblio in MARCXML. For instance with cURL: curl -H 'Accept: application/marcxml+xml' \ http://koha.local/api/v1/public/biblios/<biblionumber> You should see encoding issues. 6. Apply the patch and restart Koha. 7. Repeat step 5. You should see no encoding issues. 8. You should also test /api/v1/biblios/<biblionumber> (this one requires authentication, so cURL might not be the most practical tool; use your favorite tool for this) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Julian Maurice from comment #4)
8. You should also test /api/v1/biblios/<biblionumber> (this one requires authentication, so cURL might not be the most practical tool; use your favorite tool for this)
FYI you can use -u koha:koha ;) curl -v -s -u koha:koha --request GET 'http://localhost:8081/api/v1/biblios/1' --header 'Accept: application/marcxml+xml' -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #146846|0 |1 is obsolete| | --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 148614 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148614&action=edit Bug 32336: Add unit test for UNIMARC encoding problems in REST API Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #146847|0 |1 is obsolete| | --- Comment #7 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 148615 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148615&action=edit Bug 32336: Fix encoding of MARCXML output for REST API (UNIMARC) Test plan: 1. Be sure to test on a UNIMARC instance 2. Enable system preferences RESTPublicAPI and RESTPublicAnonymousRequests 3. Create a biblio with some diacritics 4. Restart koha and do not interact with OPAC or the staff interface (this may change $MARC::File::XML::RecordFormat) 5. Query the public API to retrieve the created biblio in MARCXML. For instance with cURL: curl -H 'Accept: application/marcxml+xml' \ http://koha.local/api/v1/public/biblios/<biblionumber> You should see encoding issues. 6. Apply the patch and restart Koha. 7. Repeat step 5. You should see no encoding issues. 8. You should also test /api/v1/biblios/<biblionumber> (this one requires authentication, so cURL might not be the most practical tool; use your favorite tool for this) Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |major Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- I think we need some more here: we should really be using my $metadata = $biblio->metadata; my $schema = $biblio->schema // C4::Context->preference("marcflavour"); Should we also file a new bug for list()? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 148628 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148628&action=edit Bug 32336: (QA follow-up) Use $metadata->schema When we worked on Koha::Biblio::Metadata, we agreed biblio_metadata.schema would be the sourceof truth when it comes to the record schema. This patch acknowledges that, while retaining the fallback to the syspref. To test: 1. Apply this patch 2. Run: $ prove t/db_dependent/api/v1/biblios.t => SUCCESS: Tests pass, behavior is unchanged. 3. Sign off :-D Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- If you agree with my follow-up please set it back to PQA, otherwise let's discuss it properly. Best regards. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |33329 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33329 [Bug 33329] GET /biblios encoding wrong when UNIMARC -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA --- Comment #11 from Julian Maurice <julian.maurice@biblibre.com> --- Patch makes sense. Back to Passed QA Thanks for the follow-up Tomas -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #12 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.05. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.05.00 |23.05.00,22.11.05 released in| | Status|Pushed to master |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 --- Comment #13 from Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> --- Nice work everyone! Pushed to stable for 22.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=34514 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |34515 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34515 [Bug 34515] MARCXML output of OAI-PMH may be badly encoded (UNIMARC) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32336 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=34467 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org