[Bug 39296] New: Provide a template plugin to return MARC::Record for MARCXML metadata
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Bug ID: 39296 Summary: Provide a template plugin to return MARC::Record for MARCXML metadata Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Reports Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au QA Contact: testopia@bugs.koha-community.org ExtractValue() is a useful MySQL function but it's limited in what it can do. Sometimes, you want to do reporting off a more nuanced understanding of MARC. I had thought about exposing XML::LibXML::Document and XML::LibXML::XPathContext objects, but they're too powerful/vulnerable to abuse. Instead, I decided that MARC::Record should give all the power we need while not upsetting the status quo or security (after all, we already allow access to MARC::Record objects via $biblio objects in Templates). In this case, you fetch the whole "metadata" blob from the database in your report, and you transform it using a template using Koha::Template::Plugin::KohaMarcRecord It lets you perform more advanced reporting using the full MARC record. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |dcook@prosentient.com.au |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Created attachment 179135 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=179135&action=edit Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML This change takes MARCXML data (e.g. from the biblio_metadata table) and returns a MARC::Record object which can be used in Reports templates to allow for more sophisticated reporting using MARCXML data. Test plan: 0. Apply the patch and koha-plack --restart kohadev 1. Go to http://localhost:8081/cgi-bin/koha/tools/letter.pl 2. Add a new notice for Reports with the following content: [% USE KohaMarcRecord %] [% SET all_subjects = {} %] [% FOREACH row IN data %] [% SET record= KohaMarcRecord.from_xml(row.metadata) %] [% SET fields = record.field('6..') %] [% FOREACH field IN fields %] [% tmp = field.delete_subfield('code','9') %] [% SET subject = field.as_string() %] [% IF ( all_subjects.$subject ) %] [% all_subjects.$subject = all_subjects.$subject + 1 %] [% ELSE %] [% all_subjects.$subject = 1 %] [% END %] [% END %] [% END %] <table> <tr><th>Subject</th><th>Count</th></tr> [% FOREACH key IN all_subjects.nsort.reverse %] <tr><td>[% key %]</td><td>[% all_subjects.$key %]</td></tr> [% END %] </table> 3. Go to http://localhost:8081/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql 4. Create a new report with the following SQL: 5. Click "Run with template" and choose the template/notice you've created above 6. Note that you get a list of MARC subjects sorted by the most used to the least used -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |phil@chetcolibrary.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #2 from Phil Ringnalda <phil@chetcolibrary.org> --- But if we do that, all my knowledge about how to ExtractValue repeatable fields like 650 as individuals with datafield[@tag=650][1] ... datafield[@tag=650][20] will be useless! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #179135|0 |1 is obsolete| | --- Comment #3 from Phil Ringnalda <phil@chetcolibrary.org> --- Created attachment 179189 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=179189&action=edit Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML This change takes MARCXML data (e.g. from the biblio_metadata table) and returns a MARC::Record object which can be used in Reports templates to allow for more sophisticated reporting using MARCXML data. Test plan: 0. Apply the patch and koha-plack --restart kohadev 1. Go to http://localhost:8081/cgi-bin/koha/tools/letter.pl 2. Add a new notice for Reports with the following content: [% USE KohaMarcRecord %] [% SET all_subjects = {} %] [% FOREACH row IN data %] [% SET record= KohaMarcRecord.from_xml(row.metadata) %] [% SET fields = record.field('6..') %] [% FOREACH field IN fields %] [% tmp = field.delete_subfield('code','9') %] [% SET subject = field.as_string() %] [% IF ( all_subjects.$subject ) %] [% all_subjects.$subject = all_subjects.$subject + 1 %] [% ELSE %] [% all_subjects.$subject = 1 %] [% END %] [% END %] [% END %] <table> <tr><th>Subject</th><th>Count</th></tr> [% FOREACH key IN all_subjects.nsort.reverse %] <tr><td>[% key %]</td><td>[% all_subjects.$key %]</td></tr> [% END %] </table> 3. Go to http://localhost:8081/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql 4. Create a new report with the following SQL: select metadata from biblio_metadata 5. Click "Run with template" and choose the template/notice you've created above 6. Note that you get a list of MARC subjects sorted by the most used to the least used Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #4 from Phil Ringnalda <phil@chetcolibrary.org> --- I made a guess at the missing SQL and added it to the test plan. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- (In reply to Phil Ringnalda from comment #4)
I made a guess at the missing SQL and added it to the test plan.
Thanks, Phil. Not sure what happened there! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #6 from David Cook <dcook@prosentient.com.au> --- (In reply to Phil Ringnalda from comment #2)
But if we do that, all my knowledge about how to ExtractValue repeatable fields like 650 as individuals with datafield[@tag=650][1] ... datafield[@tag=650][20] will be useless!
You delight me, Phil. Haha. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Heather <flyingendpaper@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |flyingendpaper@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39184 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com, | |nick@bywatersolutions.com --- Comment #7 from Jonathan Druart <jonathan.druart@gmail.com> --- Thinking aloud. We have some magic in the reports already to send items to batch mod tools, if the headers contain "itemnumber". Could we imagine the same here? If itemnumber or biblionumber in the header then we send the Koha::Objects to generate the notice. The notice template would be: [% FOREACH biblio IN biblios %] [% SET record = biblio.metadata.record %] We won't need an additional TT plugin and that will be useful in different other use cases. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #8 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 188347 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=188347&action=edit Bug 39296: [POC] Send objects to the templates [% SET all_subjects = {} %] [% FOREACH d IN data %] [% SET record= d.biblio.metadata.record %] [% SET fields = record.field('6..') %] [% FOREACH field IN fields %] [% tmp = field.delete_subfield('code','9') %] [% SET subject = field.as_string() %] [% IF ( all_subjects.$subject ) %] [% all_subjects.$subject = all_subjects.$subject + 1 %] [% ELSE %] [% all_subjects.$subject = 1 %] [% END %] [% END %] [% END %] <table> <tr><th>Subject</th><th>Count</th></tr> [% FOREACH key IN all_subjects.nsort.reverse %] <tr><td>[% key %]</td><td>[% all_subjects.$key %]</td></tr> [% END %] </table> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #9 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Jonathan Druart from comment #8)
Created attachment 188347 [details] [review] Bug 39296: [POC] Send objects to the templates
Far from perfect but this is what I had in mind, for discussion. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #7)
Thinking aloud.
We have some magic in the reports already to send items to batch mod tools, if the headers contain "itemnumber".
Could we imagine the same here? If itemnumber or biblionumber in the header then we send the Koha::Objects to generate the notice.
The notice template would be: [% FOREACH biblio IN biblios %] [% SET record = biblio.metadata.record %]
We won't need an additional TT plugin and that will be useful in different other use cases.
I think I thought a bit about this originally, but I was worried a bit about how methods could be chained together to leak data. I'll put my idea in a separate comment. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #7)
We won't need an additional TT plugin and that will be useful in different other use cases.
Yeah, I thought it would be interesting to have a solution that was powerful/flexible across a range of use cases, but then I thought about all the ways it could go wrong, so I decided to narrowly scope the template plugin. I don't think I could see any security flaws of just introducing MARC::Record via the template plugin. Limited utility overall, but I do use it in production for a few of my libraries, especially for analyzing usage of authorities in bib records. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39296 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org