[Bug 23793] New: Add an EmbedItems RecordProcessor filter for MARC::Record objects
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Bug ID: 23793 Summary: Add an EmbedItems RecordProcessor filter for MARC::Record objects Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org This should help reduce the current overhead, and by using the features introduced by bug 22454, probably give a performance boost. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |glasklas@gmail.com, | |kyle@bywatersolutions.com, | |martin.renvoize@ptfs-europe | |.com, | |nick@bywatersolutions.com Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=20664 CC| |ere.maijala@helsinki.fi -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=19884 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #1 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 93971 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93971&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch is work in progress. I submit early with the aim to provide people thinking about performance on the biblio/records/items front an idea of what I'm thinking. The synopsis on the new filter is self explanatory. The idea is that we calculate as much as we can outside the loops. And this simpe implementation is clean and light as well. Problems: - Filters are supposed to support passing multiple MARC records, this one doesn't, beacuse I didn't think enough of a way to pass all the items and biblionumbers in a way that is safe... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- I paste the synopsis for easier analysis: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { $_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items, item_tag => $item_tag, biblionumber => $biblionumber } } ); $processor->process( $record ); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93971|0 |1 is obsolete| | --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 93980 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93980&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch is work in progress. I submit early with the aim to provide people thinking about performance on the biblio/records/items front an idea of what I'm thinking. The synopsis on the new filter is self explanatory. The idea is that we calculate as much as we can outside the loops. And this simpe implementation is clean and light as well. Problems: - Filters are supposed to support passing multiple MARC records, this one doesn't, beacuse I didn't think enough of a way to pass all the items and biblionumbers in a way that is safe... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |23807 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23807 [Bug 23807] Add Koha::Item->as_marc_field -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93980|0 |1 is obsolete| | --- Comment #4 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 94030 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94030&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch is work in progress. I submit early with the aim to provide people thinking about performance on the biblio/records/items front an idea of what I'm thinking. The synopsis on the new filter is self explanatory. The idea is that we calculate as much as we can outside the loops. And this simpe implementation is clean and light as well. Problems: - Filters are supposed to support passing multiple MARC records, this one doesn't, beacuse I didn't think enough of a way to pass all the items and biblionumbers in a way that is safe... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #5 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Simplified version, based on 23807. Update synopsis here: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items } } ); $processor->process( $record ); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|Small patch |Trivial patch Status|ASSIGNED |Needs Signoff CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94030|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #6 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 94095 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94095&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch introduces a RecordProcessor filter for MARC::Record objects that embeds MARC::Field's for each item that is passed to the processor. Target usage: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items->as_list}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items } } ); $processor->process( $record ); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #7 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 94096 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94096&action=edit Bug 23793: Unit tests This patch introduces full test coverage for the EmbedItems filter. To test: 1. Apply this patchset 2. Run: $ kshell k$ prove t/db_dependent/Koha/Filter/EmbedItems.t => SUCCESS: Tests pass! 3. Sign off :-D 4. Use it to improve Koha! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #8 from Ere Maijala <ere.maijala@helsinki.fi> --- This looks great, but wouldn't it be nice to put it into use right away? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Ere Maijala from comment #8)
This looks great, but wouldn't it be nice to put it into use right away?
Yes, but I wanted your opinion first! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #10 from Ere Maijala <ere.maijala@helsinki.fi> --- +1 for this, I'm going to close my old optimization bug. :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Ere Maijala from comment #10)
+1 for this, I'm going to close my old optimization bug. :)
Can we talk you into a sign-off? :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_19_11_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94095|0 |1 is obsolete| | --- Comment #12 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 94599 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94599&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch introduces a RecordProcessor filter for MARC::Record objects that embeds MARC::Field's for each item that is passed to the processor. Target usage: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items->as_list}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items } } ); $processor->process( $record ); Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94096|0 |1 is obsolete| | --- Comment #13 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 94600 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94600&action=edit Bug 23793: Unit tests This patch introduces full test coverage for the EmbedItems filter. To test: 1. Apply this patchset 2. Run: $ kshell k$ prove t/db_dependent/Koha/Filter/EmbedItems.t => SUCCESS: Tests pass! 3. Sign off :-D 4. Use it to improve Koha! Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #14 from Ere Maijala <ere.maijala@helsinki.fi> --- Ok, fine. :) I trust that there will be something using this soon. I'm generally not in favour of adding functionality without anything using it since there's always a risk that something happens and it ends up never being used. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94599|0 |1 is obsolete| | --- Comment #15 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 94602 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94602&action=edit Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch introduces a RecordProcessor filter for MARC::Record objects that embeds MARC::Field's for each item that is passed to the processor. Target usage: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items->as_list}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items } } ); $processor->process( $record ); Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94600|0 |1 is obsolete| | --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 94603 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94603&action=edit Bug 23793: Unit tests This patch introduces full test coverage for the EmbedItems filter. To test: 1. Apply this patchset 2. Run: $ kshell k$ prove t/db_dependent/Koha/Filter/EmbedItems.t => SUCCESS: Tests pass! 3. Sign off :-D 4. Use it to improve Koha! Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #17 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Passes QA scripts, Passes tests.. Passing QA. I trust Tomas to followup with uses of this in new bugs soon :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |19.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #18 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work! Pushed to master for 19.11.00 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_19_11_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=24403 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #19 from David Cook <dcook@prosentient.com.au> --- (In reply to Ere Maijala from comment #14)
Ok, fine. :) I trust that there will be something using this soon. I'm generally not in favour of adding functionality without anything using it since there's always a risk that something happens and it ends up never being used.
I don't think there's any core functionality that is using it today unfortunately. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #20 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to David Cook from comment #19)
(In reply to Ere Maijala from comment #14)
Ok, fine. :) I trust that there will be something using this soon. I'm generally not in favour of adding functionality without anything using it since there's always a risk that something happens and it ends up never being used.
I don't think there's any core functionality that is using it today unfortunately.
opac/opac-ISBDdetail.pl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- (In reply to Tomás Cohen Arazi from comment #20)
opac/opac-ISBDdetail.pl
Cool. I'm going to use it in some local code now. The sample code in the documentation doesn't work but working out the right way of doing it... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |29764 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29764 [Bug 29764] EmbedItems RecordProcessor filter POD incorrect -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23793 --- Comment #22 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to David Cook from comment #21)
(In reply to Tomás Cohen Arazi from comment #20)
opac/opac-ISBDdetail.pl
Cool. I'm going to use it in some local code now. The sample code in the documentation doesn't work but working out the right way of doing it...
I fixed the POD on bug 29764. Please test. The only change is parenthesis for square brackets in the filters list. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org