[Bug 14639] New: Extend Koha::MetadataRecord to handle serialization format
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Bug ID: 14639 Summary: Extend Koha::MetadataRecord to handle serialization format Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: gmcharlt@gmail.com Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org Koha::MetadataRecord was introduced to abstract records, and carry information about the records. It is currently missing a way to know the serialization format used. This is pretty important when it comes to using Koha::RecordProcessor to apply filters on records, as different filter could be needed depending on the serialization format: for example, a source could be a USMARC record and another one an XML record, both representing MARC21 metadata. The same applies if you think of JSON records, etc. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jcamins@cpbibliography.com Assignee|gmcharlt@gmail.com |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #1 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41315 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41315&action=edit Bug 14639: (regression tests) Extend Koha::MetadataRecord to handle serialization format In order to use Koha::MetadataRecord as a container for moving records around it is important to let it carry the serialization format of the record object it was built with, so it is easier and cheaper to make decisions about records. This patch introduces regression tests for the changes to be made. The 'format' param is introduced, and also sets default values: schema => 'marc21' format => 'usmarc' so the default behaviour is preserved, and no changes are needed in places Koha::MetadataRecord is used. ->new also returns undef if no record is passed, and raises a carped warning. To test: - Apply the patch - Run the new tests $ prove t/Koha_MetadataRecord.t => FAIL: Tests shoud fail as the changes are not implemented on Koha::MetadataRecord -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41316 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41316&action=edit Bug 14639: Extend Koha::MetadataRecord to handle serialization format The description of this changes is on the regression tests commit message. To test: - Run $ prove t/Koha_MetadataRecord.t => FAIL: Tests fail because changes are not implemented - Apply this patch - Run $ prove t/Koha_MetadataRecord.t => SUCCESS: tests pass - Run $ prove t/Koha_Util_MARC.t => SUCCESS: it still passes - Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41315|0 |1 is obsolete| | --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41357 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41357&action=edit Bug 14639: (regression tests) Extend Koha::MetadataRecord to handle serialization format and record id In order to use Koha::MetadataRecord as a container for moving records around it is important to let it carry the serialization format of the record object it was built with, so it is easier and cheaper to make decisions about records. This patch introduces regression tests for the changes to be made. The 'format' param is introduced, and also sets default values: schema => 'marc21' format => 'marc' A new (optional) 'id' param is added so the record carries its own id outside of it. The default behaviour is preserved, and no changes are needed in places Koha::MetadataRecord is used. ->new also returns undef if no record is passed, and raises a carped warning. To test: - Apply the patch - Run the new tests $ prove t/Koha_MetadataRecord.t => FAIL: Tests shoud fail as the changes are not implemented on Koha::MetadataRecord -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41316|0 |1 is obsolete| | --- Comment #4 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41358 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41358&action=edit Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id The description of this changes is on the regression tests commit message. To test: - Run $ prove t/Koha_MetadataRecord.t => FAIL: Tests fail because changes are not implemented - Apply this patch - Run $ prove t/Koha_MetadataRecord.t => SUCCESS: tests pass - Run $ prove t/Koha_Util_MARC.t => SUCCESS: it still passes - Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |14645 Referenced Bugs: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14645 [Bug 14645] Koha::RecordProcessor should deal with Koha::MetadataRecord objects -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41357|0 |1 is obsolete| | --- Comment #5 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41360 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41360&action=edit Bug 14639: (regression tests) Extend Koha::MetadataRecord to handle serialization format and record id In order to use Koha::MetadataRecord as a container for moving records around it is important to let it carry the serialization format of the record object it was built with, so it is easier and cheaper to make decisions about records. This patch introduces regression tests for the changes to be made. The 'format' param is introduced, and also sets default values: schema => 'marc21' format => 'MARC' A new (optional) 'id' param is added so the record carries its own id outside of it. The default behaviour is preserved, and no changes are needed in places Koha::MetadataRecord is used. ->new also returns undef if no record is passed, and raises a carped warning. To test: - Apply the patch - Run the new tests $ prove t/Koha_MetadataRecord.t => FAIL: Tests shoud fail as the changes are not implemented on Koha::MetadataRecord Edit: made serialization format be upper-case to match what is used on Koha::Filter's -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41358|0 |1 is obsolete| | --- Comment #6 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41361 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41361&action=edit Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id The description of this changes is on the regression tests commit message. To test: - Run $ prove t/Koha_MetadataRecord.t => FAIL: Tests fail because changes are not implemented - Apply this patch - Run $ prove t/Koha_MetadataRecord.t => SUCCESS: tests pass - Run $ prove t/Koha_Util_MARC.t => SUCCESS: it still passes - Sign off :-D -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41360|0 |1 is obsolete| | --- Comment #7 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 41386 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41386&action=edit Bug 14639: (regression tests) Extend Koha::MetadataRecord to handle serialization format and record id In order to use Koha::MetadataRecord as a container for moving records around it is important to let it carry the serialization format of the record object it was built with, so it is easier and cheaper to make decisions about records. This patch introduces regression tests for the changes to be made. The 'format' param is introduced, and also sets default values: schema => 'marc21' format => 'MARC' A new (optional) 'id' param is added so the record carries its own id outside of it. The default behaviour is preserved, and no changes are needed in places Koha::MetadataRecord is used. ->new also returns undef if no record is passed, and raises a carped warning. To test: - Apply this test patch - Run the new tests $ prove t/Koha_MetadataRecord.t => FAIL: Tests shoud fail as the changes are not implemented on Koha::MetadataRecord Edit: made serialization format be upper-case to match what is used on Koha::Filter's Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41361|0 |1 is obsolete| | --- Comment #8 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 41387 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41387&action=edit Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id The description of this changes is on the regression tests commit message. To test: - Apply the test patch - Run $ prove t/Koha_MetadataRecord.t => FAIL: Tests fail because changes are not implemented - Apply this patch - Run $ prove t/Koha_MetadataRecord.t => SUCCESS: tests pass - Run $ prove t/Koha_Util_MARC.t => SUCCESS: it still passes - Sign off :-D NOTE: Tested as above. Read code. Seems to cover all cases. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtompset@hotmail.com Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Comment on attachment 41387 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41387 Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id Review of attachment 41387: --> (http://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=14639&attachment=41387) ----------------------------------------------------------------- ::: Koha/MetadataRecord.pm @@ +84,5 @@
+ + my $record = $params->{ record }; + my $schema = $params->{ schema } // 'marc21'; + my $format = $params->{ format } // 'MARC'; + my $id = $params->{ id };
Don't you think these 4 params should be mandatory? I would raise an exception if one of them is not given to the constructor. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Jonathan Druart from comment #9)
Comment on attachment 41387 [details] [review] Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id
Review of attachment 41387 [details] [review]: -----------------------------------------------------------------
::: Koha/MetadataRecord.pm @@ +84,5 @@
+ + my $record = $params->{ record }; + my $schema = $params->{ schema } // 'marc21'; + my $format = $params->{ format } // 'MARC'; + my $id = $params->{ id };
Don't you think these 4 params should be mandatory? I would raise an exception if one of them is not given to the constructor.
I agree. But my plan was to do that on a later stage, so the transition into using this is more smooth. So for now I just fallback to the current behaviour. Once this is in, I will provide the relevant regression tests for an API change. Also, we need to get the exception handling code pushed before that too. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Ok, so just one thing prevents to pass QA I think. The default value for schema should be C4::Context->preference('marcflavour'). And all calls where the pref is passed can be removed. Don't you think? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41386|0 |1 is obsolete| | --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 42312 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42312&action=edit Bug 14639: (regression tests) Extend Koha::MetadataRecord to handle serialization format and record id In order to use Koha::MetadataRecord as a container for moving records around it is important to let it carry the serialization format of the record object it was built with, so it is easier and cheaper to make decisions about records. This patch introduces regression tests for the changes to be made. The 'format' param is introduced, and also sets default values: schema => 'marc21' format => 'MARC' A new (optional) 'id' param is added so the record carries its own id outside of it. The default behaviour is preserved, and no changes are needed in places Koha::MetadataRecord is used. ->new also returns undef if no record is passed, and raises a carped warning. To test: - Apply this test patch - Run the new tests $ prove t/Koha_MetadataRecord.t => FAIL: Tests shoud fail as the changes are not implemented on Koha::MetadataRecord Edit: made serialization format be upper-case to match what is used on Koha::Filter's Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #41387|0 |1 is obsolete| | --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 42313 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42313&action=edit Bug 14639: Extend Koha::MetadataRecord to handle serialization format and record id The description of this changes is on the regression tests commit message. To test: - Apply the test patch - Run $ prove t/Koha_MetadataRecord.t => FAIL: Tests fail because changes are not implemented - Apply this patch - Run $ prove t/Koha_MetadataRecord.t => SUCCESS: tests pass - Run $ prove t/Koha_Util_MARC.t => SUCCESS: it still passes - Sign off :-D NOTE: Tested as above. Read code. Seems to cover all cases. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl Patch complexity|--- |Small patch QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #14 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Marcel, I have let a comment 11, please don't ignore it. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #15 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 42316 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42316&action=edit Bug 14639: (QA followup) make schema mandatory This patch makes the 'schema' param mandatory. It is passed in every call on the current codebase, so it makes no harm now, but makes the code less error-prone. Tests for this situation are added to t/Koha_MetadataRecord.t (schema param is omitted and new() returns undef and a carped warning). Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |tomascohen@gmail.com --- Comment #16 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patches pushed to master. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #17 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- It seems that this patch breaks some tests: prove t/db_dependent/Koha_Authority.t t/db_dependent/Koha_Authority.t .. 5/? # Failed test 'Object authid is correct' # at t/db_dependent/Koha_Authority.t line 60. # got: undef # expected: '1' # Looks like you failed 1 test of 11. t/db_dependent/Koha_Authority.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/11 subtests (less 3 skipped subtests: 7 okay) Test Summary Report ------------------- t/db_dependent/Koha_Authority.t (Wstat: 256 Tests: 11 Failed: 1) Failed test: 6 Non-zero exit status: 1 Files=1, Tests=11, 2 wallclock secs ( 0.04 usr 0.00 sys + 1.24 cusr 0.06 csys = 1.34 CPU) Result: FAIL -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #18 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Jonathan Druart from comment #17)
It seems that this patch breaks some tests:
prove t/db_dependent/Koha_Authority.t t/db_dependent/Koha_Authority.t .. 5/? # Failed test 'Object authid is correct' # at t/db_dependent/Koha_Authority.t line 60. # got: undef # expected: '1' # Looks like you failed 1 test of 11. t/db_dependent/Koha_Authority.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/11 subtests (less 3 skipped subtests: 7 okay)
Test Summary Report ------------------- t/db_dependent/Koha_Authority.t (Wstat: 256 Tests: 11 Failed: 1) Failed test: 6 Non-zero exit status: 1 Files=1, Tests=11, 2 wallclock secs ( 0.04 usr 0.00 sys + 1.24 cusr 0.06 csys = 1.34 CPU) Result: FAIL
I've tested them in Jessie and Trusty and cannot reproduce the failing tests. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #19 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Just retested with a clean DB and it passes correctly. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 --- Comment #20 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- This patch set has introduced a big issue on Koha::Authority (now Koha::Metadata::Authority). The authid method does not return the authid anymore... The tests in t/db_dependent/Koha_Authority.t are bad: they skipped if the authid does not exist. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14639 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |15578 --- Comment #21 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Jonathan Druart from comment #20)
This patch set has introduced a big issue on Koha::Authority (now Koha::Metadata::Authority). The authid method does not return the authid anymore... The tests in t/db_dependent/Koha_Authority.t are bad: they skipped if the authid does not exist.
Will be fixed by bug 15578. Referenced Bugs: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15578 [Bug 15578] Authority tests skip and hide a bug -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org