[Bug 10811] New: Add a 999$c record matching rule to the MARC21 defaults
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Bug ID: 10811 Summary: Add a 999$c record matching rule to the MARC21 defaults Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: MARC Bibliographic record staging/import Assignee: gmcharlt@gmail.com Reporter: dcook@prosentient.com.au QA Contact: testopia@bugs.koha-community.org Currently, the only default matching rules for MARC21 are ISBN and ISSN. However, I'm sure that many people would profit from having a 999$c "Koha biblionumber" record matching rule as well. 1) If you export your bib records to do some global/batch changes with another program (like MarcEdit), you can use this rule to replace the existing bib records with your modified records. 2) With my upcoming OAI-PMH harvesting feature, if you add a OAI-PMH repository with this rule in the configuration, you'll be able to automagically match updated records with the original records that you imported via OAI-PMH (the magic is that I check the incoming unique identifier against records in Koha, and if there is already a bib record in Koha matching that identifier, I write the biblionumber for that record back into the incoming record so that they match. It sounds convoluted but it works well). 3) People might have other ideas about this as well. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Created attachment 20752 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=20752&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply CC| |gitbot@bugs.koha-community. | |org --- Comment #2 from I'm just a bot <gitbot@bugs.koha-community.org> --- Applying: Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults Using index info to reconstruct a base tree... M installer/data/mysql/updatedatabase.pl Falling back to patching base and 3-way merge... Auto-merging installer/data/mysql/updatedatabase.pl CONFLICT (content): Merge conflict in installer/data/mysql/updatedatabase.pl Patch failed at 0001 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults The copy of the patch that failed is found in: /home/christopher/git/koha/.git/rebase-apply/patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #20752|0 |1 is obsolete| | --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Created attachment 21216 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21216&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply When did the bot| |2013-09-29 last check this| | --- Comment #4 from I'm just a bot <gitbot@bugs.koha-community.org> --- Applying: Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults Using index info to reconstruct a base tree... M installer/data/mysql/updatedatabase.pl Falling back to patching base and 3-way merge... Auto-merging installer/data/mysql/updatedatabase.pl CONFLICT (content): Merge conflict in installer/data/mysql/updatedatabase.pl Patch failed at 0001 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults The copy of the patch that failed is found in: /home/christopher/git/koha/.git/rebase-apply/patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21216|0 |1 is obsolete| | --- Comment #5 from Chris Cormack <chris@bigballofwax.co.nz> --- Created attachment 21587 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21587&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- When did the bot|2013-09-29 00:00:00 |2013-10-27 last check this| | --- Comment #6 from I'm just a bot <gitbot@bugs.koha-community.org> --- Patch applied cleanly, go forth and signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Nicole C. Engard <nengard@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nengard@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21587|0 |1 is obsolete| | --- Comment #7 from David Cook <dcook@prosentient.com.au> --- Created attachment 23772 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=23772&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 --- Comment #8 from M. Tompsett <mtompset@hotmail.com> --- Comment on attachment 23772 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=23772 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults Review of attachment 23772: --> (http://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=10811&attachment=23772) ----------------------------------------------------------------- ::: installer/data/mysql/updatedatabase.pl @@ +7857,5 @@
+if ( CheckVersion($DBversion) ) { + if ( C4::Context->preference("marcflavour") eq 'MARC21' ) { + $dbh->do(" + INSERT INTO marc_matchers (code, description, record_type, threshold) + VALUES ('KohaBiblio', '999$c', 'biblio', 1000);
The $c is taken as a variable, causing this to fail the QA tool. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |mtompset@hotmail.com --- Comment #9 from M. Tompsett <mtompset@hotmail.com> --- REVISED TEST PLAN ----------------- 1) Apply the patch 2) Run the QA test tool 3) Drop the koha database 4) Create the koha database 5) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the installer/data/mysql/en/marcflavour/marc21/optional/marc21_default_matching_rules.sql gets triggered. 6) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted. 7) restore an older database 8) Go to the staff client page to trigger an update. 9) Check the mysql tables to make sure the values were inserted. I think there may be some way of checking one could do from within Koha to test the validity of the addition, but I don't know it off the top of my head. Marked as Failed QA, because I ran up to the QA test tool step. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 --- Comment #10 from M. Tompsett <mtompset@hotmail.com> --- You can't do multiple statements in the $dbh->do(); http://search.cpan.org/~timb/DBI-1.630/DBI.pm#do -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21587|1 |0 is obsolete| | --- Comment #11 from M. Tompsett <mtompset@hotmail.com> --- Comment on attachment 21587 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21587 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This is closer to correct. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #23772|0 |1 is obsolete| | --- Comment #12 from M. Tompsett <mtompset@hotmail.com> --- Comment on attachment 23772 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=23772 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This breaks the do() in addition to the $c issue. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff Patch complexity|Trivial patch |Small patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21587|0 |1 is obsolete| | --- Comment #13 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 24421 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=24421&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I wasn't able to do steps 5-7 properly, because of the lack of 999$c MARC records in my data as far as I know. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 --- Comment #14 from David Cook <dcook@prosentient.com.au> --- (In reply to M. Tompsett from comment #13)
Created attachment 24421 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults
This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index.
This is part of the optional marc21 SQL, as well as in the updatabase.pl script.
TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :)
I wasn't able to do steps 5-7 properly, because of the lack of 999$c MARC records in my data as far as I know.
Thanks for the update. I've been putting this one off as a trivial enhancement for a little while. You should be able to do steps 5-7. Your Koha biblionumber is stored as your 999$c. If you don't have one in your Koha records, something is seriously wrong. That said, it won't show up for you in Koha, as it's a hidden field everywhere. You'd have to verify its existence by checking the marcxml in the biblioitems table. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 --- Comment #15 from M. Tompsett <mtompset@hotmail.com> --- There are multiple ways to export. The first way is probably the best. Home -> Tools -> Export data AND Home -> Search the catalogue -> find a specific biblio -> Save -> MARC (Unicode / UTF-8) (Only the UTF-8 and MARCXML ones will save 999$c data) I also ran into a nasty save problem for a title: Identification of an ‘Unidentified’ Holle List (The): The Last Word (list) I suspect the ticks or brackets caused the issue, but that is a different bug. :) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #24421|0 |1 is obsolete| | --- Comment #16 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 24441 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=24441&action=edit [SIGNED OFF] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I corrected any problems I found with the patch, particularly the $c issue. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #24441|0 |1 is obsolete| | --- Comment #17 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 24528 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=24528&action=edit Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I corrected any problems I found with the patch, particularly the $c issue. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA CC| |kyle@bywatersolutions.com --- Comment #18 from Kyle M Hall <kyle@bywatersolutions.com> --- I would question whether this match point should be added for upgrades, as the installer may have opted not to install the default match points. I believe that is a question for the release manager though. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gmcharlt@gmail.com --- Comment #19 from Galen Charlton <gmcharlt@gmail.com> --- (In reply to Kyle M Hall from comment #18)
I would question whether this match point should be added for upgrades, as the installer may have opted not to install the default match points. I believe that is a question for the release manager though.
Either the database update should be skipped, or the MARC matching rules SQL should be mandatory during installation. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #20 from Galen Charlton <gmcharlt@gmail.com> --- Pushed to master. Note that I dropped the database update and added a follow-up to ensure that matching would work regardless of the QueryParser mode. By the way, I wouldn't be opposed to making the default matching rules mandatory during installation. Thanks David! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10811 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mathsabypro@gmail.com -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org