[Bug 18273] New: Bulkmarcimport inserts authority duplicates
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Bug ID: 18273 Summary: Bulkmarcimport inserts authority duplicates Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: gmcharlt@gmail.com Reporter: matthias.meusburger@biblibre.com QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz When importing authorities through bulkmarcimport with the -match option, there are 3 possibilities: 1) An existing authority has been found in koha according to the -match criteria. The authority is replaced if needed (005 are checked first). This is ok. 2) No authority is found in koha according to the -match criteria. The authority is inserted. This is also ok. 3) Multiple authorities have been found in koha according to the -match criteria. The authority is inserted. => Is this the expected behavior? As the -match criteria is used for deduping, the existing authorities are very likely to be already existing duplicates. Do we really want to add another one? My feeling is that in this case, the authority should NOT be inserted. Can I have your opinion on this? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 --- Comment #1 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- Created attachment 61106 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61106&action=edit Bug 18273: Do not insert authority if there already are duplicates in koha -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #2 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- The bibliographic import overlays the one with the lower (or higher?) biblionumber. Not sure if we should make it consistent. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 --- Comment #3 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- Are you sure about that Katrin?
From what I've seen, the behaviour is the same for biblios: if multiple matching biblios have been found, a new biblio is inserted.
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hm, I was pretty sure, but was thinking of the staged marc tool. Didn't realize then it will probably be different for bulkmarcimport. Makes sense then. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Marjorie Barry-Vila <marjorie.barry-vila@ccsr.qc.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marjorie.barry-vila@ccsr.qc | |.ca -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Koha Team Lyon 3 <koha@univ-lyon3.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@univ-lyon3.fr -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Matthias Meusburger <matthias.meusburger@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |matthias.meusburger@biblibr | |e.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion CC| |fridolin.somers@biblibre.co | |m --- Comment #5 from Fridolin Somers <fridolin.somers@biblibre.com> --- Need signoff ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Patch doesn't apply --- Comment #6 from Fridolin Somers <fridolin.somers@biblibre.com> --- Needs an easy rebase -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 --- Comment #7 from Koha Team University Lyon 3 <koha@univ-lyon3.fr> --- Hi, it would be nice to have a follow-up on this one. Thx Sonia -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #61106|0 |1 is obsolete| | --- Comment #8 from David Nind <david@davidnind.com> --- Created attachment 144213 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=144213&action=edit Bug 18273: Do not insert authority if there already are duplicates in koha -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #9 from David Nind <david@davidnind.com> --- I think I managed to successfully rebase this one, but I'm not sure how to test. If we could get a test plan, I'll have a go at testing. It would be great if the test plan could include: - the bulkmarcimport command to use, and - authority record samples to use to test the scenarios outlined in the bug description (or how to create). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Patch still applies, but is missing a test plan. I think some hints, especially on the CLI parameters to use with bulkmarcimport could help to get this further. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Matthias Meusburger <matthias.meusburger@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #144213|0 |1 is obsolete| | --- Comment #11 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- Created attachment 171385 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=171385&action=edit Bug 18273: Do not insert authority if there already are duplicates in koha Test plan: 1) Create an authority with a unique value in a searchable field, for instance a PERSO_NAME authority with 'BZ18273' in the 100$a field. 2) Create a second authority with the same value in the same field. 3) Export this second authority as iso2709 (Save -> MARC (Unicode/UTF-8)) 4) Run bulkmarcimport.pl with the following parameters: perl bulkmarcimport.pl -a --file=/kohadevbox/koha/auth-xxxx.utf8 --match='Heading,100$a' -l /kohadevbox/koha/bulk.log 5) Check in the log file and/or through search that a third authority has been added (despite using the --match option) 6) Apply the patch 7) Delete the 3 authorities and repeat step 1 to 4. 8) Check that a third authority has not been added (--match worked) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Matthias Meusburger <matthias.meusburger@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #12 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- Hi, I've added a test plan. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #171385|0 |1 is obsolete| | --- Comment #13 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 172049 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172049&action=edit Bug 18273: Do not insert authority if there already are duplicates in koha Test plan: 1) Create an authority with a unique value in a searchable field, for instance a PERSO_NAME authority with 'BZ18273' in the 100$a field. 2) Create a second authority with the same value in the same field. 3) Export this second authority as iso2709 (Save -> MARC (Unicode/UTF-8)) 4) Run bulkmarcimport.pl with the following parameters: perl bulkmarcimport.pl -a --file=/kohadevbox/koha/auth-xxxx.utf8 --match='Heading,100$a' -l /kohadevbox/koha/bulk.log 5) Check in the log file and/or through search that a third authority has been added (despite using the --match option) 6) Apply the patch 7) Delete the 3 authorities and repeat step 1 to 4. 8) Check that a third authority has not been added (--match worked) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off Patch complexity|--- |Trivial patch CC| |martin.renvoize@ptfs-europe | |.com QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #14 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Thanks Matts, Super trivial patch working as I'd expect. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal Summary|Bulkmarcimport inserts |bulkmarcimport.pl inserts |authority duplicates |authority duplicates --- Comment #15 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Updating this to "bug". -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |24.11.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.11! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Koha collecto <koha@collecto.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@collecto.ca -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This fixes the release notes| |misc/migration_tools/bulkma | |rcimport.pl script when | |importing authority records | |so that the "--match" | |option works as expected, | |and no duplicates are | |created. Previously, this | |option was not working for | |authority records and | |duplicate records were | |being created even when | |there was a match. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|24.11.00 |24.11.00,24.05.06 released in| | CC| |lucas@bywatersolutions.com Status|Pushed to main |Pushed to stable --- Comment #17 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Backported to 24.05.x for upcoming 24.05.06 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |Pushed to oldoldstable Version(s)|24.11.00,24.05.06 |24.11.00,24.05.06,23.11.11 released in| | --- Comment #18 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 23.11.x for 23.11.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18273 Jesse Maseto <jesse@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jesse@bywatersolutions.com Resolution|--- |FIXED Status|Pushed to oldoldstable |RESOLVED --- Comment #19 from Jesse Maseto <jesse@bywatersolutions.com> --- Not pushed to LTS. Marked Resolved. If you feel this should be in LTS please reply with your reason. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org