[Bug 11496] New: The number of tested records when searching for duplicates during import is too low
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Bug ID: 11496 Summary: The number of tested records when searching for duplicates during import is too low Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: gmcharlt@gmail.com Reporter: matthias.meusburger@biblibre.com QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Currently, only the first 10 records returned by koha for a given matchpoint are tested. This is not enough in some cases, because a significant matching record can be beyond these 10 first results. Please note that when the checkpoint is about fields with unique identifiers, like ISBN, the problem won't occur. On the other hand, when the checkpoint is about something less precise, like an author name, the matching record can be beyond the 10 first results. When that happens, the matchpoint will fail, the match threshold won't be reached, and a duplicate record will be created. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 --- Comment #1 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- Created attachment 24006 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=24006&action=edit proposed patch Please note that this is a naive implementation of the fix (I've arbitrarily raised the matching records from 10 to 1000). If you agree that this is an issue, a better patch with a configurable amount of records to search might be proposed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Matthias Meusburger <matthias.meusburger@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtompset@hotmail.com --- Comment #2 from M. Tompsett <mtompset@hotmail.com> --- Couldn't we have logic that knows about tighter matches (ISBN) vs. less precise matches (authors)? And for the less precise ones, couldn't even 1000 still fail? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |matthias.meusburger@biblibr | |e.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 --- Comment #3 from Matthias Meusburger <matthias.meusburger@biblibre.com> --- There is no need amha to implement such logic, because for tighter matches, even if the limit is high, only a few results will be returned. ie: having a limit of 1000 will not slow down things for isbn, because only a few isbn's will be returned. We can safely have a high limit even for tighter matches. As for your second point, 1000 could possibly fail, but very less likely than with 10. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #24006|0 |1 is obsolete| | --- Comment #4 from M. Tompsett <mtompset@hotmail.com> --- Created attachment 37626 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37626&action=edit Bug 11496: Raise number of tested records when searching for duplicate during import. - Previously, only the first 10 records returned by koha for a given matchpoint were tested. This is not enough in many cases, because a significant matching record can be beyond these 10 first results. This patch raise the search to 1000 records, for more accurate, but slower results. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 --- Comment #5 from M. Tompsett <mtompset@hotmail.com> --- Rescued this from a 'Patch Does Not Apply' situation. Also, please provide a test plan. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal CC| |jonathan.druart@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=9988 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11496 Liz Rea <liz@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |ASSIGNED CC| |liz@catalyst.net.nz --- Comment #6 from Liz Rea <liz@catalyst.net.nz> --- This needs a test plan please, before we can test it. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org