https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|3.22 |unspecified -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com --- Comment #2 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #1)

What do you suggest? Do you think bug 12704 should be reverted because it's not valid (matching rules should be used instead)?

Jonathan, is that the right bug? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |kyle@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #4 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 51439 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=51439&action=edit Bug 16493 - acq matching on title and author When you order from a staged file you're getting duplicate warnings that are inaccurate. For example, when you order a file of 50 DVDs for example and they don't have ISBNs they're matching on the books. And then you have to order them one by one. This patch replaces the use of FindDuplicates with Koha's match point system. This means you can select from the same match points defined and used in the batch record importer, or you can opt to skip matching altogether! Test Plan: 1) Import a record with a title, isbn and author. 2) Delete the from the record and stage it again 3) Attempt to add it to a basket via the staged record 4) You should note the gives you the "No records imported" message 5) Apply this patch 6) Create a matcher for ISBN 7) Create a matcher for Author/Title 8) Attempt to add the record to your basket using the ISBN matcher 8) Koha should find no match and import the record to the basket 9) Stage the record again 10) Attempt to add the record to your basket using the Title/Author matcher 11) You should recieve the "No records imported" message. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Nicole C. Engard <nengard@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Nicole C. Engard <nengard@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|unspecified |16.05 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #7 from Nicole C. Engard <nengard@gmail.com> --- (In reply to Koha Team Lyon 3 from comment #6)

Hi, I'm not sure to understand so I need more explanations :) Now, since BZ 12074 was pushed to master, when someone adds a full batch from a staged file, there's an alert for duplicates. But this alert is based on what is found by FindDuplicate. Could we change Findduplicate as it can find duplicate on another field than ISBN rather to avoid to use findDuplicate ?

I have no preference, all what I want is that we can find duplicates for books, DVDs or whatever materials when we're importing a batch records.

But, is it better to do it without FindDuplicates ? Sonia

Sonia, The problem was that FindDuplicates was hardcoded and the library had no way to change that. So the following scenario is just one example of where FindDuplicates fails libraries: 1. Stage DVD Marc Records 2. Use a rule to match on 024 (the identifier on many DVDs) 3. Find duplicates 4. Go to Acq and add staged file to basket 5. Koha says there are duplicates but there aren't instead it's matching on Title & Author and matching DVDs to books because the DVDs don't have ISBNs By using this patch we allow for libraries to choose how to find duplicates instead of forcing them to use a hard coded script that they have no control over. Nicole -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Koha Team Lyon 3 from comment #8)

Ok, Thanks Nicole, I understand and I share the "setting principle" that allow librarians to choose the key to deduplicate. I will test the patch with unimarc file to see if it's ok for us.

Do you think you will have time to test it before the release? It's a critical one and we need it fixed soon. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #10 from Koha Team Lyon 3 <koha@univ-lyon3.fr> --- Ok, I can test it on monday. Is it too late ? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #12 from Koha Team Lyon 3 <koha@univ-lyon3.fr> --- Hi, I have just finished my tests. The patch works as described. I'm a little worried because I'm afraid that some of my colleagues forget to choose a matching rule when they add orders from a batch file. It will be a regression for them as they have nothing to choose by now to have a deduplicationw to have a deduplication. So, I'm wondering if it would be possible to have a default matching rule selected when we add a orders from a batch file. Or, perhaps it would be better to keep a deduplication by FindDuplicate when no matching rules is chosen and to switch to a matching rule if the user choose one. What do you think about it ? PS : It's not related to this patch but I have some difficulties to set the matching rules. In the documentation, there's : Threshold of 1000 Match Point on 020$a 1000 Match Point on 022$a 1000 In the example above, a match on either the 020$a or the 022$a will result in a successful match. When I test this patch, I have to put a number in the match point greater than the threshold, if I put a number equal to the thershold, there's no deduplication alert... And I try to put 2 matchpoints with a result greater to the threshold and if none of the two is greater than the threshold, there's no deduplication alert. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #51441|0 |1 is obsolete| | --- Comment #14 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 52181 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=52181&action=edit Bug 16493 - acq matching on title and author When you order from a staged file you're getting duplicate warnings that are inaccurate. For example, when you order a file of 50 DVDs for example and they don't have ISBNs they're matching on the books. And then you have to order them one by one. This patch replaces the use of FindDuplicates with Koha's match point system. This means you can select from the same match points defined and used in the batch record importer, or you can opt to skip matching altogether! Test Plan: 1) Import a record with a title, isbn and author. 2) Delete the from the record and stage it again 3) Attempt to add it to a basket via the staged record 4) You should note the gives you the "No records imported" message 5) Apply this patch 6) Create a matcher for ISBN 7) Create a matcher for Author/Title 8) Attempt to add the record to your basket using the ISBN matcher 8) Koha should find no match and import the record to the basket 9) Stage the record again 10) Attempt to add the record to your basket using the Title/Author matcher 11) You should recieve the "No records imported" message. Signed-off-by: Barbara Fondren <bfondren@roundrocktexas.gov> Signed-off-by: Nicole C Engard <nengard@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #16 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Sonia, could you test it again with the follow-up please? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #19 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 52904 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=52904&action=edit Bug 16493 [QA Followup] - Restore title and author match as an option, make it the default -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Koha Team Lyon 3 <koha@univ-lyon3.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #22 from Koha Team Lyon 3 <koha@univ-lyon3.fr> --- Hi, I've tested it on a bywaters sandbox (they work !) But I don't manage to have correct deduplication. The default choice for deduplication "author and title" don't actually deduplicate by author and title. - I have imported a file in a basket - I have changed the original file to remove ISBN - I have imported this second file with the "title and author" matching checked : all the titles have been added to the basket, though title and author are exactly the same. - I have tried to re-import the first fil with the ISBN with "title and author" matching : no record have been imported : the deduplication works. I don't know what is exactly behind the matching key "title and author" ? Perhaps should we call it "Title/author/ISBN" ? Sonia -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #52903|0 |1 is obsolete| | Attachment #52904|0 |1 is obsolete| | --- Comment #23 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 57139 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=57139&action=edit Bug 16493 - acq matching on title and author When you order from a staged file you're getting duplicate warnings that are inaccurate. For example, when you order a file of 50 DVDs for example and they don't have ISBNs they're matching on the books. And then you have to order them one by one. This patch replaces the use of FindDuplicates with Koha's match point system. This means you can select from the same match points defined and used in the batch record importer, or you can opt to skip matching altogether! Test Plan: 1) Import a record with a title, isbn and author. 2) Delete the from the record and stage it again 3) Attempt to add it to a basket via the staged record 4) You should note the gives you the "No records imported" message 5) Apply this patch 6) Create a matcher for ISBN 7) Create a matcher for Author/Title 8) Attempt to add the record to your basket using the ISBN matcher 8) Koha should find no match and import the record to the basket 9) Stage the record again 10) Attempt to add the record to your basket using the Title/Author matcher 11) You should recieve the "No records imported" message. Signed-off-by: Barbara Fondren <bfondren@roundrocktexas.gov> Signed-off-by: Nicole C Engard <nengard@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #25 from Kyle M Hall <kyle@bywatersolutions.com> --- Good Catch Sonia, there was a bug in my followup that was causing it to not actually use the Title/Author matching. Please give it another try if you like! (In reply to Koha Team Lyon 3 from comment #22)

Hi, I've tested it on a bywaters sandbox (they work !) But I don't manage to have correct deduplication. The default choice for deduplication "author and title" don't actually deduplicate by author and title. - I have imported a file in a basket - I have changed the original file to remove ISBN - I have imported this second file with the "title and author" matching checked : all the titles have been added to the basket, though title and author are exactly the same. - I have tried to re-import the first fil with the ISBN with "title and author" matching : no record have been imported : the deduplication works.

I don't know what is exactly behind the matching key "title and author" ? Perhaps should we call it "Title/author/ISBN" ?

Sonia

-- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #28 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to master for 16.11! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com --- Comment #30 from Mason James <mtj@kohaaloha.com> --- Pushed to 16.05.x, will be in 16.05.06 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16493 --- Comment #31 from Nick Clemens <nick@bywatersolutions.com> --- *** Bug 16495 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.