[Bug 25189] New: Make AutoCreateAuthorities less of a footgun with LinkerModule: Default
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Bug ID: 25189 Summary: Make AutoCreateAuthorities less of a footgun with LinkerModule: Default Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: phil@chetcolibrary.org QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl If you read every word of the description of AutoCreateAuthorities in the system prefs page, and then read every word about it in the manual, even if you also read every word in the description and in the manual about LinkerModule, you'll have no idea that by turning on AutoCreateAuthorities without changing LinkerModule from its default value of Default, you've just given yourself infinite authority duplicates. The behavior of LinkerModule: Default is to link a field in a bib to an authority when there is one and only one matching authority record. Without AutoCreateAuthorities, that means if you have zero matches or two or more matches, the field is unlinked. Fine, no problem. With AutoCreateAuthorities on, the behavior is that if there are zero matches, an authority record is created, if there is one match, the field is linked to that existing authority record, if there are two matches a third duplicate authority is created, and if there are three matches, a fourth duplicate match is created, and if there are 2661 matches, a 2662nd duplicate match is created (that being how many authority records my system had for the genre Thrillers (Fiction) when I noticed this behavior). Two ways to not create infinite duplicates occur to me: Change what a LinkerModule returns, so it returns the match, whether it was a fuzzy match, and also whether more than one match was found, so that in the AutoCreateAuthorities case when more than one was found it can be left unlinked rather than creating another duplicate, or, Have C4/Linker/Default.pm check whether AutoCreateAuthorities is on, and if it is and $behavior isn't either first or last, just force it to first rather than creating infinite duplicates. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@bywatersolutions.com --- Comment #1 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- I like your first option better: teach AutoCreateAuthorities the difference between "there is no match" and "there's more than one possible match and therefore the linker didn't link anything." -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #2 from Phil Ringnalda <phil@chetcolibrary.org> --- Yeah, I was too focused on my half million duplicates that actually are duplicates resulting from this bug, and forgot that "duplicates" which only look like duplicates to the linker exist since it doesn't map indicators and possibly subfield 2 to 008/11 and 040$f in the authority to tell apart "650 _ 1 ‡aFantasy" a kids work of fantasy fiction and "650 _ 0 ‡aFantasy" an adult work about daydreaming. So forcing FirstMatch, which can only work correctly in certain limited circumstances, wouldn't be right. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 103858 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103858&action=edit Bug 25189: Unit tests -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #103858|0 |1 is obsolete| | --- Comment #4 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 103870 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103870&action=edit Bug 25189: Unit tests Adds new test for not adding authority if some already exist Also replaces use of 'SearchAuthorities' as it is Zebra specific -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #5 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 103871 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103871&action=edit Bug 25189: Don't create authority if results found Automatic authority creation assumes that if we don't match we need a new authority. Using the Default linker, however, we don't match if there exists more than one match. This leads to repeatedly generating authorities once there is a duplicate in the system We shoudl instead only create a new authority if there are no results To test: 1 - Set Linker Module to 'Default' 2 - Enable AutoCreateAuthorities and BiblioAddsAuthorities and CatalogModuleRelink and LinkerRelink 3 - Add two copies of a single authority via Z39 4 - Add a heading for that authority to a bib record 5 - Save the record and note a new authority is generated 6 - Repeat and see another is generated 7 - Apply patch 8 - Restart all the things 9 - Save the record again, no new authority created -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com, | |mkstephens@lancasterseminar | |y.edu, | |severine.queune@bulac.fr -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Make AutoCreateAuthorities |AutoCreateAuthorities can |less of a footgun with |repeatedly generate |LinkerModule: Default |authority records when | |using Default linker CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #103870|0 |1 is obsolete| | --- Comment #6 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 103894 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103894&action=edit Bug 25189: Unit tests Adds new test for not adding authority if some already exist Also replaces use of 'SearchAuthorities' as it is Zebra specific Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Andrew Fuerste-Henry <andrew@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #103871|0 |1 is obsolete| | --- Comment #7 from Andrew Fuerste-Henry <andrew@bywatersolutions.com> --- Created attachment 103895 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103895&action=edit Bug 25189: Don't create authority if results found Automatic authority creation assumes that if we don't match we need a new authority. Using the Default linker, however, we don't match if there exists more than one match. This leads to repeatedly generating authorities once there is a duplicate in the system We shoudl instead only create a new authority if there are no results To test: 1 - Set Linker Module to 'Default' 2 - Enable AutoCreateAuthorities and BiblioAddsAuthorities and CatalogModuleRelink and LinkerRelink 3 - Add two copies of a single authority via Z39 4 - Add a heading for that authority to a bib record 5 - Save the record and note a new authority is generated 6 - Repeat and see another is generated 7 - Apply patch 8 - Restart all the things 9 - Save the record again, no new authority created Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | Severity|enhancement |normal -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #103894|0 |1 is obsolete| | --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 106073 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106073&action=edit Bug 25189: Unit tests Adds new test for not adding authority if some already exist Also replaces use of 'SearchAuthorities' as it is Zebra specific Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #103895|0 |1 is obsolete| | --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 106074 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106074&action=edit Bug 25189: Don't create authority if results found Automatic authority creation assumes that if we don't match we need a new authority. Using the Default linker, however, we don't match if there exists more than one match. This leads to repeatedly generating authorities once there is a duplicate in the system We shoudl instead only create a new authority if there are no results To test: 1 - Set Linker Module to 'Default' 2 - Enable AutoCreateAuthorities and BiblioAddsAuthorities and CatalogModuleRelink and LinkerRelink 3 - Add two copies of a single authority via Z39 4 - Add a heading for that authority to a bib record 5 - Save the record and note a new authority is generated 6 - Repeat and see another is generated 7 - Apply patch 8 - Restart all the things 9 - Save the record again, no new authority created Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #10 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Change in behaviour makes sense and works as expected, tests pass, qa script passes. Passing QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 20.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #12 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 20.05.x for 20.05.02 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.11.00 |20.11.00, 20.05.02 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net Version(s)|20.11.00, 20.05.02 |20.11.00, 20.05.02, released in| |19.05.17 --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported to 19.05.x branch for 19.05.17 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #14 from David Cook <dcook@prosentient.com.au> --- I think that this is a beneficial change, as I have a library generating thousands of duplicates due to getting too many search results (due to a Zebra ICU bug it seems). However, I've been thinking it might be good if we can prevent cataloguing editor saves if we can't link headings to authorities, or provide a summary of the linking. The latter would be better for batch imports. I mention this here only because interested parties CCed here might have thoughts... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #15 from Fridolin Somers <fridolin.somers@biblibre.com> --- Victor and me are working on backport to 19.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.11.00, 20.05.02, |20.11.00, 20.05.02, released in|19.05.17 |19.11.13 --- Comment #16 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Victor Grousset/tuxayo from comment #13)
Backported to 19.05.x branch for 19.05.17
oops, old message sneaked in. Backported to 19.11.x branch for 19.11.13 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- Question for Phil Ringnalda: Phil, do you know if you're using CHR or ICU indexing? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #18 from Myka Kennedy Stephens <mkstephens@lancasterseminary.edu> --- (In reply to David Cook from comment #14)
However, I've been thinking it might be good if we can prevent cataloguing editor saves if we can't link headings to authorities, or provide a summary of the linking. The latter would be better for batch imports.
I mention this here only because interested parties CCed here might have thoughts...
I love this idea, David! It would probably need to be two enhancements. One for individual record saves that would prompt for authorities that would be created or can't be linked if the auto-create permission is turned off. The second would be a summary report of linking, failed links, and newly created authorities for batch imports. Our copy cataloging workflow is a one-at-a-time import via Z39.50, so something that would prompt before the record is saved would probably work better for us. If prompted, the cataloger could go back through the record to manually link any fields that came back with too many search results to link. Please share when you have bug numbers for this and thank you for the idea! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=11299 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 --- Comment #19 from David Cook <dcook@prosentient.com.au> --- Nick added a See Also to Bug 11299 which adds a button for doing the linking while cataloguing along with a summary at that time. Not quite what I had in mind, but it seems like it would be very useful. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |28676 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28676 [Bug 28676] AutoCreateAuthorities can repeatedly generate authority records when using Default linker and heading is cached -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org