[Bug 41714] New: MARC21 authorities thesaurus link hardcoded
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Bug ID: 41714 Summary: MARC21 authorities thesaurus link hardcoded Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: MARC Authority data support Assignee: koha-bugs@lists.koha-community.org Reporter: tomascohen@gmail.com QA Contact: testopia@bugs.koha-community.org UNIMARC heading mapping to thesaurus is read from the MARC bibliographic frameworks. On the other hand, MARC21 has them hardcoded in `C4::Heading::MARC21`. This has the side effect of not letting the linker tools honor the linking configuration from the frameworks. For example, if you define your own thesaurus `CEPAL_TOPIC` which you maintain with your own tools/process, and link it to `650$a`, running the linker will still look for `TOPIC_TERM` yielding bad results. We should have an implementation that: * Allows opting in to a UNIMARC-like behavior * Has a fallback mechanism for not-configured mappings so we wouldn't expect behavior change besides the frameworks-defined mappings * It doesn't reinvent the wheel on an old library -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com, | |tomascohen@gmail.com Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucasmontoya@theke.io -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #1 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192058 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192058&action=edit Bug 41714: Add system preference Patch from commit 842d8f1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #2 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192059 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192059&action=edit Bug 41714: Add tests Patch from commit ecc86dd -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #3 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192060 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192060&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) Patch from commit 3058cae -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #4 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- I ran out of time at the desk to write a proper test plan, but wanted to share early so others can see this. You should basically create a new authority type (CUSTOM_TERM), link it to a field, and check the linker doesn't work to link automatically (e.g. 650 is hardcoded to TOPIC_TERM, so it won't find your CUSTOM_TERM heading). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192058|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192059|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192060|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #5 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192101 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192101&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #6 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192102 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192102&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #7 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 192103 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192103&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |phil@chetcolibrary.org Status|Needs Signoff |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192101|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192102|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192103|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #8 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198890 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198890&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #9 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198891 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198891&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #10 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198892 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198892&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #11 from Phil Ringnalda <phil@chetcolibrary.org> --- If I run the tests with the preference set to "Don't use" they pass, but if I run them with it set to "Use" I get t/db_dependent/Heading/MARC21.t .. 1/6 # Failed test 'Uses framework override when preference enabled' # at t/db_dependent/Heading/MARC21.t line 109. # got: 'CUST_TERM' # expected: 'DjidPDEn_ZM' # Looks like you failed 1 test of 3. t/db_dependent/Heading.t ......... 1/7 # Failed test 'Uses framework authority type when enabled' # at t/db_dependent/Heading.t line 465. # got: 'CUST_TERM' # expected: 'eU_Sw39QVURvIx' # Looks like you failed 1 test of 1. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #198890|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #198891|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #198892|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #12 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198911 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198911&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #13 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198912 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198912&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #14 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 198913 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198913&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #15 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- (In reply to Phil Ringnalda from comment #11)
If I run the tests with the preference set to "Don't use" they pass, but if I run them with it set to "Use" I get
t/db_dependent/Heading/MARC21.t .. 1/6 # Failed test 'Uses framework override when preference enabled' # at t/db_dependent/Heading/MARC21.t line 109. # got: 'CUST_TERM' # expected: 'DjidPDEn_ZM' # Looks like you failed 1 test of 3.
t/db_dependent/Heading.t ......... 1/7 # Failed test 'Uses framework authority type when enabled' # at t/db_dependent/Heading.t line 465. # got: 'CUST_TERM' # expected: 'eU_Sw39QVURvIx' # Looks like you failed 1 test of 1.
Fixed! The test was a bit fragile I must admit! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #16 from Phil Ringnalda <phil@chetcolibrary.org> --- Still fails for me with the pref set. Apply patches, start ktd, set LinkerUseFrameworkAuthTypes to Use, ktd --shell, prove t/db_dependent/Headin* t/db_dependent/Heading/MARC21.t .. 1/6 # Failed test 'Uses framework override when preference enabled' # at t/db_dependent/Heading/MARC21.t line 117. # got: 'TOPIC_TERM' # expected: 't9QeHL' # Looks like you failed 1 test of 3. t/db_dependent/Heading.t ......... 1/7 # Failed test 'Uses framework authority type when enabled' # at t/db_dependent/Heading.t line 473. # got: 'TOPIC_TERM' # expected: 'tIWjEM8X' # Looks like you failed 1 test of 1. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #198911|0 |1 is obsolete| | Attachment #198912|0 |1 is obsolete| | Attachment #198913|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #17 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 199921 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199921&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #18 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 199922 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199922&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #19 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 199923 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199923&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #20 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- (In reply to Phil Ringnalda from comment #16)
Still fails for me with the pref set.
Apply patches, start ktd, set LinkerUseFrameworkAuthTypes to Use, ktd --shell, prove t/db_dependent/Headin*
t/db_dependent/Heading/MARC21.t .. 1/6 # Failed test 'Uses framework override when preference enabled' # at t/db_dependent/Heading/MARC21.t line 117. # got: 'TOPIC_TERM' # expected: 't9QeHL' # Looks like you failed 1 test of 3.
t/db_dependent/Heading.t ......... 1/7 # Failed test 'Uses framework authority type when enabled' # at t/db_dependent/Heading.t line 473. # got: 'TOPIC_TERM' # expected: 'tIWjEM8X' # Looks like you failed 1 test of 1.
This was a tricky caching issue. Glad you pointed it out. SOLVED! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199921|0 |1 is obsolete| | Attachment #199922|0 |1 is obsolete| | Attachment #199923|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #21 from Phil Ringnalda <phil@chetcolibrary.org> --- Created attachment 199997 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199997&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #22 from Phil Ringnalda <phil@chetcolibrary.org> --- Created attachment 199998 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199998&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #23 from Phil Ringnalda <phil@chetcolibrary.org> --- Created attachment 199999 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=199999&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #199997|0 |1 is obsolete| | Attachment #199998|0 |1 is obsolete| | Attachment #199999|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #24 from Magnus Enger <magnus@libriotech.no> --- Created attachment 200141 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200141&action=edit Bug 41714: Add `LinkerUseFrameworkAuthTypes` system preference This patch adds a new syspref for controlling if thesaurus links from teh frameworks should be followed in the MARC21 scenario, or if the hardcoded links should be used (current behavior). The current behavior would be kept. Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Magnus Enger <magnus@libriotech.no> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #25 from Magnus Enger <magnus@libriotech.no> --- Created attachment 200142 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200142&action=edit Bug 41714: Add tests This patch adds tests for the syspref-controlled behavior. Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Magnus Enger <magnus@libriotech.no> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41714 --- Comment #26 from Magnus Enger <magnus@libriotech.no> --- Created attachment 200143 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=200143&action=edit Bug 41714: Add a way to let the authority linker honor frameworks (MARC21) This patchset implements a way to make the authority linker follow the cataloguing frameworks when it comes to linked authority type. The current behavior is this for UNIMARC, but in the MARC21 case it just has hardcoded links (e.g. 650 is always linked to `TOPIC_TERM`). This is accomplished by: * Replicating what UNIMARC does, in MARC21 * Moving the code around so both use the same codebase * Adding tests * Adding the `LinkerUseFrameworkAuthTypes` system preference for opting into this new behavior. To test: 1. Launch a fresh KTD instance: $ ktd --proxy --name authorities --search-engine es9 up -d $ ktd --name authorities --wait-ready 120 2. On the staff interface (http://authorities-admin.localhost) open record 344 3. Change the cataloguing framework to the `Default` one and adjust the details so it can be saved. 4. Edit the `Default` framework to unlink the subfield `650$a` from the `TOPIC_TERM` thesaurus. 5. Go back to editing the record => SUCCESS: Field 650$a can be edited freely (normal behavior for unlinked subfields) 6. Add the following headings to repeatable `650$a`: * YERBA * MATE * BOMBILLA * AGUA CALIENTE 7. Create a new Authority type called `CUST_TERM` in `Admin > Authority types` * Make the 'field to copy' be 150 * Export the `TOPIC_TERM` authority type structure into CSV * Import the exported authority type onto `CUST_TERM` => SUCCESS: `TOPIC_TERM` and `CUST_TERM` are the same thing. 8. Add the following authorities of type `CUST_TERM` * YERBA * MATE * BOMBILLA 9. Edit the `Default` framework so `650$a` is now linked to `CUST_TERM` 10. Run: $ ktd --name authorities --shell k$ perl misc/link_bibs_to_authorities.pl --bib-limit "biblionumber=344" --auth-limit "authtypecode='CUST_TERM'" -l -g 650 => FAIL: You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:40:42 Run ended at: 01/27/26 14:40:43 Total run time: 276 ms Number of bibs checked: 1 Number of bibs modified: 0 Number of bibs with errors: 0 Number of headings linked: 1 Number of headings unlinked: 4 Number of headings fuzzily linked: 0 ``` This means it kept the currently linked heading, and the other 4 were kept unliked even though they exist on the `CUST_TERM` thesaurus. 11. Apply this patchset 12. Run: k$ updatedatabase k$ restart_all # just in case => SUCCESS: You see a syspref has been added 13. Repeat 10 => SUCCESS: Same result! 14. Set the `LinkerUseFrameworkAuthTypes` system preference to `Do` 15. Repeat 10: => SUCCESS: Yay! You see: ``` Bib authority heading linking report ======================================================= Linker module: C4::Linker::Default Run started at: 01/27/26 14:41:15 Run ended at: 01/27/26 14:41:16 Total run time: 302 ms Number of bibs checked: 1 Number of bibs modified: 1 Number of bibs with errors: 0 Number of headings linked: 4 Number of headings unlinked: 1 Number of headings fuzzily linked: 0 ``` This means *YERBA*, *MATE* and *BOMBILLA* where linked, the existing link was kept [1] and *AGUA CALIENTE* didn't get linked (of course, as it doesn't exist in `CUST_TERM`. 16. Run the tests: k$ prove t/db_dependent/Headin* => SUCCESS: Tests pass! 17. Sign off :-D [1] This really depends on how `LinkerKeepStale` is set, but what I describe is the default behavior. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Magnus Enger <magnus@libriotech.no> -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org