https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #2 from Stefan Berndtsson <stefan.berndtsson@ub.gu.se> --- Ok. Thank you. Just to make it clear to me. Sorry if I'm just repeating the obvious here. The main reason it would be hard to make it into a Koha Preference is that such a setting won't be usable in the Zebra files, but it could be used for ElasticSearch, at least in a possible future? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I think another reason might be that Koha was using $9 when there was no $0 subfield in MARC21 yet. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #6 from Stefan Berndtsson <stefan.berndtsson@ub.gu.se> --- We're in the process of testing and managing migration to Koha, and since our current database already contains multiple uses of $9, we may have to implement code changes to move Koha to $0 and change the zebra files as well then. We do intend on using ElasticSearch, but haven't got that bit setup yet. If we do such a code change, and make the Koha side of the change into a system preference, would that be something that should currently be placed in the Labs section with a big warning sign on it? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Josef Moravec <josef.moravec@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |josef.moravec@gmail.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |frank.hansen@ub.lu.se -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Johanna Räisä <johanna.raisa@koha-suomi.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |johanna.raisa@koha-suomi.fi Status|NEW |Needs Signoff --- Comment #10 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- Created attachment 88049 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=88049&action=edit Bug 17511 - Authority linking subfield Bug 17511: Authority linking subfield Koha uses $9 subfield for linking authority records. MARC21 format says the subfield should be $0. This commit will change Koha to use subfield of selected marc flavour. For UNIMARC the subfield is defined to use $9 and for others $0. To change the old linkings run batchAuthorityLinking.pl, modify index mappings and reindex. TEST PLAN: 1) Add new authority linking to a record. 2) Verify that the authority records ID is on $9 subfield. 3) Check index that authority has one record chosen. 4) Apply the patch. 5) Replace zebradb/marc_defs with new files or/and reset ES configurations. 6) Add new authority linking to a record. 7) Verify that the authority records ID is now on the subfield $0. 8) Check index again that authority has record chosen. Sponsored-by: Koha-Suomi Oy -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Johanna Räisä <johanna.raisa@koha-suomi.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|unspecified |master -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #12 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- The prefix is mainly the indicator from which system the authority has come. Now the script only finds $9 subfield and changes it to $0 and doesn't do anything else so if there are more $0 subfields those are still staying. The main problem is that now this doesn't support MARC21 format and the $9 subfield is needed for other things. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #14 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I realize you haven't made any changes to the XSLT files in your patch. They use $9 to create links and woudl need to be adapted. With multiple $0 there will be problems: - We need to figure out which is the one to use for linking. - We need to make sure that the indexing is exact enough. Currently () are ignored, so (XXXX)1235 woudl be found when looking for 1235. If you have multiple $0 that could cause issues. Right now, the linking is exact. I am not saying we shouldn't discuss this, but I think it needs more work. I see that some authority fields are hardcoded - a lot of libraries seem to put authority control on other fields as well as Koha allows this in configuration. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Change authority linking |Change authority linking |subfield from $0 to $9 |subfield from $9 to $0 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Johanna Räisä <johanna.raisa@koha-suomi.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #88049|0 |1 is obsolete| | --- Comment #16 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- Created attachment 114433 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=114433&action=edit Bug 17511 - Authority linking subfield Bug 17511: Authority linking subfield Koha uses $9 subfield for linking authority records. MARC21 format says the subfield should be $0. This commit will add a system preference AuthoritySubfieldLink which is used to define the linking subfield. The default subfield is set to $9. To change the old linkings run batchAuthorityLinking.pl, modify index mappings and reindex. TEST PLAN: 1) Add new authority linking to a record. 2) Verify that the authority records ID is on $9 subfield. 3) Check index that authority has one record chosen. 4) Apply the patch. 5) Check AuthoritySubfieldLink to be 9. 6) Repeat steps 1, 2 and 3. 7) Change the preference value 8) Replace zebradb/marc_defs with new files or/and reset ES configurations. 9) Add new authority linking to a record. 10) Verify that the authority records ID is now on the chosen subfield. 11) Check index again that authority has record chosen. Sponsored-by: Koha-Suomi Oy -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #18 from David Cook <dcook@prosentient.com.au> --- I was thinking about this idea just the other day... If you look at https://www.loc.gov/marc/bibliographic/ecbdcntf.html, you'll see that $0 can also be a URI. That could be pretty cool. It seems too that the $0 is repeatable so that you could have different identifiers... I guess so you could reference VIAF and something else? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #20 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #18)

I was thinking about this idea just the other day...

If you look at https://www.loc.gov/marc/bibliographic/ecbdcntf.html, you'll see that $0 can also be a URI. That could be pretty cool.

It seems too that the $0 is repeatable so that you could have different identifiers... I guess so you could reference VIAF and something else?

We could index a URI, so that Koha's bib to auth link search would still work... We could actually build in a RDF dereferencing system eventually. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Johanna Räisä <johanna.raisa@koha-suomi.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #114433|0 |1 is obsolete| | --- Comment #22 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- Created attachment 114474 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=114474&action=edit Bug 17511 - Authority linking subfield The patch had wrong author information, replaced with my own. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #24 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- (In reply to David Cook from comment #23)

(In reply to Johanna Räisä from comment #21)

...

Do you mean putting Koha's internal link to authority linking subfield?

I'm not 100% sure what you mean, but I think so, yes.

I meant that the authid would be changed to URI and saved that instead of integer value.

...

We are using national metadata repository and mainly the URI in $0 subfield has a link to that. It might be a good idea that Koha would use URI in authority linking subfield also.

Yeah that sounds good to me. So you wouldn't be storing authority records in Koha?

No we won't.

...

We are facing problem with this $9 subfield because it is used in our national metadata repository for other things and it should not change if we are pulling data to Koha. This is why it would be nice to choose which subfield is used for the authority linking.

Hmm, how are you pulling the authority data to Koha?

We are making a plugin which fetches the authority via SRU or REST API while writing it to the field. We have already a similar approach to get vocabularies https://github.com/KohaSuomi/Koha/commit/9dec82f3c28b41946f8b301dd9200cc36e9... -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #26 from David Cook <dcook@prosentient.com.au> --- (In reply to Johanna Räisä from comment #16)

TEST PLAN: 1) Add new authority linking to a record. 2) Verify that the authority records ID is on $9 subfield. 3) Check index that authority has one record chosen. 4) Apply the patch. 5) Check AuthoritySubfieldLink to be 9. 6) Repeat steps 1, 2 and 3. 7) Change the preference value 8) Replace zebradb/marc_defs with new files or/and reset ES configurations. 9) Add new authority linking to a record. 10) Verify that the authority records ID is now on the chosen subfield. 11) Check index again that authority has record chosen.

Could you elaborate on Step 8? I think it would be wise to add an additional example patch (marked with something like DONOTPUSH) with your desired configuration for etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl That'll make it easier for us to test. Otherwise, the tester and QA have to guess at the correct configuration. I think that this sounds like a good change though. Wise to keep the $9 for default, and maybe we can announce a deprecation in favour of moving to $0 at some point... -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #28 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- (In reply to Johanna Räisä from comment #27)

Created attachment 114918 [details] [review] Bug 17511 - Authority linking subfield

Koha uses $9 subfield for linking authority records. MARC21 format says the subfield should be $0. This commit will add a system preference AuthoritySubfieldLink which is used to define the linking subfield. The default subfield is set to $9.

To change the old linkings run batchAuthorityLinking.pl, modify index mappings and reindex.

TEST PLAN: 1) Add new authority linking to a record. 2) Verify that the authority records ID is on $9 subfield. 3) Check index that authority has one record chosen. 4) Apply the patch. 5) Check AuthoritySubfieldLink to be 9. 6) Repeat steps 1, 2 and 3. 7) Change the preference value to 0 8) Replace zebra defs and elastic mapping file with the commit sample files for testing. 9) Add new authority linking to a record. 10) Verify that the authority records ID is now on the $0 subfield. 11) Check index again that authority has record chosen.

Sponsored-by: Koha-Suomi Oy

I added only indexing file samples to this patch. Have to think how the URI would work with indexing. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #30 from David Cook <dcook@prosentient.com.au> --- (In reply to Johanna Räisä from comment #28)

Have to think how the URI would work with indexing.

I have been thinking about the same thing. At a glance, with Zebra we're using Koha-Auth-Number with a "w" register, which is fine for integers, but that probably won't work very well with URIs, since it's for storing "words". It's normalized and tokenized. That will lead to problematic indexing. We could use a "p" register which is normalized and not tokenized but it's possible for 2 different URIs to be normalized into the same data, so that's not ideal either. Perhaps we should use a "0" register which is not normalized and not tokenized. To use that we'd need to use the 4=3 structure attribute and I don't think we'd be using it as the Bib-1 spec says it should be used (https://www.loc.gov/z3950/agency/bib1.html), but I think it would work with Zebra. I'd have to try it out though. Technically, the is a special "u" register for URLs, but we also need to optionally store integers, so that won't work either. I don't use Elasticsearch, so I can't speak to it. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Johanna Räisä <johanna.raisa@koha-suomi.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #114918|0 |1 is obsolete| | --- Comment #32 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- Created attachment 115002 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=115002&action=edit Bug 17511 - Authority linking subfield Koha uses $9 subfield for linking authority records. MARC21 format says the subfield should be $0. This commit will add a system preference AuthoritySubfieldLink which is used to define the linking subfield. The default subfield is set to $9. To change the old linkings run batchAuthorityLinking.pl, modify index mappings and reindex. TEST PLAN: 1) Add new authority linking to a record. 2) Verify that the authority records ID is on $9 subfield. 3) Check index that authority has one record chosen. 4) Apply the patch. 5) Check AuthoritySubfieldLink to be 9. 6) Repeat steps 1, 2 and 3. 7) Change the preference value to 0 8) Replace zebra defs and elastic mapping file with the sample files from second patch for testing. 9) Add new authority linking to a record. 10) Verify that the authority records ID is now on the $0 subfield. 11) Check index again that authority has record chosen. Sponsored-by: Koha-Suomi Oy -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #34 from Johanna Räisä <johanna.raisa@koha-suomi.fi> --- (In reply to David Cook from comment #30)

(In reply to Johanna Räisä from comment #28)

...

Have to think how the URI would work with indexing.

I have been thinking about the same thing.

At a glance, with Zebra we're using Koha-Auth-Number with a "w" register, which is fine for integers, but that probably won't work very well with URIs, since it's for storing "words". It's normalized and tokenized. That will lead to problematic indexing.

We could use a "p" register which is normalized and not tokenized but it's possible for 2 different URIs to be normalized into the same data, so that's not ideal either.

Perhaps we should use a "0" register which is not normalized and not tokenized. To use that we'd need to use the 4=3 structure attribute and I don't think we'd be using it as the Bib-1 spec says it should be used (https://www.loc.gov/z3950/agency/bib1.html), but I think it would work with Zebra. I'd have to try it out though.

This sounds good to me, although I have little experience configuring Zebra.

Technically, the is a special "u" register for URLs, but we also need to optionally store integers, so that won't work either.

I don't use Elasticsearch, so I can't speak to it.

I can look Elasticsearch. I have work with it more than I have worked with Zebra. Maybe we should make another bug for this? At least I think this is another enhancement. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #35 from David Cook <dcook@prosentient.com.au> --- I'm intrigued by this one but super busy at the moment, unfortunately. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |phil@chetcolibrary.org -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 --- Comment #38 from David Cook <dcook@prosentient.com.au> --- (In reply to Phil Ringnalda from comment #37)

It would be a good idea to remove the last mentions of $0 from the patch, and also from the bug summary, since this isn't at all a $0 implementation. $0 is a repeatable subfield, with specified contents of "(MARC org code from 003 in an authority record)control number of an authority record", which isn't what this is doing.

I haven't reviewed these patches in a while, so I can't speak to the first part of that. However, note that $0 is more complex than "(MARC org code from 003 in an authority record)control number of an authority record". See "$0 - Authority record control number or standard number" at the following URL for more details: https://www.loc.gov/marc/bibliographic/ecbdcntf.html -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17511 Esther Melander <esther.melander@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |esther.melander@bywatersolu | |tions.com --- Comment #39 from Esther Melander <esther.melander@bywatersolutions.com> --- To add a wrinkle to this discussion. OCLC is using $0 in authority records, perhaps as intended with linking information. An example authority: Conduct of life. $2 fast $0 (OCoLC)fst00874563 In this case, a library wants to retain this subfield as a reference point. It would cause a conflict if a library is using $0 for internal linking and brought in a record which contains reference data in this field for a different system. -- You are receiving this mail because: You are watching all bug changes.