https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #2 from Janusz Kaczmarek <januszop@gmail.com> --- Yes, I agree, definitely. The (relatively) new $1 should be used for identifying Real Would Objects. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |paul.poulain@biblibre.com --- Comment #3 from Paul Poulain <paul.poulain@biblibre.com> --- One of our libraries is interested by this feature, they asked me to have a look at this ticket. Some explanations (from history) : I'm the developer who wrote the MARC management in Koha (in 2003...) The $9 was chosen to store a link between biblio & authority because it was for local use (as any field/subfield with a 9). In UNIMARC, the $3 was a possible field (as it contains the linkage), in MARC21 there was no subfield for that. But anyway, I thought (and still think) it was better to keep the $3 as it is and use a local field because it stores a local information (the authid of the authority in the local [Koha] system) In France, the SUDOC (national academic catalogue) makes an extensive usage of the $3, we must preserve it. Otherwise : That's a very good idea to add the wikidata reference (or any other one) and your specification seems OK, I have a couple of comment though: - switch the feature on/off => not sure it's needed, if the next syspref is empty, it's OFF ;) - source of the standard IDs (possibly different for different biblio fields?) (standard in MARC 21 authority: 024, but may be also 010/035) (a list) => I agree, it must not be at a global level. But counter-proposal : define that in the authority level. In the authority definition, we define which authority field must be copied in the biblio field [for example, corporate name is copied from the 110 of the authority]. Shouldn't we have another option to say "024$a" is copied in the $1 of the biblio ? [the $1 is possible for UNIMARC, it's an unused field] - how many ids should be copied (= how may $0 added): only the first found or all of them => I would say "copy only one". Of maybe add another option in the authority definition to say "add the 1st where source/$2=[_____]" [$2 is valid for UNIMARC too] - if an extra link/icon should be generated in display (OPAC / librarian) => I think we just need to improve the XSLT for that. Very interesting proposal though ! I'll continue talking with the library to see if they can sponsor the dev -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Thinking along these lines about it: Data looks like 024 7# - OTHER STANDARD IDENTIFIER Standard number or code: https://id.rijksmuseum.nl/31018715 Source of number or code: uri 024 7# - OTHER STANDARD IDENTIFIER Real World Object URI: https://rkd.nl/explore/artists/15189 Source of number or code: rkda 024 7# - OTHER STANDARD IDENTIFIER Real World Object URI: http://www.wikidata.org/entity/Q454568 Source of number or code: wikidata Add a pref to know which field numbers to copy (024 for MARC21). Copy $1 from auth to $1 at biblio side. (Repeatable) Leave #2 alone. It is something different. In the above case we need to decide on the best way to show all (3) additional URIs on the detail page that come with this author (or another auth type). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- For MARC21 I think identifiers for $0 are often found in 001/035: https://www.loc.gov/marc/authority/ad035.html -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to bruno.forment@orpheusinstituut.be from comment #6)

(In reply to Katrin Fischer from comment #5)

...

For MARC21 I think identifiers for $0 are often found in 001/035: https://www.loc.gov/marc/authority/ad035.html

Those identifiers work only on the level of the complete record, not with respect to the various authorities (personal, geographic, topical terms etc.).

Hi Brono, I meant the 035 in authorities, not 035 in the bibliographic record. In our case for example one of the 035 of the authority has the ID from the national authority database for this person/subject/topical term. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #9 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 155891 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155891&action=edit Bug 22972: Modify MARC21 XSLT to include link to external data ($1) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff CC| |julian.maurice@biblibre.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Hi Julian, How does this work together with pref AuthorityMergeMode ? If you are in strict mode, the merge is overwriting or removing subfields. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Julian Maurice from comment #12)

Data is not copied to the biblio field while merging authorities. I haven't tested but I suppose that if you are in strict mode, the biblio field's $1 is removed.

Yes, thats my point. It would be a temporary enrichment. But note that 99% is on loose mode luckily. (Checked HEA numbers.) Should this be noted somewhere? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Thibault Keromnès <thibault.keromnes@univ-paris8.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |thibault.keromnes@univ-pari | |s8.fr Status|Needs Signoff |Patch doesn't apply --- Comment #15 from Thibault Keromnès <thibault.keromnes@univ-paris8.fr> --- Patch doesn't patch apply : CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/admin/authtypes.tt -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Baptiste Wojtkowski (bwoj) <baptiste.wojtkowski@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #17 from Baptiste Wojtkowski (bwoj) <baptiste.wojtkowski@biblibre.com> --- Created attachment 170444 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=170444&action=edit Bug 22972: Modify MARC21 XSLT to include link to external data ($1) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Baptiste Wojtkowski (bwoj) <baptiste.wojtkowski@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |baptiste.wojtkowski@biblibr | |e.com --- Comment #19 from Baptiste Wojtkowski (bwoj) <baptiste.wojtkowski@biblibre.com> --- Rebased on main -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Janusz Kaczmarek <januszop@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #21 from Janusz Kaczmarek <januszop@gmail.com> --- Thank you, Baptiste, for supplying this patch. We tested it: it works fine for adding $1 (or any other subfield indicated in configuration) in the standard editor. What doesn't work: 1. [advanced editor] In the advanced editor, if a authority record has more than one 024 field, all occurrences of the indicated subfield are concatenated into a single string and placed in a single $1 subfield. 2. [update/merge] Adding the 024 field with the subfield indicated in the configuration to the authority record does not add subfield $1 to the heading field in bibliographic records linked with the authority record in question. 3. [link] In a situation where authority and bibliographic records have been loaded into the system from an external source, and then the linking process has been started (link_bibs_to_authorities.pl), the 024 $1 subfields (or other subfields indicated in the configuration, respectively) are not added as $1 subfields in the corresponding headings of the bibliographic records. There is also the question of how Koha should behave in the case of authority records with the 024 $1 subfields already present in the database and linked to bibliographic records - wouldn't we need a script to rewrite the headings and add the $1 subfields. And a final note: displaying information from the subfield $1 in the bibliographic record view seems very unergonomic to us, especially for URIs outside the list hardcoded in the XSLT (currently: viaf and wikidata), such as ISNI. We would suggest integrating this information into the functionality introduced by bug 29948, as a possible follow-up. Would you be ready to work further on 1, 2, 3? And what do you think about final remarks... -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #23 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Janusz Kaczmarek from comment #21)

1. [advanced editor] In the advanced editor, if a authority record has more than one 024 field, all occurrences of the indicated subfield are concatenated into a single string and placed in a single $1 subfield. It looks like a bug in the advanced editor. I reproduce it on main branch with other subfields.

2. [update/merge] Adding the 024 field with the subfield indicated in the configuration to the authority record does not add subfield $1 to the heading field in bibliographic records linked with the authority record in question. Confirmed. I'll send a fix

3. [link] In a situation where authority and bibliographic records have been loaded into the system from an external source, and then the linking process has been started (link_bibs_to_authorities.pl), the 024 $1 subfields (or other subfields indicated in the configuration, respectively) are not added as $1 subfields in the corresponding headings of the bibliographic records. I'm not familiar with the linker, but it looks like it does not even try to update the biblio field (except adding a $9 subfield of course). So I think this should not be changed in the context of this bug.

There is also the question of how Koha should behave in the case of authority records with the 024 $1 subfields already present in the database and linked to bibliographic records - wouldn't we need a script to rewrite the headings and add the $1 subfields. I think that misc/maintenance/update_authorities.pl may help. For instance: misc/maintenance/update_authorities.pl -merge -reference $authid -c But that requires calling it for each authority

And a final note: displaying information from the subfield $1 in the bibliographic record view seems very unergonomic to us, especially for URIs outside the list hardcoded in the XSLT (currently: viaf and wikidata), such as ISNI. We would suggest integrating this information into the functionality introduced by bug 29948, as a possible follow-up.

I'll look into it -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Mark Hofstetter <mark@hofstetter.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mark@hofstetter.at -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #25 from Fridolin Somers <fridolin.somers@biblibre.com> --- Needs tidy right ? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #27 from Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> --- Created attachment 204603 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204603&action=edit Bug 22972: Modify MARC21 XSLT to include link to external data ($1) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #29 from Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> --- Created attachment 204605 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=204605&action=edit Bug 22972: Support 'subfield_to_report_in_1' when merging authorities -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |thibaud.guillot@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA CC| |jonathan.druart@gmail.com --- Comment #31 from Jonathan Druart <jonathan.druart@gmail.com> --- QA review: 1. There is no check on the value entered. If you enter 0241 instead of 024$1 it saved as it without warning, but then it fails silently when cataloguing. 2. The placeholder is "024$a" but the hint says: "Enter the authority subfield that should be copied from the authority record to the bibliographic record in $1. E.g., in MARC21, field 024$1 in the authority record should be copied to field 700$1 in the bibliographic record" Shouldn't the placeholder be "024$1" then? 3. Shouldn't we display the value in the authority types table in a new column? 4. If there are several $1 ("aaa" and "bbb"), the advanced editor is not correct: 700 _ _ ‡aCameron, Debra.‡1aaabbb‡9380 5. "this works in UNIMARC too" But we only modify MARC21's XSLTs? 6. Missing tests for "sub merge" 7. Isn't there a problem when merging?: the $1 on a linked biblio field is only replaced when the target authority has values to copy. So removing an authority's 024$1 or merging it into one without an ID will keep the old $1 on the biblio. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@openfifth.c | |o.uk --- Comment #32 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- There's also no output from the atomicupdate to say whether the update ran successfully or failed ;) The UNIMARC claim doesn't hold up either. I checked Koha's UNIMARC default frameworks directly: - UNIMARC authority format has no field 024 at all — that tag doesn't exist in authorities_normal_unimarc.yml. The test-plan's own example (024$1) is literally inapplicable to UNIMARC. - UNIMARC bibliographic 700 has no subfield 1 defined in the default framework at all. - Where UNIMARC does define subfield 1 (authority field 160), it means "Linking Data" — part of the 4XX/5XX cross-reference/tracing mechanism — a completely different, incompatible semantic from MARC21's "Real World Object URI". Reusing $1 for a Wikidata link in a UNIMARC record would collide with an already-reserved meaning. - Consistent with that, no UNIMARCslim2intranetDetail.xsl / UNIMARCslim2OPACDetail.xsl changes exist anywhere in this patchset. I think perhaps we should actually make this MARC21 only unless you're aware of people wanting it for UNIMARC and then it should really have it's own path for UNIMARC using the field appropriate for that scheme? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #33 from Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> --- Created attachment 206672 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206672&action=edit Bug 22972: (QA follow-up) Restrict admin config to MARC21, validate and label The 'Authority subfield to copy in $1' setting only makes sense for MARC21 (the $1 subfield does not have the same meaning in UNIMARC), so hide the field and the equivalent list column when marcflavour is not MARC21. Also: - Fix the placeholder to show '024$1' instead of the mismatched '024$a', to stay consistent with the help text underneath. - Add a 'pattern' attribute. - Validate the format server-side and report an error instead of silently storing an invalid value. - Add a 'Copied to biblio $1' column to the authority types list. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #35 from Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> --- Created attachment 206674 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206674&action=edit Bug 22972: (QA follow-up) Fix concatenation of repeated values The [code] marker was emitted once per SUBFIELD_LOOP entry, then all values of a repeated subfield were concatenated inside the inner loop (e.g. 1aaabbb instead of 1aaa1bbb). Move the marker inside the inner FOREACH so it is repeated in front of each value. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #37 from Thibaud Guillot (thibaud_g) <thibaud.guillot@biblibre.com> --- Created attachment 206676 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=206676&action=edit Bug 22972: (QA follow-up) Add tests for subfield_to_report_in_1 in merge Cover the nominal case (repeated 024$1 values correctly copied as distinct $1 subfields on the linked biblio) and the regression fixed in the previous commit (a stale $1 must be cleared from the biblio once the source 024$1 is gone). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #39 from David Cook <dcook@prosentient.com.au> --- Could someone post a consolidated test plan? I'm looking through the comments and I have no idea what this is supposed to do. Although it sounds very strange... I'm hoping I'm misunderstanding the comments so far... -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #41 from David Cook <dcook@prosentient.com.au> --- I might be misunderstanding the discussion here... but this sounds super wrong. Look at https://www.loc.gov/marc/bibliographic/ecbdcntf.html and https://www.loc.gov/marc/authority/ecadcntf.html The Auth 024 field is completely irrelevant for a Bib XXX field. If it was a case of copying an Auth 100$1 to a Bib XXX field, that would be different. But there is no Auth 100$1 subfield. Which makes sense, because in the Bib XXX field we're linking to a local Authority record. We *should* migrate from the $9 to the $0. But the $1 would be for pointing to external entities. I mean... I get the idea... in theory the 024$1 could conceptually be imported into the Bib XXX field, but I'm seeing *zero evidence* that MARC is designed to work that way. This sounds like a hack designed for a particular library or libraries. Do we really want this in core? We should be trying to simplify the Koha core to make it easier to maintain rather than bolting on more and more stuff to it. This would be a perfect example of something where a Koha Plugin would be better suited to the solution. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #43 from David Cook <dcook@prosentient.com.au> --- Apologies for the grumpy tone. I know there's a balance between shiny new features and long-term maintenance and stability. But as AI accelerates the discovery and exploitation of security vulnerabilities, we really should be trying to make Koha as solid as we can. I'm all for experiments and innovations. But I think we really should be questioning what should be in core and what should be a plugin/extension. As we move forward with the API backend and Vue.js frontend, it becomes easier and easier to built pluggable functionality. Feel free to change the status away from "In Discussion". I just wanted to slow things down a little and invite people to reflect before surging forward. Thank you. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #45 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #44)

Ok quick question... what is the actual objective here?

Looking at Mattermost, Paul has said: "is there someone around here that could QA BZ22972 ? dutch libraries are hoping to have it in 26.11 (look at the sponsors ;) )"

Can someone from the Dutch libraries explain what the goal is here?

Looking at the diff, it looks like the goal is to show $1 fields through the XSLTs. Ok. Fair enough I guess. I don't really see the utility but OK. And then there's an editor change, a change to how authorities work in general. Why not just do the XSLT change to start? Then look at optional changes to the cataloguing editor? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #47 from David Cook <dcook@prosentient.com.au> --- I promise I'm about to stop. Let's look at a Library of Congress example. Authority: https://id.loc.gov/authorities/names/no2015039717.html Bib/Work: https://id.loc.gov/resources/works/24239840.html Notice that the Authority has links to VIAF and OCLC, but the Bib/Work does not. On the Bib/Work, they link to the local LoC entity: http://id.loc.gov/rwo/agents/no2015039717 Looking at a LCSH authority: https://id.loc.gov/authorities/subjects/sh85025303.html Then on that authority page, you have the links to all the other URLs for the same concept. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #49 from David Cook <dcook@prosentient.com.au> --- (In reply to bruno.forment@orpheusinstituut.be from comment #48)

I want to check in as one of the libraries behind this patch, on the question of whether it's "in tune with MARC."

Thanks, Bruno. That's much appreciated.

I'd push back gently on using strict MARC field semantics as the bar here. MARC was never designed with Linked Open Data or FAIR principles in mind - it predates them by decades, and its rigidity around what belongs in which subfield is precisely one of the things libraries are trying to move past. If we treat "MARC wasn't designed to work that way" as a blocking argument, we will structurally block almost every attempt to bring external, resolvable identifiers into bibliographic data, because MARC's field definitions simply don't anticipate this use case anywhere.

Except that Koha is a MARC-based system and it does have $0 and $1 subfields which do work semantically here.

The actual goal of this patch is straightforward: authority records already accumulate valuable external identifiers (VIAF, Wikidata, ISNI, etc.), and we want those identifiers to be discoverable directly from the bibliographic record, not locked behind a join to the authority table that most external tooling, harvesters, and discovery layers never perform. That's a FAIR-compliance goal - and it's squarely in the direction libraries are moving, whether or not it maps onto a pre-existing MARC pattern.

Except if we look at the Library of Congress (the entity behind BIBFRAME), we can see that they don't bring in all those valuable external identifiers into the Bib/Work. They keep them in the Authority record which is linked from the Bib/Work. In this example, MARC is actually irrelevant. Conceptually, authority and bibliographic identifiers are kept where they're supposed to be. Authority: https://id.loc.gov/authorities/names/no2015039717.html Bib/Work: https://id.loc.gov/resources/works/24239840.html

On the specific technical objections: they're valid critiques of the current implementation (handling of repeated fields, $2, UNIMARC scoping, validation) and the QA follow-ups have already addressed most of them. That's normal patch maturation, not evidence the underlying idea is wrong. I don't think "this isn't how MARC traditionally does things" should be conflated with "this shouldn't be built."

It's not about tradition. It's about standards. An open source system like Koha is wonderful because you can see the code and make any changes you want. But that doesn't mean that it's the right decision for all the Koha libraries in the world.

On core vs. plugin: this is a small, opt-in, configurable feature restricted to MARC21. That's a reasonable footprint which directly serves libraries actively investing in LOD.

It's small but it touches core libraries and it touches them without any system-level configuration/system preference. The only configuration is in the auth type which is a library-level config. It makes Koha's authority system, which is already too bespoke even more bespoke. To me, if you wanted those 024$a identifiers in the $1, I don't see why you wouldn't copy them by hand. Personally, I don't see the merit of including them automatically. Essentially, copying the Auth 024$a into the $1 of the Bib record is just denormalising the data, and for what gain? If it was a display thing, then an API call to fetch the authority data at display time would work well. If it's 3rd party tools, why would they be interested in the other $1 URLs? Surely, it would be enough to link to 1 $1 URL to create the linkage for the Linked Open Data. I mean that's part of the point of linking. So that you don't have to copy data like this.

In sum, we'd like to keep this moving so that everyone can benefit from this investment.

That makes a lot of sense, and I want to re-iterate that I don't want to be a blocker. I'm happy for people to reset the status. I just want to have people re-consider if this is really the best design for Koha. Thank you for sharing your perspective. It's really valuable. I would love to hear from more libraries interested in working with LoD. And I would love to hear more from you, Bruno, about how you'd like to work with LoD and Koha. The more use cases we get from libraries, the more we can shape Koha to work for you (and other libraries around the world). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #51 from David Cook <dcook@prosentient.com.au> --- (In reply to bruno.forment@orpheusinstituut.be from comment #50)

$0 indeed works for the local authority link. But $1 in bibliographic records is still relatively rarely used in production, despite being defined for Real World Object URIs. The fact that it "works semantically" per the MARC documentation for a different purpose doesn't mean this use is wrong - it just means it hasn't been exercised this way yet. Every extension of practice has to start somewhere.

I'm confused... isn't the point of these patches to add $1 subfields?

...

"Except if we look at the Library of Congress [...] they don't bring in all those valuable external identifiers into the Bib/Work. They keep them in the Authority record which is linked from the Bib/Work."

That's true for the LoC ecosystem, where every consumer of bibliographic data is assumed to also be able to resolve the authority side via id.loc.gov. That's not the reality for many Koha installations. Our bib data constantly leaves the system without the authority records attached: OAI-PMH harvesting, MARCXML export to union catalogues, indexing into Solr/Elasticsearch for discovery layers, data exchange with APIs, etc. All of those consumers see only the bib record. If the external ID lives solely in the authority, it's invisible to that entire chain unless every downstream consumer sets up a live link into our Koha authority table - which in practice nobody does.

But you were saying before that you want to create Linked Open Data. If you're going to use a local authority, then it would make sense to link to that local authority. That's why I think we should shift from the $9 for an authid to a $0 with a canonical URL to the authority record.

...

"To me, if you wanted those 024$a identifiers in the $1, I don't see why you wouldn't copy them by hand. [...] I don't see the merit of including them automatically."

Copying by hand undermines exactly what authority control is for: one point of maintenance, automatically consistent everywhere the authority is used. With hundreds or thousands of bib records linked to a single authority, manual upkeep isn't realistic, and any correction or merge of the authority would again require touching every linked record by hand. That's precisely the problem authority control already solves for $a/$9; we're only asking to extend that same logic to $1.

What I mean is... you don't need all those $1s but if you want to add them you can add them. Why would a correction require touching every linked record by hand? You've already made the link via the URL. Unless you're talking about changes to URLs. Then you're already in a different realm of bulk/batch changes.

...

"Essentially, copying the Auth 024$a into the $1 of the Bib record is just denormalising the data, and for what gain?"

Yes, it's denormalisation - just like the 6XX$a we already populate with the authority's preferred term, or the $9 we already populate with the authid. Denormalisation for the sake of portability is already standard practice in library data, not an exception. The gain is that the external identifier can leave the bib record without requiring an extra join.

No, the $9 is not denormalization. It's a linkage. It's the key to create that "join". You do have me thinking again though... LoD can be a pain because following links on-demand/at run-time is a real big pain. Much more efficient to denormalize. That's true... but in a different context. It's true in the context where you might periodically cache fetched data from external URLs to improve performance. Especially if you're using a graph database. In this Koha context, all you're doing is displaying the URLs. There's other ways of easily fetching that data rather than embedding it into the bib record. That said, I take your point. You want an easy way to automate the population of $1 URLs in a bib record from the 024$a URLs in the authority record. While I don't see the merit overall, I can understand the merit in terms of your specific goal for sure. But it seems to me that a plugin to the cataloguing editor would make more sense here, because really that's what the change is all about, right? You want to show the data via the XSLTs and you want to copy the URLs from the authority record to the bib record at catalogue time. I suppose what I'm trying to say is... this change adds to the existing technical debt by adding more code on an already poorly designed mechanism. While it's a small change to the existing system, it's adding to the burden. We want to be easing the burden. But easing the burden takes more work in the short-term so it's rarely done.

...

"If it's 3rd party tools, why would they be interested in the other $1 URLs? Surely, it would be enough to link to 1 $1 URL to create the linkage."

There isn't one universal "best" identifier: one consumer wants Wikidata (as we do – it's a citizen science-compliant recognized Digital Public Good), another wants VIAF (which is mostly included in Wikidata entities), another wants ISNI, etc. Multiple $1s let each consumer pick up the identifier relevant to their use case, without us as a library having to decide in advance which consumer we're serving.

I think part of the problem here is that you're thinking in terms of identifiers instead of concepts. There is no universal "best" identifier, but in practice for libraries you typically would link to an authority record. Traditionally, that authority record has lived in Koha, but it would be interesting to link to external authorities instead. (Now that would be a really great new feature to help support Linked Open Data in Koha cataloguing!) What are the use cases of your consumers? Real or hypothetical; I'm curious. As a library, what you're doing is you're saying "Hey this is a resource we have, and this metadata description I'm using has an authoritative form which you can find in X URL location".

Finally, you're right that this sits in core and that the configuration is at the authority-type level rather than a system preference. That seems like a fair point to carry into further QA - an explicit system preference to toggle this globally could address the concern about "silent, bespoke behaviour."

I think I would be more amenable to this change if there were a system preference for it - for sure.

But the core point stands for us: without this, external identifiers stay locked in the authority silo, invisible to anything outside Koha itself. That's exactly the problem the Semantic Web is meant to solve, and it's why we consider this worth pursuing further - not out of stubbornness, but because it answers a real interoperability problem.

I disagree. For instance, take this real life document that uses LOD from the University of Vienna using Fedora Commons: https://phaidra.univie.ac.at/detail/o:2353677 and https://phaidra.univie.ac.at/api/object/o:2353677/json-ld Specifically, look at this section: "edm:hasType": [ { "@type": "skos:Concept", "skos:exactMatch": [ "https://pid.phaidra.org/vocabulary/PYRE-RAWJ" ], "skos:prefLabel": [ { "@language": "eng", "@value": "other" } ] } ], So the LoD URL is https://pid.phaidra.org/vocabulary/PYRE-RAWJ. That is the canonical identifier for an "authority record". If you follow that link, you will see that at that canonical URL there are "Links to other vocabularies", which includes http://purl.org/coar/resource_type/c_1843 and then that includes the concept rendered in many different languages. Now in terms of denormalisation, they've included the "eng" label here. So they can use the English label easily without needing to dereference the LoD links. Note on the HTML UI you do not see any links to http://purl.org/coar/resource_type/c_1843. All you see is that English label "other" linking to https://pid.phaidra.org/vocabulary/PYRE-RAWJ Does that make sense? That's how Linked Open Data systems work in the real world. So when you have a $0 https://mykoha/authority/1 it links that bib record to that authority record. Someone (or something if it's a bot) can then go to that authority record and go "Oh it's equivalent to this VIAF record or this WikiData record". That's what the Semantic Web is all about. Or rather having machine-readable machine-understandable linkages using canonical URLs to link objects with semantic meaning is what the Semantic Web is all about. The idea is to provide URLs to metadata in structured formats that machines can understand and navigate to connect related content in automated ways.

Happy to keep working toward a form that's also acceptable to you - a system preference layered on top, for instance.

For what it's worth, I'm not trying to be a gatekeeper. If Jonathan wanted to pass QA and Pedro as release manager wanted to push it to main, I could not and would not stop them. Rather, I'm trying to bring all my knowledge and experience in cataloguing and linked open data to this discussion. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #53 from David Cook <dcook@prosentient.com.au> --- After I wrote that paper and presented at VALA 2018 on Linked Data, I had a phone call with the research division at OCLC. Back then, they used to publish JSON-LD for all WorldCat records which was findable on the Worldcat HTML pages, which was really cool. Unfortunately, OCLC couldn't really see much benefit for open Linked Data on the Worldcat website, and they eventually removed it. But maybe it's because VIAF (which is hosted by OCLC anyway) is a better source of Linked Data anyway. -- Personally, I think it would be awesome to see people in Koha cataloguing using $1 and pointing to VIAF, and then Koha could index the URLs and group together records locally using those external authorities. That would be a very powerful way of working with LinkedData. Bruno, now you might say that you agree and that's what you're trying to do, but I think there's a difference between cataloguing a bib record and pointing to an authority, and just blanket copying URLs from a local authority's 024$a. Anyway, I think that's it for me. I appreciate you taking the time to have this discussion with me. Please, do feel free to change the status back to "Signed Off", if you're happy moving forward. I won't mind. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22972 --- Comment #55 from David Cook <dcook@prosentient.com.au> --- Sorry, I was answering a query on the listserv, when I noticed that the Koha Plugin hook "get_valuebuilder" wasn't documented on https://wiki.koha-community.org/wiki/Koha_Plugin_Hooks#Implemented_hooks I've added some documentation for that now. I think that really could be a very good path forward for this one (and a good long term direction for Koha - not Koha Plugins per se but rather using Perl Modules whether Koha Core or Koha Plugin for handling individual field editors). -- You are receiving this mail because: You are watching all bug changes.