https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 Andreas Jonsson <andreas.jonsson@kreablo.se> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andreas.jonsson@kreablo.se --- Comment #1 from Andreas Jonsson <andreas.jonsson@kreablo.se> --- It would be nice to have one table for control fields and one table for data-fields. We could make it configurable which fields that should be indexed in these tables if storage space would be an issue. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- (In reply to Magnus Enger from comment #0)

Koha itself does not care too much about these fields, but sometimes they are used when importing/syncing in records from external sources, and then it can be work intensive to match on WHERE ExtractValue( metadata,\'//controlfield[@tag="001"]\' ). This would be much easier if the 001 and 003 fields from the record had their own database columns we could match on.

Similar to: Bug 11529 - Add subtitle, medium and part fields to biblio table

I'm trying to wrap my head around how we'd do this in practice. In theory, Koha should be resetting the 001 and 003, but it doesn't do that currently. I guess you'd be relying on that? Trying to match off the original 001 and 003 to improve the ease of matching records from external sources. I can see the logic there for sure. Personally, I wish that we'd move the 001 and 003 to the 035 during import, although that would present its own matching problems, plus the 035 is a repeatable field. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- But in any case... I think there's a difference between the Koha 001 and 003... and the original 001 and 003. I think we abuse the 001 and 003 in Koha, and I'm not sure we should develop functionality to further entrench that abuse. But I would be interested in a table that tracks record imports better, so that it would be easier to find a local Koha identifier (ie biblionumber / 999$c) that was created during an import using the 001 and 003 (or OAI-PMH identifier and baseUrl). However, how likely is it for source MARC data to have the 001 and 003 properly filled out? Looks like import_biblios has a control_number field, but cleanup_database.pl deletes all the records from the import_* tables periodically, so that's not useful. Anyway, I'll stop spamming now. I think that I'm probably just being too idealistic. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> ---

...

However, how likely is it for source MARC data to have the 001 and 003 properly filled out?

In the case of Sweden and Norway: very. And I bet the Germans do it properly too?

For libraries participating in a union catalog (a lot) 001 and 003 will be filled out. A lot of libraries not participating still use records from union catalogs in their cataloguing, so there is probably a high percentage there as well. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #8 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hm, using ExtractValue is slow, especially on big databases. As Magnus says in lot of use cases a quick access to 001 and 003 for matching is really helpful. We have started to map the biblioitems.ean to 001 for use in reports etc. as the 001 is so important to us. The column is indexed and not mapped by default in MARC21 - but it remains a bit of a workaround and I'd be happy about a real controlnumber column. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #8)

Hm, using ExtractValue is slow, especially on big databases. As Magnus says in lot of use cases a quick access to 001 and 003 for matching is really helpful. We have started to map the biblioitems.ean to 001 for use in reports etc. as the 001 is so important to us. The column is indexed and not mapped by default in MARC21 - but it remains a bit of a workaround and I'd be happy about a real controlnumber column.

Hmm, ok, I think I'm convinced. We're still living in a MARC centric world, so there's no reason not to add it. And I doubt Koha will ever use the 001 instead of the 999$c, so even if it isn't ideal from a MARC perspective, it does seem like a practical move. I know I hate using ExtractValue in databases in the hundreds of thousands or millions of records O_O. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@trust-box.at -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #11 from Mark Hofstetter <koha@trust-box.at> --- actually extractvalue is no problem if you do it like this - we have this working for libraries with 1M+ records alter table biblio_metadata add column controlnumber varchar(100) GENERATED ALWAYS AS (trim(extractvalue(metadata,'//controlfield[@tag="001"]'))) STORED create index controlnumber_ind on biblio_metadata (controlnumber); What we are doing at the moment is to implement OAI-PMH for a customer where the identifier is the controlnumber So in the OAI method of concern we'll simply look up the biblionumber for the given identifer (in our case the controlnumber) but 001+003 or 035a would also be trivial -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #13 from Mark Hofstetter <koha@trust-box.at> --- it helps insofar as you can address a record as quickly (databasewise) with the controlnumber column as with the biblionumber so for OAI you can easily a quickly map controlnumber to biblionumber and you are good to go (changing identifer url etc) but in the simplest case for our customer this works: my ($biblionumber) = $dbh->selectrow_array('select biblionumber from biblio_metadata where controlnumber = ?', {}, $controlnumber); (with error handling etc) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #15 from David Cook <dcook@prosentient.com.au> --- (In reply to Mark Hofstetter from comment #13)

it helps insofar as you can address a record as quickly (databasewise) with the controlnumber column as with the biblionumber

so for OAI you can easily a quickly map controlnumber to biblionumber and you are good to go (changing identifer url etc) but in the simplest case for our customer this works:

my ($biblionumber) = $dbh->selectrow_array('select biblionumber from biblio_metadata where controlnumber = ?', {}, $controlnumber);

(with error handling etc)

I'm sorry I still don't understand. Are you changing your OAI-PMH server code, or is this a custom OAI-PMH server, or something else? (In reply to Mark Hofstetter from comment #14)

if that was the question: it does not impact importing (I doubt you could measure the difference)

Magnus said something about importing but I did not understand. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- (In reply to Mark Hofstetter from comment #16)

I am aware that this not a proper patch which can be applied globally, but it's good enough (tm) for us at the moment

what we should be used imho

oai:description/oai_id:oai-identifier/oai_id:repositoryIdentifier

I don't understand what you're saying here.

is this to define which identifier is to be used in a given koha-instance

I still don't understand why it would matter though. The OAI-PMH record identifier just needs to be internally unique within the OAI-PMH server/repository. Technically, it can be arbitrary. I'm guessing you're trying to synchronize different systems using a control number which is meaningful in a third-party context? Without more information, I reckon this is not the direction we want to be heading with Koha. But I am curious about this use case. Realistically, we should be moving to UUIDs for Koha internal record identifiers. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25603 Mathieu Saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mathsabypro@gmail.com -- You are receiving this mail because: You are watching all bug changes.