https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #1)

Could you add Franklin Street into qa tools to stop new ones, Jonathan?

Wrong report, sorry -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=21708 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Adding a see also to 21708. It helps to insert the 003/024/035 fields for PIDs at the right position in the MARC record. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 106461 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106461&action=edit Bug 24544: Introduce Koha::PID::Controller with unit test This new module will facilitate inserting/updating PIDs in authority, biblio and item records. Test plan: Run t/db_dependent/Koha/PID/Controller.t Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 106463 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106463&action=edit DO NOT PUSH: Example plugins This patch contains a few example plugins for testing and illustrating the use of the various plugin hooks. Prerequisites: [1] Fill PID_Domain with http://[your_domain] Note: This domain is used for pid recognition. [2] Fill PID_Field with say 'auth:024,biblio:024,item'. In other words: enable PIDs for all categories and use 024. [3] Where we refer to (un)install plugin X below, we mean: perl -MKoha::Plugin::Test::PID::X -e"Koha::Plugin::Test::PID::X->install;" perl -MKoha::Plugin::Test::PID::X -e"Koha::Plugin::Test::PID::X->uninstall;" Replace X by the plugin name. Test plan: [1] Koha::Plugin::Test::PID::External with opac/svc/pid_test This part shows the use of pid_create_hook. [a] Install the External plugin. [b] Execute shell command: export PID_SERVICE=http://[your_server:opac_port]/cgi-bin/koha/svc/pid_test [c] Edit your first authority record, add a 024 with $a. Make sure that it fully includes your PID_Domain. [d] Run insert_persistent_id.pl -a -c -v -records 2 Should report back: records 2, updates 1. [e] Verify that your first authority was not modified again and that your second authority record contains a new 024. [2] Koha::Plugin::Test::PID::Orgcode This part shows all three hooks, one hook from External and two other hooks from Orgcode. For instance note the move from $a to $z in the insert hook here. [a] Keep the External plugin, also install the Orgcode plugin. [b] Replace PID_Field by biblio:035$a [c] Fill MARCOrgCode with some string. [d] Edit first biblio record. Add a 035 with $a containing orgcode but with another domain than PID_Domain. Edit the second biblio record. Add a 035 with PID_Domain in $a but without a prefixing orgcode. [e] Run insert_persistent_id.pl -a -c -v -updates 2 Should report back: records 2, updates 2 [f] Check that the first two biblio records contain a 035 with your orgcode in $a before PID_Domain and a new identifier. The first biblio should contain a 035 with $a and $z where $z contains the edit from step [d]. The second biblio record should also contain the 035 from step [d]. [g] Replace PID_Field by: item [h] Run insert_persistent_id.pl -a -c -v -updates 4 Should insert pid's in 4 item records. [3] Koha::Plugin::Test::PID::Cleanup This part shows a 'creative' use of get and insert in order to actually remove identifiers from a record, just illustrating the power of plugins. [a] Uninstall plugins External and Orgcode. [b] Install the Cleanup plugin. [c] Run insert_persistent_id.pl -a -c -v -updates 3 Verify that first three items were cleaned up. Fourth untouched. [d] Uninstall Cleanup. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25801 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- And if we do want to store the PID at the record level, I think it would be best to put that into the relational database (we could export it into the MARC at indexing/exporting time). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #12)

And if we do want to store the PID at the record level, I think it would be best to put that into the relational database (we could export it into the MARC at indexing/exporting time).

See ^^ -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #16 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #13)

See also comment9. I chose here to not add stuff directly to core Koha routines. But work my way thru the data with a cron job. Surely this would be a next step. Problem with a cataloguing plugin is that you just know the record number only after saving it. Although a PID generator might theoretically not need it, many implementations, including my own, do use it.

I could see the utility of that cronjob for you, but maybe not for all Koha users? That's a good point about the PID generator.

Not sure how much you saw from the patches, but this patch set provides an interface via plugins to an external PID service.

I skimmed through that code, but found it a bit difficult to read.

No Koha should not mint its own PIDs.

Ok excellent.

Formally the PID generator may have its own PID lookup table. We do not really care here. (My local generator does not, since it is based on a Koha identifier. Its result can be found with a Standard-identifier index in Koha or even another future ILS. In that way actually turning my ES or Zebra index into a PID lookup table..)

I'm not sure that I understand this part. So you use the Koha identifier to mint a PID with a local non-Koha generator, then you store that in the Koha record and index it. Your organisation resolver then forward to a local non-Koha resolver which then queries Zebra/ES to get the record that matches... I assume not a full URL but a partial path?

But even with a full resolver having its own table, I would still argue to save a copy of the PID in the MARC record too for optimization, while respecting the lookup table as authoritative. As a side note: Could you give me another example of vital data on biblio level that we do not store in MARC? Not meaning optimization or calculated aggregates etc.

I don't know what you mean by "vital data" in this case, but some standouts are biblio.frameworkcode, biblio.datecreated (debateable), biblio_metadata.format, biblio_metadata.schema. To be honest, I think that I see where you're coming from. In the past, I wanted to store OAI-PMH identifiers in the 024 field for imported MARC bibliographic records and then look them up with a Zebra index search, but then I realized was problematic for me. While those MARC fields exist, the data in those fields aren't descriptive metadata about the record. They're metadata about the metadata record. In my case, I ended up putting the OAI-PMH identifier in the relational database, as it was much more robust than putting it in the MARC record and indexing it into Zebra (especially since there can be a lag for index updates). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #17)

That's an interesting question. In my mind, a biblionumber represents a conceptual metadata entity. While the actual metadata can change, that entity is tied to the point in time that the biblionumber is created. While we use the MySQL/MariaDB autonumber for that at the moment, in theory we could use a UUID or some other generated number (like we do for the item barcode rather than the item number).

Yes, I think that creation time works best.

For me, the MARCXML isn't the bibliographic record, but rather a view of it. In the future, that bibliographic record might have a RDF/XML view as well. They're both describing the same bibliographic entity, and that entity is the thing that should have the PID. After all, HTML, MARCXML, RDF/XML, they're just different content types available via content negotiation, right?

Just as the biblionumber the persistent identifier belongs/points to a conceptual thing, not talking about where we store it. This thing can indeed be described in MARCXML, RDF etc. What would be wrong about showing the PID in such views just like we often include biblionumber? Finally, storing it in a MARC field (just as many save biblionumber in 001) or not (only exporting or displaying it) is an implementation thing/technical optimization? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #20 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Katrin Fischer from comment #19)

I was trying to follow the discussion here a bit, if I understand correctly, the difference between the PID and the fields you list would be that the PID is supposed to be exported via Z39.50/SRU etc. - so if not included in the MARC file, it would make things much more complicated?

Thx for asking. For me frameworkcode or metadata format are quite different from the persistent identifier. The frameworkcode is bound to the MARC view/representation. Which may not stay the only view. The reason we made metadata format. That field is repeatable and belongs to the corresponding view. The PID on the other hand is a unique identifier that does not belong to a particular view but imo c/should be shown in each view. MARC21 and UNIMARC also added fields for such identifiers (auth 024, unimarc 003, etc.) Although we could save the field outside MARC and just copy it in at export time or display time, I dont mind adding it in MARC as a technical optimization. See comment18. Strict formally, the PID has been generated outside Koha and can be dereferenced outside Koha. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #22 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> ---

From myself on July 20 on the dev mailing list: === What do you think? If your authority, biblio or item has a PID, should we save it in MARC too? Noting that MARC is just a representation or view of your data. We already have a biblio_metadata table that allows for more. What if we already had multiple views? If you can, you are welcome to contribute to that discussion on the bug report. ===

Note that this did not trigger any response or discussion sofar. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ian.bays@ptfs-europe.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106460|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106461|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106463|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24544 --- Comment #24 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #23)

It was heartwarming to see the amount of interest and feedback on this development ;) I decided to move things to local plugins. And will close this report now.

I empathize. I have had a few developments that have been dear to me but haven't received much interest/feedback from others, unfortunately. I hope that local plugins are able to provide an adequate solution for you. -- You are receiving this mail because: You are watching all bug changes.