[Bug 21079] New: Unify metadata schema across backends
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Bug ID: 21079 Summary: Unify metadata schema across backends Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: ILL Assignee: koha-bugs@lists.koha-community.org Reporter: andrew.isherwood@ptfs-europe.com Currently different ILL backends return metadata in disparate property names, for example, an article will have the property "./metadata/titleLevel/title" for it's containing journal title on the BLDSS backend, whereas the FreeForm backend will store it in "title". Some work was done by Martin in Bug 20640 that introduced the mapping of certain metadata properties to a common schema. For example, the property described above is mapped to "container_title". This mapping work currently only exists for the BLDSS backend. Work on Bug 21063 has highlighted the fact that these metadata properties vary according to the backend. This bug proposes extending the mapping to other backends. This will be carried out for the FreeForm backend by Andrew Isherwood. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |andrew.isherwood@ptfs-europ |ity.org |e.com Depends on| |7317, 20640 Blocks| |21063 Status|NEW |ASSIGNED Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7317 [Bug 7317] Add an Interlibrary Loan Module to Circulation and OPAC https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20640 [Bug 20640] Allow migrating a request between backends https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21063 [Bug 21063] Allow columns in intranet ILL request datatable to be customisable -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|master |unspecified --- Comment #1 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- It's more than likely this work won't touch core Koha, I just thought it was worth creating a bug to highlight that this will need to be done for any other backends in existence. I'll post details of what was done to the FreeForm backend once it is complete. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #2 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Created attachment 77035 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77035&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties: - container_title (this should correspond with 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #77035|0 |1 is obsolete| | --- Comment #3 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Comment on attachment 77035 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77035 Bug 21079: Unify metadata schema across backends Patch contains debugging code, obsoleting -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #4 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Created attachment 77036 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77036&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties: - container_title (this should correspond with 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #5 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- So this bug did touch Koha core after all. We need to transform existing article FreeForm request metadata in order to copy some properties to new metadata properties with names that correspond with our unified schema. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Bug 21079 depends on bug 7317, which changed state. Bug 7317 Summary: Add an Interlibrary Loan Module to Circulation and OPAC https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7317 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |20996 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20996 [Bug 20996] Fix API response time on ILL request endpoint -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #77036|0 |1 is obsolete| | --- Comment #6 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Created attachment 79720 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=79720&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties: - container_title (this should correspond with 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #79720|0 |1 is obsolete| | --- Comment #7 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Created attachment 79725 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=79725&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties (if their values were originally populated): - container_title (this should correspond with what *was* 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 PTFS Europe Sandboxes <sandboxes@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #79725|0 |1 is obsolete| | --- Comment #8 from PTFS Europe Sandboxes <sandboxes@ptfs-europe.com> --- Created attachment 80288 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=80288&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties (if their values were originally populated): - container_title (this should correspond with what *was* 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') Signed-off-by: Stephen Graham <s.graham4@herts.ac.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Depends on|7317 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7317 [Bug 7317] Add an Interlibrary Loan Module to Circulation and OPAC -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|20640 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20640 [Bug 20640] Allow migrating a request between backends -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |20640 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20640 [Bug 20640] Allow migrating a request between backends -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|20640 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20640 [Bug 20640] Allow migrating a request between backends -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Andrew Isherwood <andrew.isherwood@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|21063 |20640 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20640 [Bug 20640] Allow migrating a request between backends https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21063 [Bug 21063] Allow columns in intranet ILL request datatable to be customisable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Josef Moravec <josef.moravec@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Josef Moravec <josef.moravec@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #80288|0 |1 is obsolete| | --- Comment #9 from Josef Moravec <josef.moravec@gmail.com> --- Created attachment 81740 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81740&action=edit Bug 21079: Unify metadata schema across backends This patch contains a database upgrade that takes appropriate metadata properties relating to articles in the FreeForm backend and creates metadata that corresponds with the metadata being created by the BLDSS backend. This enables us to create templates that can display metadata equally across any backends that contain this metadata; To test: - Ensure you have at least one article request created with the FreeForm backend - Check the metadata for the request: => TEST: You should have properties such as 'article_title', 'article_author' populated => TEST: You should NOT have properties such as 'container_title' & 'pages' - Run the upgrade => TEST: For the same requests you should now have the following properties (if their values were originally populated): - container_title (this should correspond with what *was* 'title') - title (this should correspond with 'article_title') - pages (this should correspond with 'article_pages') - author (this should correspond with 'article_author') Signed-off-by: Stephen Graham <s.graham4@herts.ac.uk> Signed-off-by: Josef Moravec <josef.moravec@gmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Josef Moravec <josef.moravec@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |josef.moravec@gmail.com CC| |josef.moravec@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com --- Comment #10 from Nick Clemens <nick@bywatersolutions.com> --- This adds Try::Tiny to the updatedatabase, I think this is okay and there seems to be agreement polling the QA team. The AutoCommit and RaiseError flags need to be reset after the attempt. It appears the version is set even if the update fails, is this intentional? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #11 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- (In reply to Nick Clemens from comment #10)
This adds Try::Tiny to the updatedatabase, I think this is okay and there seems to be agreement polling the QA team.
There will have been some logic behind taking this approach, though I can't remember what it was now. Investigating, the common approach in the DBI community appears to be to "eval" the contents of what I've got inside my "try" block, then test for "$@" to determine an error. I'm happy to switch to that approach if it would make people happier?
The AutoCommit and RaiseError flags need to be reset after the attempt.
Good spot! Thanks.
It appears the version is set even if the update fails, is this intentional?
Nope, an error, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #12 from Andrew Isherwood <andrew.isherwood@ptfs-europe.com> --- Created attachment 82092 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82092&action=edit Bug 21079: (follow-up) Move from Try::Tiny to eval In line with similar examples already in Koha, we now eval the transaction, instead of putting it in a try/catch block. This patch also addressed the other comments made in https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079#c10 - AutoCommit & RaiseError are now reset - Don't set the version if the update fails -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 --- Comment #13 from Nick Clemens <nick@bywatersolutions.com> --- Awesome work all! Pushed to master for 18.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Bug 21079 depends on bug 20996, which changed state. Bug 20996 Summary: Fix API response time on ILL request endpoint https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20996 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21079 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |RESOLVED Resolution|--- |FIXED --- Comment #14 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- The enhancement will not be backported to 18.05.x series. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org