[Bug 41182] New: Sushi harvest should send a full date
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Bug ID: 41182 Summary: Sushi harvest should send a full date Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: ERM Assignee: jonathan.druart@gmail.com Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: jonathan.druart@gmail.com, jonathan.field@openfifth.co.uk, martin.renvoize@openfifth.co.uk, matt.blenkinsop@openfifth.co.uk, pedro.amorim@openfifth.co.uk A library attempting to get counter reports from Proquest found they were failing. In testing, I was able to narrow the problem down to the date range. We send: https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11&end_date=2025-10&customer_id={ID}&api_key={KEY} They expect: https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11-01&end_date=2025-10-31&customer_id={ID}&api_key={KEY}
From what I can tell in the standard: https://www.niso.org/sites/default/files/schema-files/sushi1_7_ReportRequest... Date is expected as yyyy-mm-dd
We should send the full date and not truncate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #1 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 189044 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=189044&action=edit Bug 41182: Pass 10 digit date to SUSHI -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #189044|0 |1 is obsolete| | --- Comment #2 from David Nind <david@davidnind.com> --- Created attachment 190713 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=190713&action=edit Bug 41182: Pass 10 digit date to SUSHI Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com Assignee|jonathan.druart@gmail.com |nick@bywatersolutions.com Text to go in the| |This updates how the date release notes| |is sent for SUSHI harvest | |requests, as it was failing | |for Proquest counter | |requests. The date is now | |sent in full as 10 digits | |(YYYY-MM-DD), instead of 7 | |digits (YYYY-MM). | | | |Example: | |- Previously sent as: | |https://sushi.proquest.com/ | |counter/r5/reports/pr?begin | |_date=2024-11&end_date=2025 | |-10&customer_id={ID}&api_ke | |y={KEY} | |- Now sent as: | |https://sushi.proquest.com/ | |counter/r5/reports/pr?begin | |_date=2024-11-01&end_date=2 | |025-10-31&customer_id={ID}& | |api_key={KEY} --- Comment #3 from David Nind <david@davidnind.com> --- I've signed off, even though I don't have a way to test. Have updated the assignee. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Sushi harvest should send a |SUSHI harvest should send a |full date |full date -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #190713|0 |1 is obsolete| | --- Comment #4 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192041 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192041&action=edit Bug 41182: Pass 10 digit date to SUSHI These patches add a new dateformat field to erm_usage_data_providers to allow for choosing the format sent to sushi To test: 1 - Enable ERMModule 2 - Add a new Data providers 3 - Click 'Add manually' and add a provider without setting the 'Date format' field 4 - Confirm the provider is saved with dateformat 7 digit 5 - Edit and set the provider date format to 10 digit 6 - Confirm it saves correctly 7 - Edit and set to 7 and confirm saved 8 - Edit and clear the field, it saves as 7 9 - Run unit tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #5 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192042 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192042&action=edit Bug 41182: DO NOT PUSH - Schema updates -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #6 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192043 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192043&action=edit Bug 41182: Database changes -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #7 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192044 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192044&action=edit Bug 41182: Adjust interface for new field -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #8 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192045 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192045&action=edit Bug 41182: Unit tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #9 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Updated to allow for choice of date format - back to NSO -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |pedro.amorim@openfifth.co.u |y.org |k -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #192041|0 |1 is obsolete| | Attachment #192042|0 |1 is obsolete| | Attachment #192043|0 |1 is obsolete| | Attachment #192044|0 |1 is obsolete| | Attachment #192045|0 |1 is obsolete| | --- Comment #10 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192046 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192046&action=edit Bug 41182: Allow choosing format of date passed to SUSHI These patches add a new dateformat field to erm_usage_data_providers to allow for choosing the format sent to sushi The formats supported are: yyyy-mm yyyy-mm-dd yyyy-mm is the current assumption and remains the default To test: 0 - Apply patches, updatedatabase, yarn build, restart all 1 - Enable ERMModule 2 - Add a new Data providers 3 - Click 'Add manually' and add a provider without setting the 'Date format' field 4 - Confirm the provider is saved with dateformat 'YYYY-MM' 5 - Edit and set the provider date format to 'YYYY-MM-DD' 6 - Confirm it saves correctly 7 - Edit and set to 'YYYY-MM' and confirm saved 8 - Edit and clear the field, it remains as 'YYYY-MM' 9 - Edit and save as 'YYYY-MM-DD' 10 - Edit and clear the field and save 11 - Confirm it remains 'YYYY-MM-DD' 12 - Run unit tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #11 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192047 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192047&action=edit Bug 41182: Database changes -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #12 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192048 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192048&action=edit Bug 41182: Adjust interface for new field -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #13 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192049 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192049&action=edit Bug 41182: Unit tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #14 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 192050 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=192050&action=edit Bug 41182: DO NOT PUSH - Schema updates -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #15 from David Nind <david@davidnind.com> --- For me, for step 8, the date remains as YYYY-MM-DD (instead of YYYY-MM as stated in the test plan). Testing notes (using KTD): 1. To run tests: prove t/db_dependent/Koha/ERM/EUsage/UsageDataProvider.t -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |minor See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41306 --- Comment #16 from Michaela Sieber <michaela.sieber@kit.edu> --- Hi all, we can not confirm that the day is mandatory in COUNTER 5.1 which is the current release since January 2025 and obligatory for all providers. We harvested ProQuest data (R5.1) via the following Example-URL https://sushi.proquest.com/counter/r51/reports/tr?customer_id=1234567890&api_key=XYZ&begin_date=2025-02&end_date=2025-12 The COUNTER standard (R5 as well as R5.1) only requires months: "Reports MUST be provided monthly." "The reports MUST allow the report consumer the flexibility to specify a date range, in terms of months" (see https://cop5.countermetrics.org/en/5.1.0.1/05-delivery/index.html) We therefor recommend to adjust the date picker in the user interface to allow only months (not days) as suggested in Bug 41306 There is an issue with harvesting ProQuest R5 data. I change the importance to minor as R5 is not supported any more and ProQuest data for 2025 are available in R5.1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41182 --- Comment #17 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- I can confirm moving the site to use 5.1 with proquest now works -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org