[Bug 44050] New: ERM usage access_method from COUNTER data is not stored in erm_usage_mus
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44050 Bug ID: 44050 Summary: ERM usage access_method from COUNTER data is not stored in erm_usage_mus Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: ERM Assignee: koha-bugs@lists.koha-community.org Reporter: michaela.sieber@kit.edu QA Contact: testopia@bugs.koha-community.org CC: jonathan.druart@gmail.com, jonathan.field@openfifth.co.uk, martin.renvoize@openfifth.co.uk, matt.blenkinsop@openfifth.co.uk, pedro.amorim@openfifth.co.uk Target Milestone: --- Currently, the attribute Access_Method is missing from the schema: DESCRIBE erm_usage_mus or https://schema.koha-community.org/26_05/tables/erm_usage_mus.html Access_Method is not stored in the database and is therefore not taken into account when importing COUNTER files. Documentation: COUNTER 5 https://cop5.countermetrics.org/en/5.0.3/03-specifications/03-counter-report... COUNTER 5.1 https://cop5.countermetrics.org/en/5.1.1/03-specifications/03-counter-report... To reproduce: Upload a COUNTER file with access_method (Sample files see https://cop5.countermetrics.org/en/5.0.3/appendices/h-sample-counter-master-... ) 1.) enable ERM Module 2.) create a provider manually (no need for credentials) with report type TR 3.) Manual Upload of the file 4.) check that there is no access_method attribute in the database Additional information: Hier ist eine ausführlichere Version des Abschnitts "Additional information": --- Additional information: Since Bug 42218, only usage with `Access_Method = Regular` is imported. The reasoning behind this decision: 1. COUNTER reports can contain several rows for the same item and period that differ only in Access_Method. COUNTER defines two values: `Regular` (normal usage by human users) and `TDM` (usage for Text and Data Mining, typically automated and in bulk). 2. Access_Method is not stored in `erm_usage_mus`. After the import, rows with `Regular` and `TDM` would therefore look identical in the database: same title, same platform, same metric type, same month, with only the count differing. 3. This leads to incorrect statistics. If both kinds of rows were imported, they would be indistinguishable and could be summed or displayed side by side as if they were the same thing. TDM usage can be very high compared to regular usage, so the numbers would be inflated and misleading. COUNTER explicitly separates the two, and the Standard Views are filtered to `Regular` by default. 4. Importing only `Regular` was the safest interim solution. It guarantees that the stored numbers match what COUNTER defines as regular usage and what libraries expect to see in their reports. The drawback is that TDM usage is lost entirely at import and cannot be analyzed later. 5. The restriction is temporary. Once `Access_Method` is added to the database (this bug), the restriction to `Regular` can be lifted. All usage can then be imported, the two access methods can be told apart, and TDM usage can be tracked and reported separately. See Bug 42218, attachment "Import the Regular usage only". -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44050 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44050 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=42218 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44050 --- Comment #1 from Pedro Amorim (ammopt) <pedro.amorim@openfifth.co.uk> --- Hi all, Since bug 42218, yearly usage (erm_usage_yus) is no longer stored at import. It is rebuilt by Koha::ERM::EUsage::YearlyUsages->rebuild, which sums the monthly usage (erm_usage_mus) grouped by the monthly key columns. When this bug adds access_method to erm_usage_mus and lifts the Regular restriction, the yearly side needs the same treatment: 1) Add the access_method column to the erm_usage_yus table 2) Add it to the GROUP BY in YearlyUsages->rebuild (Koha/ERM/EUsage/YearlyUsages.pm) Without both, the yearly totals mix Regular and TDM usage. Also: 3) The database update should fill in access_method for the existing monthly and yearly rows (all Regular since bug 42218) 4) Add a test checking that every erm_usage_mus column is either grouped, summed or deliberately skipped in YearlyUsages->rebuild, so a future column can't be missed (e.g. it would fail if access_method was added to erm_usage_mus but not to the rebuild GROUP BY) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org