- Koha-devel - lists.koha-community.org

Need to complete coding guidelines about when to use SQL and where.
by Victor Grousset/tuxayo 25 Jan '23

25 Jan '23

Hi :) We need some additions/changes to the SQL 8 guideline https://wiki.koha-community.org/wiki/Coding_Guidelines#SQL8:_SQL_code_in_.p… Like it allows implicitly SQL in .pm doesn't mention DBIC. In last meeting there was a discussion are here are some takeaways (https://ilbot3.kohaaloha.com/koha/2023-01-11#i_2343089) Add in the guidelines: No SQL by default in any place (in new code) except for database upgrades. But if it's not possible to do a query with the ORM or if it's much less readable or a performance bottleneck, then it should be done in a custom subroutine in the relevant DBIC schema class.[1][2] If SQL needed: SQL::Abstract syntax should be used over raw SQL in the /Koha namespace.[3] Syntax: https://metacpan.org/pod/SQL::Abstract::Classic#WHERE-CLAUSES What about command-line scripts? Should new ones/major changes use the ORM by default? Is that easy to access it from a script? SQL8: "SQL is also tolerated for reports" even new report? (assuming that still happens) [1] need more feedback on that last part. [2] Is DBIC schema class the right term? [3] why not everywhere? (except db upgrades) related to the upcoming question about scrips -- Victor Grousset/tuxayo

1 0

Next dev meeting 25 January: errata on hours
by Victor Grousset/tuxayo 25 Jan '23

25 Jan '23

I confused UTC with my local time! :o So it's this: Australia/Sydney Thu Jan 26 10:00:00 2023 AEDT NZ Thu Jan 26 12:00:00 2023 NZDT Pacific/Tahiti Wed Jan 25 13:00:00 2023 -10 America/Los_Angeles Wed Jan 25 15:00:00 2023 PST US/Central Wed Jan 25 17:00:00 2023 CST America/Lima Wed Jan 25 18:00:00 2023 -05 America/New_York Wed Jan 25 18:00:00 2023 EST UTC Wed Jan 25 23:00:00 2023 UTC -- Victor Grousset/tuxayo

1 0

perl modules to upload to debian.org
by Mason James 25 Jan '23

25 Jan '23

hi folks ping me if you have any perl modules that you want uploaded to debian.org ...perhaps excluding koha-common, at this stage :0)

1 0

Next dev meeting 25 January, 23:00 UTC (9 am east Australia, 11:00 Aotearoa/NZ, 12:00 Tahiti, 14:00 US west)
by Victor Grousset/tuxayo 25 Jan '23

25 Jan '23

Hi :) Careful with the International Date Line, it's the 26th for on the west side. https://wiki.koha-community.org/wiki/Next_IRC_meetings Calendar: https://koha-community.org/calendar/ Cheers, -- Victor Grousset/tuxayo

1 0

Re: [Koha-devel] [Koha] OAI-PMH harvester
by Arthur Suzuki 23 Jan '23

23 Jan '23

Hello there, If I may suggest a good harvester library, Catmandu may do the job pretty well. I've not used the OAI module but used it to harvest from a JSON source and transform to an UNIMARC file with pretty good success so far. It can export seamlessly to iso2709 or marcxml. https://metacpan.org/dist/Catmandu-OAI Best, Arthur On 2022-11-22 15:57, Mike D. wrote: > Hey. Hey, > I'm really glad to see the OAI-PMH harvester debate going on for Koha. > I > think if we choose a good external harvester with support, we can save > a > lot of energy and resources to implement related activities in the > system. > Shoveling the logs is only part of the story. The easy part. Since the > result of shoveling is a lot of records, most of the time we can't > avoid > post-processing, merging with the records in the local database. For > example, if you need to update records from a source where there are > millions of records, but there are hundreds of thousands in the local > database. Only a slice of that huge amount is relevant. If we design > the > processing workflow wrong, it will take unnecessarily long and burn > valuable resources. > I would hereby like to invite us to be in touch, to debate and share > our > experiences. Let's get this area moving towards a successful finish. > > Take care. > > Michal > > út 22. 11. 2022 v 15:13 odesílatel BOUIS Sonia > <sonia.bouis(a)univ-lyon3.fr> > napsal: > >> Hi, >> Thanks to David, Tomas, Michal and Michael for your replies. >> >> So we have decided to evaluate several external OAI-PMH client that >> could >> be used by Koha and to choose one in the end of January >> There a lot to do after that and we discussed about the background >> jobs >> and cronjobs seems to be appropriate. We thought that the settings in >> the >> koha intranet should be only to define URLs, SETs, or XSLT sheets (for >> example, to transform DC XML in MARCXML). >> >> We are only at the begining of the process 😊 >> >> Kind regards, >> Sonia >> >> ------------------------------ >> >> Message: 2 >> Date: Wed, 26 Oct 2022 10:37:49 +1100 >> From: "David Cook" <dcook(a)prosentient.com.au> >> To: "'Tomas Cohen Arazi'" <tomascohen(a)gmail.com>, "'BOUIS Sonia'" >> <sonia.bouis(a)univ-lyon3.fr> >> Cc: "'koha'" <koha(a)lists.katipo.co.nz>, "'koha-devel'" >> <koha-devel(a)lists.koha-community.org> >> Subject: Re: [Koha-devel] [Koha] OAI-PMH harvester >> Message-ID: <07af01d8e8ca$dfbddef0$9f399cd0$(a)prosentient.com.au> >> Content-Type: text/plain; charset="utf-8" >> >> Hi Sonia, >> >> >> >> I’m excited to hear that KohaLA would like to finance an OAI-PMH >> client in >> Koha! This functionality is always brewing in the back of my mind, >> since I >> first raised 10662 back in 2013. >> >> >> >> As Tomas says, I think that the background jobs are a key component >> for >> processing incoming OAI-PMH records. >> >> >> >> However, the ***missing component right now is the scheduling of the >> OAI-PMH harvesting tasks***, and I think this is where opinions get >> divided. Below, I’ll provide some history and opinions on Koha >> OAI-PMH. >> >> >> >> -- >> >> >> >> With 10662, the sponsored goal was for Koha library staff to schedule >> OAI-PMH harvests through the Web UI. However, Fridolin from BibLibre >> raised >> a point with me at Kohacon18 about how letting library staff control >> the >> timing of harvesting tasks could be a problem for support vendors. If >> too >> many libraries using the same public IP address tried to harvest from >> the >> same OAI-PMH repository, they could be rate limited or blocked. There >> could >> also be server load concerns. So there probably needs to be a balance >> between user configuration and system configuration. If I recall >> correctly, >> this is how DSpace’s OAI-PMH harvester works. Users set up targets and >> can >> start/stop harvests, but things like frequency and concurrency are >> handled >> by the system configuration. >> >> >> >> Based on my experience working on OAI-PMH on and off for nearly 10 >> years >> and as a Koha support vendor, I think my preference would be for >> sysadmins >> to handle most of the OAI-PMH harvesting details. >> >> >> >> The sponsorship for 10662 had certain requirements that many other >> libraries might not have, which is what made me think that it might be >> better to have an external client that connects to Koha. I thought >> maybe I >> could get the ordinary requirements pushed into Koha, and then handle >> extraordinary requirements externally. However, an external harvester >> won’t >> perform as fast as an internal harvester. (The compromise would be to >> write >> the harvester in such a way that people could provide different >> OAI-PMH >> harvester Perl modules that all stage records using the same core Koha >> modules.) >> >> >> >> Even then… the scheduling would depend on a library’s needs. Back in >> 2013, >> I had a Koha OAI-PMH harvester which worked as a cronjob. It would run >> each >> night. However, some libraries want to run OAI-PMH harvests as >> frequently >> as every 3 seconds. A cronjob’s smallest frequency is 60 seconds, so >> that >> wouldn’t work for that requirement. >> >> >> >> If a cronjob isn’t suitable, then I think you’d need a daemon created >> by a >> new command like “koha-oai --start <instance_name>”. It could read a >> configuration file and handle scheduling accordingly. With 10662, I >> used >> the POE module, because I knew it well and it has some timer tools for >> scheduling tasks. If I were to work on it again, I’d probably use >> Mojo::IOLoop instead these days, since Mojolicious is already part of >> Koha >> while POE is not. (That said, using modules like Mojo and POE are >> difficult, because they’re difficult to test using automation. That >> was one >> of the stumbling blocks with 10662. While the 10662 harvester worked >> very >> well, it was difficult to unit test. In hindsight, I should’ve written >> it >> in a way that was easier to unit test, but it had a lot of >> event-driven >> code which made things more difficult.) >> >> >> >> Another option would be to create a generic daemon for task scheduling >> in >> general (e.g. “koha-schedule”). Koha could use this for many things, >> but >> it’s a project in itself. >> >> >> >> -- >> >> >> >> The process of downloading OAI-PMH records and importing MARCXML into >> Koha >> is actually a fairly straightforward process. The difficulty is the >> task >> scheduling and management of tasks (and unit testing). >> >> >> >> I don’t know the answer that will make everyone happy. There’s lots of >> different ways of managing and scheduling the tasks. Based on my >> experience, I’d suggest targeting the simplest approach first, because >> complexity will make it less likely for the project to succeed. >> >> >> >> On that note, I’d be happy to test/QA any OAI-PMH harvester put >> forward. >> When I was writing OAI-PMH harvester patches, I found it really hard >> to get >> QA, so I’m happy to be that resource for someone else. I’ve spent a >> lot of >> time thinking about this topic, so happy to provide advice, warnings, >> emotional support 😉. >> >> >> >> David Cook >> >> Senior Software Engineer >> >> Prosentient Systems >> >> Suite 7.03 >> >> 6a Glen St >> >> Milsons Point NSW 2061 >> >> Australia >> >> >> >> Office: 02 9212 0899 >> >> Online: 02 8005 0595 >> >> >> >> From: Koha-devel <koha-devel-bounces(a)lists.koha-community.org> On >> Behalf >> Of Tomas Cohen Arazi >> Sent: Wednesday, 26 October 2022 3:46 AM >> To: BOUIS Sonia <sonia.bouis(a)univ-lyon3.fr> >> Cc: koha <koha(a)lists.katipo.co.nz>; koha-devel < >> koha-devel(a)lists.koha-community.org> >> Subject: Re: [Koha-devel] [Koha] OAI-PMH harvester >> >> >> >> I think with background jobs we have most of the framework that is >> needed >> to deal with this within Koha. >> >> >> >> Best regards >> >> >> >> El mar, 25 oct 2022 7:08, BOUIS Sonia <sonia.bouis(a)univ-lyon3.fr >> <mailto: >> sonia.bouis(a)univ-lyon3.fr> > escribió: >> >> Hi, >> KohaLA would like to finance an OAI-PMH client in Koha but, we have >> questions that we want to raise to the community. >> There was already tries to propose an OAI-PMH client : >> - https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662 : >> it's >> an old project that doesnt seem compatible with the current version of >> Koha >> - https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25905 : >> the >> scope is more to use an external OAI-PMH client and to connect it to >> Koha >> >> Our main question is about the way to handle this. Do you think that >> it's >> a better idea to use an external software or PERL routine and to find >> a way >> to connect it to Koha. Or would it be better to a new module in Koha >> from >> scratch and that Koha have his own OAI-PMH client. >> >> Please, let us hear your toughts about this projet. >> >> Kind regards >> >> Sonia >> >> Sonia BOUIS >> ------------------------------------------------------ >> Responsable du Service informatique documentaire Département d'Appui à >> la >> Recherche et aux Projets (DARP) Bibliothèques universitaires >> Université >> Jean Moulin Lyon 3 ADRESSE GÉOGRAPHIQUE > Manufacture des Tabacs | 6 >> cours >> Albert Thomas | LYON 8e ADRESSE POSTALE > Bibliothèque de la >> Manufacture | >> 1C avenue des Frères Lumière | CS 78242 - 69372 LYON CEDEX 08 >> >> Ligne directe : 33 (0)4 78 78 79 03 >> >> http://bu.univ-lyon3.fr<http://bu.univ-lyon3.fr/>| Suivez-nous > >> Facebook< >> https://www.facebook.com/bulyon3/> | >> Twitter<https://twitter.com/bulyon3>| >> Instagram<https://www.instagram.com/bu.lyon3/?hl=fr> >> >> _______________________________________________ >> >> Koha mailing list http://koha-community.org Koha(a)lists.katipo.co.nz >> <mailto:Koha@lists.katipo.co.nz> >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha >> >>

2 1

Incorrect "Progress" when importing MARC records in koha-testing-docker
by David Cook 23 Jan '23

23 Jan '23

Hi folks, Is anyone else having an incorrect value in "Progress" when importing MARC records in koha-testing-docker? I just uploaded 440 records but it says "Progress: 2680 / 440". -- As an aside, the MARCXML I'm using is the product of some MARCXML editing code I've written in just over 100 lines of HTML and Javascript. It does import jQuery for convenience, but everything else runs 100% locally. You upload it to the browser locally, use Javascript to make the modifications, and then download the MARCXML from your local browser. At some point, I want to write some PowerShell scripts to edit MARCXML as well, but I figure this is a bit more user-friendly. Right now, I've hard-coded the changes, but a person could make a nice UI to make it more flexible. The goal is to allow complex editing of MARCXML in environments where you can't install and run your own programming language interpreters or MarcEdit. I figure PowerShell is sometimes locked down in corporate environments, but browsers are often available and it has an embedded Javascript interpreter. Maybe some corporate environments block opening local HTML documents. I don't know. Anyway, It was a fun little experiment. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Adding "web pages" via Koha Plugins
by David Cook 23 Jan '23

23 Jan '23

Hi all, What do folk think about adding "web pages" via Koha Plugins? At the moment, we can inject REST API routes and serve static files, but we don't really have a way of creating web pages on the OPAC (or the Staff Interface) that can be interacted with by a user without plugin permissions. At the moment, there are a few options for providing this functionality. You could create a custom page via opac-page.pl and inject Javascript using "opac_js", but that involves some manual work from staff users. Alternatively, you could serve a HTML page from a REST API endpoint, but that's a corruption of the REST API. I suppose a Koha plugin could insert a page into the "additional_contents" table, although there doesn't seem to be a robust machine friendly way of detecting if the page was added by a plugin. Plus it could only be referenced using an integer ID rather than a plugin namespace or shortcode. Maybe the answer would be to add a column to the "additional_contents" table? Something like "source"? (Although it would need to be more than "plugin" so that different plugins aren't stepping on each other.) But I think it would be useful to be able to add a Perl-backed page via a Koha plugin in any case. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Error for "self_checkout" when trying to update borrower via "koha-mysql"
by David Cook 17 Jan '23

17 Jan '23

Hi all, Has anyone else seen the following error when trying to update via "koha-mysql" in koha-testing-docker? I can update borrowers via the Web UI but not via koha-mysql. I have no idea where this "self_checkout" value is coming from as well. -- koha-mysql kohadev Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 54 Server version: 10.5.17-MariaDB-1:10.5.17+maria~ubu2004 mariadb.org binary distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [koha_kohadev]> update borrowers set dateexpiry = '2031-04-04' where cardnumber = 1; ERROR 1292 (22007): Truncated incorrect DECIMAL value: 'self_checkout' David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

3 3

"Trident support team" unsubscribed
by Paul Poulain 16 Jan '23

16 Jan '23

Hello, We just removed "Trident support team" from subscribers to this list in order to avoid this continuous spamming. Sorry for the inconvenience. Happy day ! -- Paul Poulain, Associé-gérant / co-owner BibLibre, Services en logiciels libres pour les bibliothèques BibLibre, Open Source software and services for libraries

1 0

New request created with ID: ##19317## from koha-devel-request@lists.koha-community.org
by Trident Support Team 16 Jan '23

16 Jan '23

1 0