- Koha-devel - lists.koha-community.org

[Koha] Koha 3.22.19 security release
by Julian Maurice 31 Mar '17

31 Mar '17

The Koha community is proud to announce the release of Koha 3.22.19. This is a security release and contains several bugfixes. If you are running 3.22.x it is strongly recommended to upgrade. The full release notes are available at https://koha-community.org/koha-3-22-19-security-release/

1 0

Koha 16.11.06 released
by Katrin 31 Mar '17

31 Mar '17

The Koha community is proud to announce the release of Koha 16.11.06. This is a maintenance release of 16.11 and contains 9 bugfixes. The full release notes are available at https://koha-community.org/koha-16-11-06-release/ Packages will be available soon.

2 1

MySQL silently truncating strings due to 3 byte limit in 'utf8' charset
by David Cook 31 Mar '17

31 Mar '17

Hi all, I ran into a problem this afternoon where MySQL was silently truncating my UTF8 encoded string in biblio_metadata.metadata and biblio.notes. Admittedly, the string had some funny looking data*, but it was all valid UTF8. I had zero problems with XML::LibXML or MARC::Record. However, whenever I would use DBI or the mysql command line client, I’d get a truncated string**. This meant a completely invalid XML record which is irretrievable from the database using Koha’s XML handling methods. However, I eventually found https://bugs.mysql.com/bug.php?id=53005 where they state that the MySQL ‘utf8’ charset has a 3 byte limit (https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html) That tends to be fine in most cases, but 𝔤 is 4 bytes (f0 9d 94 a4 http://dev.networkerror.org/utf8/?start=120100 <http://dev.networkerror.org/utf8/?start=120100&end=120200&cols=4&show_uni_i…> &end=120200&cols=4&show_uni_int=on&show_uni_hex=on&show_html_ent=on&show_raw_hex=on&show_raw_bin=on). It was on this character that the truncation happened. Apparently, there is a 4 byte UTF-8 charset called utf8mb4: https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html. If you “SET NAMES = ‘utf8mb4’” for your client and set the charset for your database column, that will allow you to correctly insert the data. So the truncation doesn’t generate an error but it does generate a warning, which is much more obvious in the mysql CLI client than when using DBI: MariaDB [devkohadcook]> update biblio set abstract = 'We introduce a new perspective and a generalization of spectral networks for 4d N=2" style="position: relative;" tabindex="0" id="MathJax-Element-1-Frame" class="MathJax">=2 theories of class S" style="position: relative;" tabindex="0" id="MathJax-Element-2-Frame" class="MathJax"> associated to Lie algebras g=An" style="position: relative;" tabindex="0" id="MathJax-Element-3-Frame" class="MathJax">𝔤=An, Dn" style="position: relative;" tabindex="0" id="MathJax-Element-4-Frame" class="MathJax">Dn, E6" style="position: relative;" tabindex="0" id="MathJax-Element-5-Frame" class="MathJax">E6, and E7" style="position: relative;" tabindex="0" id="MathJax-Element-6-Frame" class="MathJax">E7. Spectral networks directly compute the BPS spectra of 2d theories on surface defects coupled to the 4d theories. A Lie algebraic interpretation of these spectra emerges naturally from our construction, leading to a new description of 2d-4d wall-crossing phenomena. Our construction also provides an efficient framework for the study of BPS spectra of the 4d theories. In addition, we consider novel types of surface defects associated with minuscule representations of g" style="position: relative;" tabindex="0" id="MathJax-Element-7-Frame" class="MathJax">𝔤.' where biblionumber = 50314; Query OK, 0 rows affected, 1 warning (0.00 sec) Rows matched: 1 Changed: 0 Warnings: 1 Because it doesn’t raise an exception, I’d say it’s silent. You have to check for it. Using mysql, you can send “SHOW WARNINGS” after the insert query and you can see something like: +---------+------+--------------------------------------------------------------------------------+ | Level | Code | Message | +---------+------+--------------------------------------------------------------------------------+ | Warning | 1366 | Incorrect string value: '\xF0\x9D\x94\xA4=A...' for column 'abstract' at row 1 | +---------+------+--------------------------------------------------------------------------------+ 1 row in set (0.01 sec) With DBI, it’s harder: http://stackoverflow.com/questions/7946655/dbi-perl-logging-mysql-warnings. Apparently you can use $dbh->{mysql_warning_count} to check for warnings, and then send “SHOW WARNINGS” via the same database handle and that should give you your warning result. Or you can run MySQL in “traditional” mode which treats warnings as errors. So this seems like an edgecase… it’s some wonky data (albeit valid UTF8 data) in an otherwise English record. However, according to MySQL (I haven’t verified for myself), Chinese, Japanese, and Korean sometimes use 4 bytes for characters. So people using these languages with Koha could be getting truncated data without realising that’s what is happening or why. I’ve tried out utf8mb4 and it works great. I think the downside is that it uses more space for certain types of columns because it needs to allocate more space for that 4th byte. I don’t know whether we should consider switching from utf8 to utf8mb4, so that we’re inclusive of CJK and comply with RFC3629 which states that there’s a 4 byte maximum. I think it’s worth discussion though. This wouldn’t just affect libraries in Asia. It would affect any library which stores records with CJK characters in a MARC 880 “Alternate Graphic Representation” field. I’m at the end of my Friday work day, so I’m going to leave it there, but food for thought! *Aforementioned strange data: We introduce a new perspective and a generalization of spectral networks for 4d N=2" style="position: relative;" tabindex="0" id="MathJax-Element-1-Frame" class="MathJax">=2 theories of class S" style="position: relative;" tabindex="0" id="MathJax-Element-2-Frame" class="MathJax"> associated to Lie algebras g=An" style="position: relative;" tabindex="0" id="MathJax-Element-3-Frame" class="MathJax">𝔤=An, Dn" style="position: relative;" tabindex="0" id="MathJax-Element-4-Frame" class="MathJax">Dn, E6" style="position: relative;" tabindex="0" id="MathJax-Element-5-Frame" class="MathJax">E6, and E7" style="position: relative;" tabindex="0" id="MathJax-Element-6-Frame" class="MathJax">E7. Spectral networks directly compute the BPS spectra of 2d theories on surface defects coupled to the 4d theories. A Lie algebraic interpretation of these spectra emerges naturally from our construction, leading to a new description of 2d-4d wall-crossing phenomena. Our construction also provides an efficient framework for the study of BPS spectra of the 4d theories. In addition, we consider novel types of surface defects associated with minuscule representations of g" style="position: relative;" tabindex="0" id="MathJax-Element-7-Frame" class="MathJax">𝔤. ** Truncated string: We introduce a new perspective and a generalization of spectral networks for 4d N=2" style="position: relative;" tabindex="0" id="MathJax-Element-1-Frame" class="MathJax">=2 theories of class S" style="position: relative;" tabindex="0" id="MathJax-Element-2-Frame" class="MathJax"> associated to Lie algebras g=An" style="position: relative;" tabindex="0" id="MathJax-Element-3-Frame" class="MathJax">𝔤 David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595

3 4

Development/ QA protocol question
by Cab Vinton 30 Mar '17

30 Mar '17

Hi, All -- A small group of small NH libraries has co-sponsored a small enhancement to get its feet wet in the Koha development world: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14365 As I understand it, once a patch has been signed off & passed on to QA, it can hit a bottleneck as there are limited number of QA Team Members. To make this as positive experience as possible for our informal consortium & to increase the chances of similar collaborations in the future, it would help if we can move through the development process as quickly as possible. So here's my question -- For sponsored (i.e., funded) projects, would it make sense to include funds to pay for the cost of a QA Team Member's time to review a patch? I'm thinking the financial commitment would be one way of boosting a patch's status in the queue. Hounding the developer is another option, of course, but that's unpleasant for both sides :-) I realize the QA person often, & perhaps preferably, will work for a different firm. And there may be other complicating factors I'm not aware of. Grateful for any thoughts from folks on the programming side of things. Cheers, Cab Vinton, Director Plaistow Public Library Plaistow, NH

5 16

Koha 3.22, Plack & EDS plugin
by Paul Poulain 27 Mar '17

27 Mar '17

Hello friends, We've deployed Koha 3.22, running Plack and the EDS plugin, and it appears that it does not work. We think it's because there are some .pl code that is out of plack scope, so it's not run. Is there anyone here who has some experience with koha / Plack / EDS ? (Frido will probably send a patch to Ebsco in the next days if there's no other ida) -- Paul Poulain, Associé-gérant / co-owner BibLibre, Services en logiciels libres pour les bibliothèques BibLibre, Open Source software and services for libraries

5 8

REST API object property naming
by Taskula Lari 25 Mar '17

25 Mar '17

Hello fellow Koha devs, As the REST API is moving forward, me and Tomás had a discussion on parameter and object property naming convention and how we should expose column names to API consumers. Since we have already renamed multiple objects in Koha (e.g. borrowers -> patrons, branches -> libraries etc), we should discuss naming convention in REST API and maintain a consistency throughout. For those who have not been following REST API development, I will quickly brief the current method that most REST API patches in Bugzilla have implemented: The endpoint name in most cases uses name of the corresponding Koha-object, for example /api/v1/patrons, /api/v1/holds, /api/v1/cities and so on. This is pretty clear. We then fetch one or more rows from the database via Koha-objects and simply convert them into JSON. The issue is that many of our database tables have inconsistency between column names and the Koha-object names - meaning that currently /api/v1/patrons returns a borrowernumber, /api/v1/holds a reserve_id and so on. The API user does not care how we internally call our objects and its parameters but only wants to see a clear and consistent naming convention. Therefore /api/v1/patrons should return a "patron_id", "library_id". Do we want this? What about implementation, how do we want Koha-objects to work with this? Koha::Object currently contains a TO_JSON method that could be suitable for outputting renamed keys. What about input when the keys are renamed, how do we want to convert them back? Any other comments or ideas? Lari Taskula Koha-Suomi Oy

4 4

Interlibrary loans module
by Alex Sassmannshausen 24 Mar '17

24 Mar '17

Hello Kohites! Andrew and I have just finished a second major revision of our proposed interlibrary loans module for Koha. The code and bug can be found at [https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7317] We believe the code has now reached the maturity it requires for wider engagement, and, more importantly, it's now reached the maturity that it should be easy to get it up and running relatively easily in your development environments. Unfortunately we cannot deploy the module using sandboxes because backends are implemented as separate code projects. And we would *love* your comments thoughts and concerns about the module if you are interested in ILL! The final comment on the bug contains basic documentation for the Module as a whole, and also some installation instructions. But for reference, I've attached the same document to this email too. Below you will find some further high-level background and a roadmap of what we would like to achieve. For now, if you're interested in this module: • have a look at the bug, and get involved in the discussion! • try to set up the module in your dev environment. It should work easily on dev boxes and dev installs that track master. • let us know if you face any issues! • start a conversation with us if you might be interested in creating a backend for your country's / organization's ILL workflows. Finally, if you don't have access to a development environment, but you would be interested in becoming involved in this project, get in touch and we might be able to provide you with access to a testing environment! Best regards, Alex Sassmannshausen PTFS Europe 1 High-level background ═══════════════════════ The ILLModule aims to provide a core framework against which different ILL workflows can be implemented within Koha. It achieves this by using 2 new tables as data store, and by using an extensible backend system to create 'connectors' to ILL providers. The data store consists of the illrequests and the illrequestattributes tables. The former is a traditional table that stores essential values associated with an ILL request. The later is a key/value store, linked to a row in the former. This store can be used to store arbitrary data provided by a backend. The backends implement highly customizable workflows for several core steps in the ILL management process. At the same time, each backend can extend the core steps (called defined as the `core_status_graph` in Koha/Illrequest.pm) with their own additional steps (defined as the `status_graph` in a backend's Base.pm). Each of these steps, both core and extensions, in turn can define any number of 'stages' required to complete each individual step. Each step has access to a template include file, which can dispatch on 'stage'. This is mirrored by each step having a sub in a backend's Base.pm, which once again can dispatch on 'stage'. The subs in Base.pm have access to the full data store provided by the ILLModule; similarly the template includes have full access to Koha template features, including access to custom JS blocks through which, for instance, external APIs can be called. The main aim of this Koha module was to provide a core that is comprehensive enough to store core data to only have to implement ILL once in Koha, whilst being extensible enough so that virtually any ILL workflow can be implemented against this core. We believe we have achieved this. I'd be very interested to hear from you if you believe you have a workflow that cannot be captured by this (obviously, third party tools that do not provide API access will be virtually impossible to seamlessly integrate into Koha). 2 Roadmap ═════════ The roadmap starts from the current release of code on the bugzilla issue. • Publication of mature beta level code (21 February 2017) ⁃ public testing ⁃ public discussion ⁃ dogfooding • Augment core functionality (~ June 2017) ⁃ add advanced configuration options for the core module ⁃ add a standard system for configuring backends ⁃ allow for an 'unmediated workflow' ⁃ finalise 'generic requests' workflow • Meet Koha QA standards (~ July 2017) • Add unit tests to core ILLModule • Ensure UI respects Koha standards • Ensure Core code passes qa scripts • Addition of Documentation ⁃ contributing a chapter to the Koha manual ⁃ ensure all pod information is correct and complete ⁃ ('writing your own backend' tutorial?) • Provide consistent error handling ⁃ standard means through which a backend can 'throw' an error. ⁃ replace uses of die with this standard route • Integration into Koha core in Koha 17.11

4 6

Introducing mana (and requesting mana-kb.koha-community.org)
by Paul Poulain 24 Mar '17

24 Mar '17

Hi, BibLibre is working on a project, called mana [1]. This project will try to build a global knowledge base for library data. It will be partly tighten to Koha, and partly not. For every data on Mana, we will provide micro-services (RESTFUL API) to : * query the kb and retrieve data * push data into the kb * let mana users declare a given data outdated or wrong. For now, mana handle: * subscriptions models: a weekly serial is weekly for everyone. With Mana, Koha users (and possibly other ILSes) will be able to quickly prepare their subscription management. * reports. This is 100% Koha tighten, because it's related to sharing SQL queries. But that will be very useful for all Koha libraries to share and quickly retrieve SQL they can run in a few click on their local DB. We are also developing an interface to clean mana-kb data. In the next weeks we will add social data: "give me an ISBN, I'll give you user reviews, user tags, user rankings". This will not be tighten to Koha at all, even if we will implement the mechanism within Koha. There are many many other data that could be stored in this KB: * kbart and ONIX-PL files for electronic resources * vendors/contact * "patron that read this also read that" * It could even include bibliographic data ? [although it would probably result in a huge DB, out of scope for now] * some specific Koha setups (libraries can add some tweaking to Koha, using JavaScript. We could have a DB of tweaks) I presented this project yesterday during the hackfest. People were quite enthusiastic ;-) The first bug should be ready to be signed-off in the next days (https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17047) BibLibre is willing to host the mana server, I propose to use a sub-domain of koha-community.org. Could we have mana-kb.koha-community.org and mana-test.koha-community.org defined and pointing to 91.121.55.79 ? That would be very helpful to signoff patches. Also note that we're already filling the database with subscriptions informations coming from our customers. If you want to add your existing data, drop me an email, I'll send you the SQL you must run on your Koha to create a CSV file that we can easily import into Mana-kb. [1] why mana ? It's a Maori word meaning "authoritative, valid, prestige" (http://maoridictionary.co.nz/search?keywords=mana) -- Paul Poulain, Associé-gérant / co-owner BibLibre, Services en logiciels libres pour les bibliothèques BibLibre, Open Source software and services for libraries

1 0

Proposal to give summer internship to students for working on Koha
by Dalton Fury 23 Mar '17

23 Mar '17

Hi, I am writing this email representing Central Library, National Institute of Technology, India. We migrated to Koha from a propitiatory software over an year ago, and are very happy with Koha. After moving to Koha, various departments have approached us so that they can use our Koha installation to mange their libraries. One obstacle we face is that Circulation Rules can be defined only based on Patron Categories. Our patrons are categorized as UG(Under graduate), PG(Post Graduate), PHD, STAFF, FACULTY etc. Sort1 and Sort2 fields are used to classify patrons into departments (Mechanical, Civil , CS, Electrical etc) and programs (B. Tech, MCA, BCA, MBA etc) respectively. Our patron categories are too broad. We need to define rules based on sort1 and sort2 fields as well. We are planning on taking a CS student as an intern during the summer vacation and get him to work on this feature. I am not a developer, and hence I want to get opinions from the koha developer community before I proceed with this. Some questions I have in mind are: Will the Koha community accept this change? Is it feasible? Can an undergrad student learn and implement this in 4 weeks of time? What is the difficulty level? I also have a high level objective of introducing a student to opensource contributions. If there is some other useful task that the intern can do for the community, please mention them too. Regards,

3 5

Koha Devel Meeting Moved - 2017-03-29, 13 UTC
by Nick Clemens 22 Mar '17

22 Mar '17

Hello All, With the hackfest going on today and nothing on our agenda currently I am going to suggest we move today's meeting to next week, March 29th, at 13 UTC. If you want to meet up and chat with hackfesters and others in IRC please do, but the official meeting will be next week :-) -Nick

1 0