February 2018 - Koha-devel - lists.koha-community.org

Length of email columns
by Julian Maurice 17 Feb '18

17 Feb '18

Hi, In the database schema, we use different data types to store email adresses. Some are TEXT, some are MEDIUMTEXT, and there is aqcontacts.email which is a VARCHAR(100). According to https://stackoverflow.com/a/574698 the maximum length of an email address is 254 characters, so VARCHAR(255) should be enough. Do you know if there is any reason why we use mostly TEXT and MEDIUMTEXT for these columns ? -- Julian Maurice <julian.maurice(a)biblibre.com> BibLibre

4 3

Call for News: February 2018 Newsletter
by kohanews 14 Feb '18

14 Feb '18

I'm collecting news for the February newsletter. Send anything noteworthy to: k o h a news AT gmail dot com News criteria: --------------------------- ** For events **: - Please include dates for past events. If I can't find dates I may not add it. - Announcements for future events with dates T.B.A. are fine ...Eg., Kohacon - For past events , **** one month back is the cut-off ****. * News items can be of any length. * Images are fine * Anything and everything Koha. * Submit by the 26th of the month. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. Thank you! -- Chad Roseburg Editor, Koha Community Newsletter

1 0

0000-00-00 is expired?
by Jonathan Druart 14 Feb '18

14 Feb '18

Hi devs, I have just caught that: Koha::Patron->is_expired returns 0 if $self->dateexpiry eq '0000-00-00' And there is a test I wrote for it on bug 17578 (commit 5a0a2ce584dec74808f471b0f92b2114c3d6029e , GetMemberDetails - Remove is_expired) But then we also have, in C4::Circulation::CanBookBeIssued: 752 if ( !defined $patron->dateexpiry || $patron->dateexpiry eq '0000-00-00') { 753 $issuingimpossible{EXPIRED} = 1; 754 } else { With a test: is( $issuingimpossible->{EXPIRED}, 1, 'The patron should be considered as expired if dateexpiry is 0000-00-00' ); So, my questions, do you have a very old Koha installations to search if there is borrowers.is_expired == "0000-00-00" in it? Do you remember if it meant expired or not at some point? I am trying to adapt our code to sql strict modes, and this value is no longer valid, we will need to update existing values. Cheers, Jonathan (yes, the same code should be used)

8 12

Update your devbox!
by Jonathan Druart 13 Feb '18

13 Feb '18

Hi devs, I have just pushed bug 12904 and 2 lines are added to the apache config files. You will need to update your misc4dev repo and reset_all You can also add these 2 lines to the config files: +RewriteRule ^(.*)_(.*).js$ $1.js [N,L] +RewriteRule ^(.*)_(.*).css$ $1.css [N,L] Otherwise you will get 404 for *.js and *.css files! Cheers, Jonathan

4 5

Hackfest in Marseille, last call ;)
by Paul Poulain 13 Feb '18

13 Feb '18

Hello everybody, The hackfest in Marseille start in just one month from now. It's still time to decide to come ! Already 50 registered people, coming from USA (4), Germany (2), Greece (3), Norway (1), Spain (1), Senegal (1), and France (others ;) ) We miss someone from Italy, Portugal, UK, Argentina, New Zealand, Australia and [enter your country here] :D the guys from reasonablegraph will be there, we have people motivated to work on Elastic Search. And of course a lot of other hacking & fun. And French cheese. -- Paul Poulain, Associé-gérant / co-owner BibLibre, Services en logiciels libres pour les bibliothèques BibLibre, Open Source software and services for libraries

1 0

Easy sign off Bug 20173
by David Cook 12 Feb '18

12 Feb '18

Hi all, I think I have an easy sign off on Bug 20173. I think I found some HTML files from 2003 that haven't been used in a very long time. I figure we may as well clean those out now. I think we all must just ignore these files after going "cd koha-tmpl". David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595

1 0

Plack : can't locate object method error...
by Philippe Blouin 09 Feb '18

09 Feb '18

Hi! I'm having issues with a freshly installed debian server on which I put a 17.05.06 koha. Plack just won't start. In the logs I had Error while loading /installdir/etc/plack.psgi: Can't load application from file "/gitdir/api/v1/app.pl": Can't locate object method "cache_dir" via package "Swagger2::SchemaValidator" at /usr/share/perl5/Swagger2.pm line 41. After futile attemps, I then "solved" it by upgrading Swagger2 (manually) to 0.89. It "solved" it, then appeared Error while loading /installdir/etc/plack.psgi: Can't load application from file "/gitdir/api/v1/app.pl": Can't locate object method "data" via package "Mojo::URL" at /usr/local/share/perl/5.20.2/Swagger2.pm line 47 Mojo and Mojolicious and everything else I check seem to be at their latest version. No patch from the community (for 17.11+) seem related. Of course, that doesn't happen on my other servers, but they are "old", so somewhere a .pm is different for sure. Any suggestion? -- Philippe Blouin, Responsable du développement informatique Tél. : (888) 604-2627 philippe.blouin(a)inLibro.com <mailto:philippe.blouin@inLibro.com> inLibro | pour esprit libre | www.inLibro.com <http://www.inLibro.com>

2 3

Re: [Koha-devel] Koha hackfest in Marseille & https://reasonablegraph.org/ platform
by David Cook 09 Feb '18

09 Feb '18

Hi Nikos, I hadn't heard of property graphs until I read your message; that sounds interesting. I'd just read about triplestores so far. I'd love to hear more about the inference engine as well. That sounds like it could have some long-term value in terms of automatic discovery and interconnection. I'm curious how it works for transforming ontologies as well. I suppose if it's all rule based it's just a matter of creating the rules. In regards to the dbpedia URI, displaying the external data to the front-end seems like it would be really slow. Say you have 30 URIs to external nodes in your RDF record, that is 30 HTTP requests just for 1 RDF record. Even if you performed them in parallel, that seems problematic. So I would think you'd want to import the external node's data and duplicate it in your system (this would also be helpful for indexing, no?). But then when you have that external data in your system, how do you know how to present it to the frontend? I was looking again at http://www.worldcat.org/title/good-omens-the-nice-and-accurate-prophecies-o…. In the Related Entities section, they have '<http://dbpedia.org/resource/New_York_City> schema:name "New York"'. That triple doesn't actually appear in dbpedia, so it must be one that OCLC created locally. That seems logical to me in a way... OCLC gets the benefit of an authoritative URI but the convenience of using a locally understood metadata schema. But that triple... it would have either had to be created manually or automatically using some reasoning (and since I don't actually see any string that just says "New York" in dbpedia... I really have to wonder). I'll check out that demo now... Do you have any way of showing the RDF for a record to a user? If I'm looking at http://humanities.reasonablegraph.org/archive/item/7961?dv=1, can I see the actual data used to generate the page? I'm intrigued by the line 'VIAF 54266158 (Personal) ⟶ Apostolidēs, Renos Eraklēs 1924-'. I've seen a number of Linked Data apps that link to VIAF, which seems such a typical use case for libraries. Clicking on the link I get taken to https://viaf.org/viaf/54266158/#Renos,_1924-2004. Now I think I can see an RDF version of this page... I just need to go to "Record Views" and click "RDF record": https://viaf.org/viaf/54266158/rdf.xml. OK... I see the string "Apostolidēs, Renos Eraklēs 1924-" has a skos:prefLabel predicate on VIAF. I assume that the RDF record in RG just has an object for https://viaf.org/viaf/54266158/#Renos,_1924-2004 and that you're dereferencing that URI and looking for that particular predicate? Where does that get defined in RG? Another example is http://data.nlg.gr/archive/item/316756?dv=1. I see that you provide RG to the National Library of Greece too. (Side note: you have a typo in your "Powered by" section on http://data.nlg.gr/archive/item/316756?dv=1) Now I see that the HTML version is XHTML+RDFa 1.0... but I'm not seeing any RDF in the HTML. OK going into the editor... I see ea:link:VIAF_Number... ahh you're not storing it as RDF. This "Data Model" or "Tree" section is an expression of your property graph I guess. Is it that you have some rule that takes any ea:link:VIAF_Number and generates a URI for it and then you use rules to infer what data you need from that URL in order to populate your own graph? It looks like you're storing a combination of plain text strings, JSON strings, and some sort of prefixed JSON I've never seen before? So I see "Person - Link" in the editor. I try putting in 90705830 (the LC number for Huckleberry Finn) but nothing happens yet... and now I see "Your submission for the updated record is being processed in the background. The updated record will be refreshed in a few moments.". That makes sense. Hmm... adding the Lccn did nothing. I'm guessing I would have had to include the URI too? But that just seems like a regular URL. Of course in the case of VIAF you just have to ask for RDF to get RDF instead of HTML (e.g. curl -H "Accept: application/rdf+xml" https://viaf.org/viaf/54266158/) On http://humanities.reasonablegraph.org/archive/item/7961?dv=1 I see an "Occupation", and in the editor I see "Field of Activity" under "Person - Local Fields". The labels in the editor and the OPAC don't seem to match up. The editor only lets you link to locally stored concepts... which is OK because otherwise it could be a mess. I see you can add new concepts. It looks like a very manual process though and that the links aren't necessarily RDF links. What schema is http://humanities.reasonablegraph.org using for its data? I'm guessing something unique to RG? Or at least locally definable? Does that serialize as RDF? I'm guessing not. But when you say that you implement FRBR and BIBFRAME... all you really need to do is implement the model and since they're graphs I suppose you can do it however you like internally. Can people link to RG and get back RDF? I just tried 'curl -H "Accept: application/rdf+xml" http://humanities.reasonablegraph.org/archive/item/7963' and that failed, so I'm guessing not? In terms of collaborating with Koha, what sort of integration is available? Apologies for the stream of consciousness. RG does seem like a generic graph-based web solution that could be used in GLAM, but I haven't seen anything yet that suggests it is capable of supporting Linked Data or even RDF. I imagine I'm missing something in the demo? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -----Original Message----- From: Nikos Papazis [mailto:papazis@altsol.gr] Sent: Thursday, 8 February 2018 11:26 PM To: David Cook <dcook(a)prosentient.com.au>; 'Paul Poulain' <paul.poulain(a)biblibre.com>; koha-devel(a)lists.koha-community.org Subject: Re: [Koha-devel] Koha hackfest in Marseille & https://reasonablegraph.org/ platform Dear David, RG models the data as a property graph. We have developed an abstraction layer on top of PostgreSQL, that implements a graph. Also we have implemented an inference engine, that can apply forward reasoning on the graph. So the dataset is modeled as a set of graph nodes with properties (for the enitites) and a set of graph arcs (for the relationships between the nodes/entities). The inference engine has 2 prime tasks: 1. to manage the graph and do housekeeping tasks on it and 2. to apply logic rules on the dataset and infer new relationships and/or nodes from the supplied data. There are separate sets of rules in the inference engine for transforming and exporting the local ontology to another one. For instance for transforming FRBR to MARC21 or to Europeana Data Model as RDF, etc. Regarding the linkage to external datasets, like dbpedia, we store the URI of the related external node. From that we could either import the external node's data and duplicate them in our system OR we could just display the external data to the frontend. Related to the above is the mechanism of copy cataloguing. For instance if an installation handles bibliographic records and is modeled under FRBR/RDA, we have an Z39.50 interface which lets the user pick a MARC21 record from an external catalogue, present it to the user and then copy it and map it to the related FRBR records of the work-expression-manifestation-item. One demo that is relevant and has significant volume of different types of material is http://humanities.reasonablegraph.org/ This demo holds both bibliographic and archival material. We have fields for storing the URL to external datasets. We do not import the external dataset. I am at your disposal for any assistance. regards, Nikos On 01/02/2018 12:13 πμ, David Cook wrote: > Thanks for your reply, Nikos. > > That certainly sounds interesting. I'd like to know more about the platform, although I won't be making it to the hackfest in Marseille this year. > > How does RG handle Linked Data relationships? Say I was entering in a place of publication and wanted to use dbpedia's entry for New York City. I would provide the URI but what happens after that? Do I add a local triple to describe that external resource using a schema supported by RG, or does it fetch the RDF document from the remote server and let me decide what to import or transform? > > If you can direct me to the appropriate demo, I'd love to look more at how your system handles Linked Data. I think I've gone as far as there is to go with RDF on its own, but curious how existing systems are handling the links to external resources. > > Cheers, > > David Cook > Systems Librarian > Prosentient Systems > 72/330 Wattle St > Ultimo, NSW 2007 > Australia > > Office: 02 9212 0899 > Direct: 02 8005 0595 > > > -----Original Message----- > From: Nikos Papazis [mailto:papazis@altsol.gr] > Sent: Thursday, 1 February 2018 1:01 AM > To: David Cook <dcook(a)prosentient.com.au>; 'Paul Poulain' > <paul.poulain(a)biblibre.com>; koha-devel(a)lists.koha-community.org > Subject: Re: [Koha-devel] Koha hackfest in Marseille & > https://reasonablegraph.org/ platform > > Dear David, > > regarding the github repo of ReasonableGraph (RG): until now the sole developers of the RG platform are us. There isn't any community yet. We do provide the platform with l-GPL license in our installation, which are listed on the platform's website and are located in Greece. So we do not allocate any resources in maintaining a public repo. If a community arises, we will switch our private repos to the public one and do there our commits. This is the reason why our github repo seems abandoned. > > We too have implemented BIBFRAME 2.0, among other standards. Primarily for proof of concept and for providing demo installations. Our purpose is to demonstrate the fact that different ontologies can be quite easily implemented on our platform. Which has configuration files for describing any ontology and the entry forms necessary to manage it. > For all our demo instalations, we provide full access to anyone who might be interested. > > Having said that, I come to your question regarding the use or possible value of RG for the KOHA ecosystem. > > We feel that RG can offer 2 valuable functions: 1) a real-time backend that transforms MARC21 or UNIMARC collections into FRBR or BIBFRAME datasets (or any other semantic model) 2) and the ability to simultaneously manage/annotate/enrich the collection on both standards. > > So it can act as the necessary tool for adopting the new standards. > Helping to keep the MARC expertise and tools and at the same time experiment with the new standards. > > And the same time it allows libraries to explore the possibility of having unified repositories for all their collections (bibliographic, archival, museum, musical, etc). > > The demos mentioned by Paul are showcasing the above. > > Thanx to David and Paul for their comments. > > Looking forward to meeting you in Koha hackfest in Marseille > > Regards, > Nikos > > PS: I am not a member of the koha-devel list yet. So please forward it > there for completeness > > On 31/01/2018 12:52 πμ, David Cook wrote: >> I'm skeptical of anything that claims to implement BIBFRAME for production, since BIBFRAME appears to still be an experimental work in progress (https://www.loc.gov/bibframe/faqs/#q09) That said, it does seem like folk like Ex Libris are trying to use BIBFRAME 2.0 in production (https://www.loc.gov/bibframe/implementation/register.html) I suppose every technology needs its early adopters. I can see why places like Oslo Public Library, the National Library of Sweden, and la Bibliothèque Nationale de France are using their own schemas though. >> >> The ReasonableGraph Github only seems to have 1 contributor and a handful of commits. Mostly from 1 year ago and 15 days ago. I wonder what is missing? I haven't had time to deep dive the code yet. >> https://github.com/reasonablegraph >> >> That said, I'm intrigued by anything that claims to use Linked Data. To date, I've mostly only seen systems that use RDF stored locally. That is, that only link internally and don't resolve external links. The most interesting example of Linked Data that I've actually found is with OCLC WorldCat: http://www.worldcat.org/title/good-omens-the-nice-and-accurate-prophecies-o…. If you open the "Linked Data" section, you can see that "New York" from the top of the webpage comes from this predicate and object: library:placeOfPublication <http://dbpedia.org/resource/New_York_City>. The interesting bit is if you go down to "Related Entities", you'll see 'schema:name "New York"'. I find that interesting because http://dbpedia.org/page/New_York_City doesn't have a predicate of schema:name. So, in theory, OCLC must be converting a triple from another vocabulary into Schema.org. Either that or they provide the link as the authoritative link an! > d ! >> then just add their own local triple saying "New York". I suppose that's a possibility too. But it kind of defeats the purpose of machine readable Linked Data. But Linked Data is hard since you never know what you're going to get back from the other end. >> >> At this point, what could Reasonable Graph offer for Koha? >> >> Apologies if that sounds rude at all. I've just been thinking a lot about RDF and Linked Data for the last couple of years. >> >> David Cook >> Systems Librarian >> Prosentient Systems >> 72/330 Wattle St >> Ultimo, NSW 2007 >> Australia >> >> Office: 02 9212 0899 >> Direct: 02 8005 0595 >> >> >> -----Original Message----- >> From: koha-devel-bounces(a)lists.koha-community.org >> [mailto:koha-devel-bounces@lists.koha-community.org] On Behalf Of >> Paul Poulain >> Sent: Tuesday, 30 January 2018 11:45 PM >> To: koha-devel(a)lists.koha-community.org >> Cc: Nikos Papazis <papazis(a)altsol.gr> >> Subject: [Koha-devel] Koha hackfest in Marseille & >> https://reasonablegraph.org/ platform >> >> Hi koha-devel, >> >> I had a hangout with Nikos (in cc:) , from Reasonablegraph. >> reasonablegraph is a platform (L-GPL) that is able to deal with MARC >> records, transform them into FRBR / Bibframe /linked data. They made >> some work with Koha >> (http://unioncatalog.reasonablegraph.org/?lang=en, >> click on "Search" top-left) >> >> At the end of Nikos demo, I suggested him to come to the hackfest for a day or two, in order to present reasonablegraph to anyone interested, and talk about any possible collaboration between Koha community and reasonablegraph. He will probably come. >> >> That's another very good reason to come to Marseille in March :D >> >> -- >> Paul Poulain, Associé-gérant / co-owner BibLibre, Services en >> logiciels libres pour les bibliothèques BibLibre, Open Source >> software and services for libraries >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel(a)lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ git : >> http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ >> >> >> >> > > > -- > AltSol team > mob: +30 6974471632 > office: +30 2109617148 > fax: +30 2109617148 > URL: http://www.altsol.gr > linkedIn: https://www.linkedin.com/in/npapazis > > > > -- AltSol team mob: +30 6974471632 office: +30 2109617148 fax: +30 2109617148 URL: http://www.altsol.gr linkedIn: https://www.linkedin.com/in/npapazis

1 0

What's on in koha-devel #17
by Jonathan Druart 08 Feb '18

08 Feb '18

Hello librarians and developers, The what's on in koha-devel #17 is available on the Koha website: https://koha-community.org/whats-on-in-koha-devel-17/ Regards, Jonathan

1 0

Last bit for utf8mb4
by Tomas Cohen Arazi 08 Feb '18

08 Feb '18

Hi everyone, bug 18336, implementing utf8mb4 encoding [1] is now failed QA (should probably be IN DISCUSSION). The reason for it to be failed is that the encoding change from utf8 into utf8mb4 alters all *text fields too (and I didn't notice it) and so there's a discrepancy between the updated schema and a fresh one. Fixing it seems easy: I just need to change all fields like this in kohastructure.sql: TEXT => MEDIUMTEXT MEDIUMTEXT => LONGTEXT As I understand it, *TEXT columns only store as part of the row the number of bytes stored (in the external object storage) for the object [2]. Given this, the changes described above means this size change: 2 bytes => 3 bytes 3 bytes => 4 bytes The move into utf8mb4 is needed, and is important for supporting supplemental characters (japanese, chinese, emojis, etc). Do you have another understanding on how this could impact (say) space? Do you think we should move forward with it? I think we should move forward and will provide a followup patch right now, but more opinions are needed here. Thanks in advance Tomás [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336 [2] https://dev.mysql.com/doc/refman/5.7/en/string-type-overview.html -- Tomás Cohen Arazi Theke Solutions (https://theke.io <http://theke.io/>) ✆ +54 9351 3513384 GPG: B2F3C15F

1 0