June 2012 - Koha-devel - lists.koha-community.org

Koha API for use in mobile app
by Robert Nay 14 Jun '12

14 Jun '12

I am looking to create an iPhone/Android app for my local library, and am wondering about the best way to go about this. Does Koha have some API that can be accessed from a third-party client (in this case, a mobile application)? Koha pages seem to have an "unapi" link in them, is this something that could be used? I am relatively new to Koha, so any help or advice would be greatly appreciated. Thanks! Robert

2 1

Kohacon12: my pics
by Matthias Meusburger 14 Jun '12

14 Jun '12

Hello ! Here are the photos I've taken (and gathered) from Kohacon12: http://pub.veggie.fr/biblibre/pics And here is a zip file: http://pub.veggie.fr/biblibre/2012_06_05-11_Kohacon_Edinburgh.zip Cheers, Matthias Meusburger

2 1

Unit tests and continuous integration with Koha
by Claire Hernandez 13 Jun '12

13 Jun '12

Hello all, One of discussed subject at kohacon12 was quality control and automated tools: Chris presented us how works unit testing in koha and what does jenkins. I tried to summurise both subjects in these two pages: http://wiki.koha-community.org/wiki/Unit_Tests http://wiki.koha-community.org/wiki/Continuous_Integration Surely incomplete and with mistakes but have a look :) Claire;

1 0

Experimental support for MARC records as a proper datatype in the PostgreSQL database
by Marc Balmer 12 Jun '12

12 Jun '12

Sitting in one of the nice pubs in Edinburgh during KohaCon, I had this idea to add MARC records as a proper datatype to the PostgreSQL database server. After a discussion with Marc Véron and Dobrica Pavlinusic about what that could mean, I decided to just try it and I have now a basic implementation (or, more a proof of concept). So here is some information on this: If MARC records are a proper datatype, that means they are stored right in the database, are backed-up, can be restored, replicated etc. just with the standard database tools. If then a function is provided to access individual fields of a MARC record, then this can be used in SQL expressions, e.g. for selects or to create views etc. As PostgreSQL supports functional indexes, you can create indexes on individual MARC fields, giving you super fast access to your data. How does it look? There is a datatype called 'marc' for now: CREATE TABLE books ( id serial, sig varchar(16), marc_record marc ); MARC records are loaded into the database as raw, binary records, encoded in hexadecimal: INSERT INTO books (sig, marc_record) VALUES ('a01b', '3030383830.....'); To access individual MARC fields, the function 'marc_field()' was created, it returns VARCHAR: SELECT id, marc_field(marc_record, '020') AS isbn FROM books WHERE sig = 'a01b'; Of course MARC fields can be used to search data: SELECT SELECT id, marc_field(marc_record, '020') AS isbn FROM books WHERE marc_field(marc_record, '245') like 'whatever%'; An index on a specific field can easily be created: CREATE INDEX books_isbn_idx ON books (marc_field(marc_record, '020')); As a syntactic sugar, the expression marc_record@'020' is equal to marc_field(marc_record, '020'). Marijana Glavica kindly let me use a MARC database of ~250'000 records to make some tests, here are some real world examples: books=# \d test_marc Table "public.test_marc" Column | Type | Modifiers --------+---------+-------------------------------------------------------- id | integer | not null default nextval('test_marc_id_seq'::regclass) marc21 | marc | books=# select count(id) from test_marc; count -------- 246727 (1 row) How many croatian books do they have? books=# select count(id) from test_marc where substring(marc_field(marc21, '008'), 37, 3) = 'hrv'; count ------- 52582 (1 row) Of course much more complex queries are possible with this, and using the right indexes it is really, really fast. One query I tested when from 8.4 seconds to 0.21 ms with the right index. That's a speedup of 40'000. Thanks to Marijana, Dobrica, and Marc for feedback, interesting discussions and ideas! Feedback and suggestions are of course more than welcome. - Marc

2 3

Error message when deleting items
by David Schuster 11 Jun '12

11 Jun '12

I was using the batch delete in 3.6.5 - by barcode and received this error: Hints as to why? -- David Schuster Plano ISD Library Technology Coordinator

2 2

Authority control - request for comments
by Jared Camins-Esakov 11 Jun '12

11 Jun '12

Good time of day! I plan to do a great deal of work on the authority module, and I wanted to request that anyone with thoughts on the matter please share them with me. The RFC is on the wiki at http://wiki.koha-community.org/wiki/C_%26_P_Authority_Control_Improvements_… I have copied the description below. Description The development is divided into 14 parts: 1. A dropdown box will be added to the authority browser in the OPAC allowing the user to choose which index to search in authorities: the keyword index, the match heading index (i.e. all heading fields), the preferred heading index, or the main entry only index. (bug 8206) 2. Because importing large authority files can result in unused authority records overwhelming used authority records in OPAC authority browse results, some libraries may want to disable the display of authorities which are not used in the bibliographic database. In order to make this possible (while still retaining the existing behavior of displaying all records in the authority file as a default), a system preference OPACShowUnusedAuthorities will be created, and a "Show all results" link added to search results when it is enabled. (bug 8205) 3. Because see also references in authority records are intended to refer to other authority records, they really ought to be turned into hyperlinks in the various authority displays. They will be made into hyperlinks (using the authid in $9 if available, or a search string otherwise) in both the OPAC and the staff client. (bug 3462) 4. At present, the only authority record view available in the OPAC is an "expanded MARC" view which is of no use to patrons, and limited use to librarians. We propose adding a user-friendly authority details view similar in design to the bib details view. (bug 8204) 5. Although the batch import functionality was originally designed with importing both bibliographic and authority records in mind, authority import was never actually implemented. This will be done quite simply, by adding an option to choose which type of record to import in the Stage MARC records tool. (bug 2060) 6. Match points will also be extended for use with authority records, again completing a feature for which stubs already existed. (bug 7475) 7. There is no way to export authority records other than via a direct SQL query, which makes it difficult for hosted libraries to get their data out of Koha without the help of their support vendors. We propose to change the "Export bibliographic/holdings" tool to the "Export data" tool, and add a tab to enable librarians with the proper permissions for the export tool to export their authority records directly from the staff client. (bug 8202) 8. Some librarians have requested the ability to save individual authority records, so a "Save authority" button will be added next to the "Edit" button when viewing authority records. (bug 8203) 9. One of the great promises of DOM indexing in Zebra (and the adoption of solr as an indexing technology) is the ability to incorporate alternate forms of headings into bib records when exporting bibs for indexing. We will add the plumbing for arbitrary record filters, and implement a filter that checks each heading in bib records for alternate forms, and includes them prior to exporting the record for indexing. (bug 7417) 10. Instead of using only textual strings for see also links, the ability to use $9 in see also fields in authority records (5xx in MARC21, NORMARC, and UNIMARC) will be added, to simplify heading disambiguation. This will enable us to create a value builder plugin using the existing thesaurus functionality that will automatically populate the see also heading control fields with metadata about the link (broader term, narrower term, etc.) (bug 8207) 11. Right now when cataloging, catalogers have only two options for adding a new authority: turn on AutoGenerateAuthorities, and later edit the authority record to justify its creation, or save the bib record they are working on, create a new authority record in the authority module, wait for the new record to be indexed, and open the bib record back up to use the authority finder plugin. We intend to add a "Create authority" button to the authority finder plugin which will allow the cataloger to create an authority in a new window and (similar to fast add in circ) automatically fill out the selected heading field in the bib record. (bug 8208) 12. At times, people may want to search for terms that are related to the one they search for. For example, someone interested in books about Feet may also be interested in all books about specific parts of feet, including books about Toes, Heels, etc. We will add three new special pseudo-CCL search prefixes which will first search the authority file for the specified heading, then search not just for that term but also for broader terms (pseudo-CCL prefix su-br:), narrower terms (pseudo-CCL prefix su-na:), or all related terms (pseudo-CCL prefix su-rl:). (bug 8211) 13. Because patrons may not always realize that they want to search for a related term instead of the term they entered, we will also add a "Did you mean?" feature to the OPAC, which suggests terms related to authorities that match the user's search. (bug 8209) 14. Although not all authority records contain useful information, many do, and depending on the type of collection, access to that information can be critical to both librarians and patrons. In order to simplify looking up authorities referred to in particular records, a link from subject headings to their related authorities will be added to the OPAC. A patch which adds this to the "normal" mode bib details display has already Passed QA (bug 5888), so unless an unsolvable problem is found with that patch, implementation will focus on the XSLT view. (bug 8210) Any questions or comments are welcome. Regards, Jared Camins-Esakov -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins(a)cpbibliography.com (web) http://www.cpbibliography.com/

2 2

"In discussion" bugs
by Paul Poulain 11 Jun '12

11 Jun '12

Hello koha-devel, Looking at bugs that are "In discussion", I see that there are more than 50. When we added this status, we decided that a bug "in discussion" had to be discussed on the mailing list, it's not supposed to be a forever-lasting status. This is a reminder = bug reporter, patch author you're responsible for managing the discussion ! So I encourage all of you to go to check if you've a bug that is "in discussion" ! cheers -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

1 0

Koha:: namespace
by Jared Camins-Esakov 11 Jun '12

11 Jun '12

Good time of day (again!), I know Paul is going to be talking about the Koha:: namespace tomorrow, but since I can't be in KohaCon in person I wanted to share a few of my thoughts and maybe help him out with getting the conversation started. As part of the authority control improvements I'll be working on, I'm going to be doing a fair amount of code cleanup, which in many cases will involve moving the non-offending parts of the code to the Koha:: namespace (I don't know if this is an American expression, but there's a saying that you can't make a silk purse out of a sow's ear, which pretty much sums up how parts of C4:: are). On the plane on the way back from Lawrence I jotted down a few notes on how parts of the Koha:: namespace might look, and wanted to share them. I'm not sure if the ASCII graphics will come through in the e-mail, but this is also at the end of the authority control improvements RFC at http://wiki.koha-community.org/wiki/C_%26_P_Authority_Control_Improvements_… Any suggestions how to improve things (or, even better, consensus!) would be most welcome. + Koha::Cache - object-oriented class for caching data in Koha | +--- Koha::Cache::Fastmmap - mmap driver class for Koha caching | (implemented in bug 8092) | +--- Koha::Cache::Memcached - memcached driver class for Koha caching | (implemented by bug 7248) | +--- Koha::Cache::Memory - in-process driver class for Koha caching | (implemented in bug 8092) | +--- Koha::Cache::Memory - in-process driver class for Koha caching (implemented in bug 8092) + Koha::Calendar - object-oriented class for handling branch opening calendars & Koha::Context - exporter class for basic configuration information (note: a better option might be Koha::Koha as an object-oriented alternative to Koha::Context; C & P will implement the start of the latter, if that is the consensus of the community) & Koha::DateUtils - exporter shim class to ease migration to DateTime from date-only strings # Koha::Filter - virtual parent class for all record processor filters | +--* Koha::Filter::[metadata schema]::* - | object-oriented classes extending Koha::RecordProcessor::Base | which implement particular record processing functionalities \ +--- Koha::Filter::MARC::EmbedItems - filter for embedding items in | bib records for the indexing process | +--- Koha::Filter::MARC::EmbedSeeFromHeadings - filter for embedding see from headings in bib records for the indexing process + Koha::Heading - object-oriented class representing | authority-controlled headings | +--- Koha::Heading::MARC21 - object-oriented MARC21 heading handler for | Koha::Heading | +--- Koha::Heading::UNIMARC - object-oriented UNIMARC heading handler for Koha::Heading + Koha::Indexer::Utils - indexer utility functions (implemented by bug 7818) # Koha::Linker - virtual parent class for heading-authority linker | modules | +--* Koha::Linker::* - object-oriented linker modules for linking | headings to authorities \ +--- Koha::Linker::Default - default linker module | +--- Koha::Linker::FirstMatch - linker module that always selects | the first match it finds | +--- Koha::Linker::LastMatch - linker module that always selects the last match it finds # Koha::Record - virtual parent class for all types of records handled | by Koha | +--- Koha::Record::Biblio - object-oriented class representing | bibliographic records | +--+ Koha::Record::Authority - object-oriented class representing | | authority records | | | &--- Koha::Record::Authority::Handler::MARC21 - exporter class | | with MARC21-specific routines for Koha::Record::Authority | | (should be used only by Koha::Record::Authority) | | | &--- Koha::Record::Authority::Handler::UNIMARC - exporter class | with UNIMARC-specific routines for Koha::Record::Authority | (should be used only by Koha::Record::Authority) | +--* Koha::Record::Holdings - hypothetical object-oriented class | representing holdings records | +--- Koha::Record::Item - object-oriented class representing item records + Koha::RecordProcessor - object-oriented class for processing records using various filters & Koha::Search - exporter class offering access to searching | functionality via Koha::Search::Engine | +--+ Koha::Search::Engine - object-oriented class which dispatches | | calls into the specific search engine module | | | +--- Koha::Search::Engine::Solr - class which interfaces with solr | | for searching | | | +--- Koha::Search::Engine::Zebra - class which interfaces with Zebra | for searching | +--# Koha::Search::Plugin - virtual parent class for search plugins | | | +--- Koha::Search::Plugin::* - plugins implementing particular search | functionality | +--+ Koha::Search::Query - object-oriented class which generates queries | +--- Koha::Search::Query::Solr - class to generate Solr queries | +--- Koha::Search::Query::Zebra - class to generate Zebra queries + Koha::Template::Plugin::* - plugins for Template Toolkit & Koha::Utils - exporter class for utility functions required by many | parts of Koha (no dependencies other than Koha::Context) | &--- Koha::Utils::Authorities - exporter class for utility functions | related to authorities that do not act on an individual authority | &--- Koha::Utils::Biblios - exporter class for utility functions related to bibliographic records that do not act on an individual biblio Legend: + indicates an object-oriented class (or classes, in some cases) +--- indicates an object-oriented class that is further down in the hierarchy (may or may not be a subclass) +--+ indicates an object-oriented subclass that is extended # indicates a virtual object-oriented parent class +--# indicates a virtual object-oriented parent class which is further down in the hierarchy & indicates an exporter class &--- indicates an exporter subclass * indicates a hypothetical or example class \ indicates that the class(es) below this symbol provide (a) specific example(s) of the hypothetical or example class above the symbol NOTE: I do not propose to implement all of this as part of the authority project. C & P's authority development will create the following classes in the Koha:: namespace: + Koha::Filter + Koha::Filter::MARC::EmbedSeeFromHeadings + Koha::Heading::* + Koha::Record::Authority::* + Koha::RecordProcessor + Koha::Utils::Authorities (possibly) Regards, Jared -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins(a)cpbibliography.com (web) http://www.cpbibliography.com/

2 3

SIP brainstorming during the hackfest
by Paul Poulain 10 Jun '12

10 Jun '12

During the hackfest, we had a discussion/brainstorming around SIP server, here is a feedback of what we said Ppl participating the discussion: Marc B, Dobrica, Kyle Hall, Colin C, Paul P, Matthias M, Adrien S The SIP server: * is not very robust (fail badly if it receive a bad line) bug & patch already in bugzilla * the SIP protocol it's not complete: - fees are grouped, "pay 50", it should be "pay 4+3+5..." - atm, a patron can't pay a payment if there is no debt. That would be useful to pay registrations a few days before my account expire Marc Balmer goal is to be deal with that, but it's not in the SIP2 protocol (the protocol does not specify how to do it) * SIP3 = the new protocol is being developed, and addresses those problems. 3M has announced that SIP will be placed in the hand of a NPO and become a NISO protocol * We should write a new SIP server for Koha, dealing with SIP3. SIP2 will continue for a while, so we must have (and maintain) both servers. * our SIP2 implementation was shared with Evergreen, but it's no more the case, and Colin thinks we should not try to merge again, because we wouldn't gain anything specific. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

1 0

DOM indexing patch pushed (call for more tests !)
by Paul Poulain 09 Jun '12

09 Jun '12

Hello koha-devel, I just pushed bug 7818, that is related to DOM indexing. This bug changes nothing or changes a lot of things, depending on your situation. Anyway, it need a lot of extensive testing, it's a good thing that it's pushed more than 4 months before the next release. If you have a Koha already running and don't change your configuration, it shouldn't change anything. If you install a new Koha, by default, the configuration will be dom by default (you still can choose the old grs1 configuration) If you have a Koha already running and want to get the benefits from DOM indexing, head to the wiki page: http://wiki.koha-community.org/wiki/Switching_to_dom_indexing, use it (and report anything missing) What does DOM indexing change ? It gives improved ranking, a more modern way of indexing in zebra and a tiny speed improvement. Note that all what is described in bug 7818 has not been made, so if you read the initial description, don't be surprised. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08

1 0