May 2005 - Koha-devel - lists.koha-community.org

Translation
by Joshua Ferraro 04 Aug '05

04 Aug '05

Hi all, OK ... this one's for the Web Site group and the Documentation Group. In bug 953, MJ requests that we setup a better translator's page complete with links to the latest .po files. Here's an example: http://audacity.sourceforge.net/community/translation Paul then pointed me to his site which is running on Kartouche, a communal translation tool: http://bureau.paulpoulain.com/kartouche/ Given the fact that many translators aren't going to be very tech savvy, I think we should make Koha translation web-based. That is, we should have a site where translators can go, setup a login, assign themselves a translation project, run the .po to get the English, and insert their language in text fields hitting submit when they are done ... so paul, can Kartouche facilitate this? Is there another software app that can? -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Koha ILS, Mambo Intranet, DiscrimiNet Filter jmf(a)liblime.com | Full Demos at http://liblime.com | 1(888)KohaILS

5 9

optimising the opac a little
by Gavin McCullagh 05 Jun '05

05 Jun '05

Hi, I'm a newbie here so please be patient with me. I'm going to open up straight away into a proposal, I hope people don't think this rude. If it can't be done, chances are you know why and I don't cause I'm new :-) I have a proposal which I'd like to suggest to optimise the Koha web based opac system. I'd like your opinions as to whether I'm right on track, doing redundant work or just plain off my rocker. It seems from a fairly cursory look that the opac system is basically constructing queries with LIKE '%xxx%' within them to interrogate the biblio table. This works fine of course but if those columns are unindexed this LIKE query seems (in my experience) a little slow. Suggestions to optimise: 1. Add FULLTEXT indexes to each of the searched columns (author,title,...) 2. Change the LIKE queries to use something like the following: SELECT ......, MATCH biblio.title AGAINST ('$titleSearchString' IN BOOLEAN MODE) AS TitleRelevance FROM biblio ..... WHERE ..... AND MATCH biblio.title AGAINST ('$titleSearchString' IN BOOLEAN MODE) ORDER BY TitleRelevance DESC The where clause does a word search of the title field which is very quick, as long as the indexes are in place. The TitleRelevance field in the select gives a match relevance so that you're best matches appear first (ORDER BY TitleRelevance DESC). I was mullign it over in my head and I suspect the way to combine two different column relevances is to multiply them. -- Reservations -- 1. FULLTEXT indexes and MATCH are in MySQL as of version 3.23.23. Is it likely people are using other databases which don't have FULLTEXT indexes? Is it likely people are using older versions of MySQL? http://dev.mysql.com/doc/mysql/en/fulltext-search.html http://www.databasejournal.com/features/mysql/article.php/1578331 http://jeremy.zawodny.com/blog/archives/000576.html 2. BOOLEAN MODE is a very recent addition. Indeed it's not in the MySQL package in Debian Woody. It is in sarge though (Mysql >v4.0.1). This is strictly optional for now as it just makes the searching more powerful (allowing use of wildcards, string removal, etc.). I'm not sure if an old version will ignore it or complain, but this could be left out for now. http://dev.mysql.com/doc/mysql/en/fulltext-boolean.html http://www.databasejournal.com/features/mysql/article.php/1587371 Most people don't use the boolean mode stuff, so it might be nice to have a random "tip" on the search page like: Tip: to search for partial strings use the * wildcard eg "Mac*" will match "MacCarthaigh". and a link to some advanced searching tips. 3. The addition of indexes on the biblio table will slow insertions/updates down. This is a price you pay for faster searching. In my experience it's worth the sacrifice. So, is this all useless nonsense? Should I crawl back under that rock? I'm willing to roll up my sleeves and get at it but I'd like to know this work is useful and it would help to have someone known in the community to keep an eye on the code. As I see it this is a mainly sql task and there shouldn't be too many changes to the perl code itself but I'm not so familiar with the greater part of the project code so I'd rather have someone to look over my shoulder now and then. Here is an example I've just done in PHP on MySQL with a similar database structure. There are ~20K records. http://simms.celt.dias.ie/~gavin/opac/ You can search the titles for 'gael*', the author for 'breatnach', keywords for 'royal' and 'ireland'. I have not yet implemented cross-linking. I'll take this down in a few days. Gavin

4 5

What Test::Harness & Test::More package do you have installed?
by Michael Lake 01 Jun '05

01 Jun '05

Hi all The Koha requirements are unclear on what versions or what authors Perl packages are required. For instance MARC-Record & HTML-Template need Test::More There is a package Test::Simple that contains Test::More by Michael G Schwern. This needs Test::Harness however. CPAN lists three Test::Harness packages: Test::Harness run perl standard test scripts with statistics POE-0.3101 - 19 May 2005 - Rocco Caputo Test::Harness Run Perl standard test scripts with statistics Test-Harness-2.48 - 22 Apr 2005 - Andy Lester Test::Harness run perl standard test scripts with statistics Net-ext-1.011 - 10 Apr 2002 - Spider Boardman My Question is: what is the author of the Test::Harness and Test::More packages that you are using? Mike -- Mike Lake Caver, Linux enthusiast and interested in anything technical. -- UTS CRICOS Provider Code: 00099F DISCLAIMER: This email message and any accompanying attachments may contain confidential information. If you are not the intended recipient, do not read, use, disseminate, distribute or copy this message or attachments. If you have received this message in error, please notify the sender immediately and delete this message. Any views expressed in this message are those of the individual sender, except where the sender expressly, and with authority, states them to be the views the University of Technology Sydney. Before opening any attachments, please check them for viruses and defects.

2 1

Circulation Lists ( gestion de périodiques )
by Luis Eduardo Samico 31 May '05

31 May '05

Hello everybody, First of all, I'm going to introduce myself: I'm a 22 years old student of ESIEE-Paris ( another ESIEE students have already worked with koha and proposed the Ldap authentication... ). Excuse me everybody if my question is destinated to user's list. So, I'm in a project that I have to choose and implement an Opensource solution for a Documentation Center where I work.. Actually they already have a SIGB but its "newer version" is very expensive, so ... Well, I've a lot of documentation about Koha, and I've found in one of them that Koha has a complete "periodics" management (like magazines). My problem is about the "periodics" circulation. I've already seen the circulation permission in the user flags, that allows the staff to work with a circulation desk. I've not found anything about circulation lists ( that's what my client needs ).... It would work like this: Every "periodic" biblio/biblioitem has a circultation list composed of Y users.... So, every time a new item ( exemplaire ) arrives, after being available to be borrowed, it goes to that Y users of these list for X days each user, where X is a delay to be defined by the administrator.... I'm sorry if Koha already has this features, but I've not managed to found it... If it doesn't, it would be a good suggestion for the newer version.... :). I've already some ideas about how it would be done in the database but, actually, how my project is for the end of July I think, unfortunatelly, I've not too much time to be an "active develloper" for koha. Sorry for my english that is not that good, but I think everybody could understand.... Thanks a lot.. Luis SAMICO Bonjour à tous, Je suis un élève en 4ème année de l'ESIEE-Paris et pour le moment je fais du stage sur migration de base de données. Mon doute c'est sur la gestion de périodiques. J'avais lu que koha, il avait un complète gestion de périodiques, mais je n'ai pas arrivé a trouver des listes de circulation ( j'en ai besoin... en effect, c'est a mon client).... Bon, la liste de circulation est un groupe des utilisateurs ( lecteurs ) associés à un périodique. Donc, dés que un exemplaire du périodique arrive, il reste quelques jours avec ceux-là ( les utilisaterus) avant d'être disponible à prêt. Alors, excuse moi si Koha a déjà c'est option mais moi, je n'ai pas trouvé.... s'il n'ai pas, ça peut être une bonne suggestion pour la prochaine version. J'ai déjà des idées mais comme c'est un stage de presque 2 mois,malheureusement, je n'aurai pas du temps de revenir un développeur koha.... Pardon pour mes erreurs de français... mais je pense qu'on peut comprendre quand même... ( en effect, je suis brésillien ... enchanté!!! ) Merci par avance Luis SAMICO --------------------------------- Découvrez le nouveau Yahoo! Mail : 1 Go d'espace de stockage pour vos mails, photos et vidéos ! Créez votre Yahoo! Mail

3 2

Bugsquashing Session Notes
by Joshua Ferraro 31 May '05

31 May '05

Hi everyone, We had a very successful bug squashing session today -- we have assigned, fixed or closed all the 9xx bugs (started wth 87 total bugs and now we have 77). Let's plan another meeting for early next week. Any time requests forward them to me please. Here's the summary of assigned/closed/fixed bugs: Paul: 986: checkmarc.pl should notice null frameworkcode 975: serials : renewal of a subscription 966: Attempting to add a biblio results in warning about possible duplicate 958: Field "notes" becomes empty at receipt 957: Bad calcul of the total in the order line after receipting 950: Issue limit set to zero 949: error when adding an acquisition wihtout budget 944: We need to be able to delete a subscription. 932: default theme is used for 'nohelp.tmpl' even if other template 938: If you delete notice without item it deletes notice linked (minor) 919: problems with subfield $0 of unimarc authorities structure (minor) 982: Internal Server Error (fixed) 981: opac-userdetails.pl crash (fixed) 961: Error message, when deleting an authority(fixed) 943: pb accentuations LDAP <=> KOHA(closed) 942: (closed) 933: Cancel process results in untemplated dialogue (fixed) 910: repeating a field (+), bring you back to the first page of m (fixed) HDL: 951: PDF::API2 mandatory in error 924: Database Update failed 2.2.0RC2 -> 2.2.0 (worksforme) Genji: 969: Overriding waiting item results in cancelled reserve (fixed) MJ: 984: Bug in installer appears to not correctly escape password 952: updatedatabase not checking whether table already populated Josh: 971: Incorrect use of term 'keyword' in searching template 953: Request for translation page 923: printer config in barcode generator gives error 987: Please create an "authority" component on bugzilla. 911: Intranet with safari (macintosh browser) 985: Deleting from OPAC Virtual Shelves crashes (closed) 967: In Katipo templates if acquistions set to normal, the links 925: Currency search doesn't appear to do anything 915: opac-MARCdetail.pl incorrectly shows request status Owen: 926: Many form tags lack method and/or action attributes 980, 978 (fixed) The following bugs were moved to 'enhancement': 976: duplicating a MARC field (with +) 917: plugins for coded fields in unimarc authorities 900: Unimarc Authorities -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Koha ILS, Mambo Intranet, DiscrimiNet Filter jmf(a)liblime.com | Full Demos at http://liblime.com | 1(888)KohaILS

1 0

whois JohnN ?
by Stephen Hedges 31 May '05

31 May '05

Does anybody know the "JohnN" who joined us for the IRC meeting on May 20? If you do, could you send me the info off-list? Thanks! -- Stephen Hedges Skemotah Solutions, USA www.skemotah.com -- shedges(a)skemotah.com

1 0

Re: [Koha-devel] inverted list Proof of Concept [quest for search]
by Paul POULAIN 30 May '05

30 May '05

Tümer Garip a écrit : > Dear Paul, (cc to koha-devel to this mail sent personnaly to me) > > Excellent idea. I wanted to write to you for a long time. Now I take > this opprtunity > I have already tried it and it looks it is definitely the way to go. > 1- I had a few problems which I thought I should share it with you. The > problem with the foreign characters lies in the fact that the perl code > changes everything to uppercase but offcourse not the accented > characters. So say a Turkish word 'çim' becomes 'çIM'. We also have say > 'ÇIM'. These two words produce identical index keys for MySql as it is > case insensitive. I know this, but it's just a proof of concept. > That is why you are getting SQL errors as Mysql is finding duplicating > keys and dropping some our your data. I have changed the PRIMARY index > on your table to INDEX and this solves the problem. nope. The correct way to go is to remove all accented characters. Seems we could do this with : use utf8; use Unicode::Normalize; $w = NFKD $w; $w =~ s/\pM//g; > 2- Is there a reason we are using 'title' as one of the keys of the > inverted file?. Are we going to use this 'title' anywhere else later on. > If not it is taking unnecessary space. I used bibid-biblionumber > combination and your code still works. yes, but you can't order the result by title without another sql query. That's why I think we need one table for each ordering we want to have with biblio-ORDERING_FIELD like : biblionumber-title, biblionumber-author, biblionumber-dewey... > If there is nothing that I am > missing I suggest we use bibid-biblionumber. This way we can reach the > MARC files or old Koha files directly if we need to at a later stage. > > 3- Another subject: > I have translated the NLP opac pages for 2.2.2b to Turkish. I do not > intend to translate intranet pages. I could not use the .po files as the > making of Turkish sentences is almost upside down. If there is any need > for these files I can e-mail them to you. Could be interesting, but the problem is that templates are changing, so you must use .po files. softwares like KDE or Gnucash uses .po files & are translated to turkish isn't it ? > > Good work, > > Tumer Garip > > NEU Library > Cyprus -- Paul POULAIN Consultant indépendant en logiciels libres responsable francophone de koha (SIGB libre http://www.koha-fr.org)

1 0

inverted list Proof of Concept [quest for search]
by Paul POULAIN 27 May '05

27 May '05

Hi, After some chat with Kados, I've spent some time to write a proof of concept for my inverted list idea. It's not perfect, but it works quite good. I've commited it in CVS a few minuts ago (misc/build_marc_Tword.pl and C4::SearchMarcTest.pm files) NPL should run it on their DB, that is really huge, and report how performances are. How it works : =============== * create the table marc_Tword with the following structure : CREATE TABLE `marc_Tword` ( `word` varchar(80) NOT NULL default '', `usedin` text NOT NULL, `tagsubfield` varchar(4) NOT NULL default '', PRIMARY KEY (`word`,`tagsubfield`) ) TYPE=MyISAM; * open a console & type export PERL5LIB & export KOHA_CONF as usual. * fill this table with misc/build_marc_Tword.pl. Warning, this script uses a very very consumming but very fast method to fill the table : it does everything in memory, then write everything. Another method is provided (& commented), but it's 100x times slower (really !) * open opac-search.pl and replace use C4::SearchMarc; by use C4::SearchMarcTest; as the API hasn't changed, it will work immediatly. * go to opac-search (advanced search) & search whatever you want. It Should work fine (except in "Any word" that is not implemented for instance) LIMITS : ========== * build_marc_Tword has problem with extended chars (accented ones mainly). So don't be afraid if you get sql errors. They are not a problem for a POC * search works always order by title, whatever you choose. * search works only search WORDA and WOARDB, not yet WORDA or WORDB or WORDA except WORDB. Due to structure, those search should cost exactly the same thing as others. you can test WORDA or WORDB very easily. in SearchMarcTest, at line 240, replace @result = keys %intersect; by @result = keys %union; Some infos on perfs/size : ============================ On my largest DB (1 900 000 lines in marc_subfield_table), the build_marc_Tword.pl script requires 60s for the 1st SQL read, then 120s for reading the whole table, then another 100s to write the table. It takes up to 350-400 MB of RAM, and at the end, the marc_Tword table is 245 000 lines, for 100MB of disk space. Note we could probably lower this number by checking better %ignore_list. Tests : ======== A search on a title word (socia*, with a *) that get 4500 results arrives in less than 2 seconds (mysql cache being empty). A search on another word (chan*) that get 590 results is the same time. A search on both words (socia* chan*) is 446 results and get the same duration. Doing the same test on a 2.2 install is VERY different : my SCSI hard disk does a lot of noise & the result for socia* chan* requires almost 20 seconds to appear. Testing with a OR (chan* OR socia*) get 4930 results, in probably less than 2 seconds. (Those results could probably be improved a little, as there are a lot of SQL queries to get item number & status & biblio subtitle.) Note that in the logs, you'll get something for every sql->execute (with the sql executed). You can see that most queries are not about the search itself, but about the item number & status. -- Paul POULAIN Consultant indépendant en logiciels libres responsable francophone de koha (SIGB libre http://www.koha-fr.org)

1 0

How we can input a very large field
by Andrés Tarallo 26 May '05

26 May '05

We are making some fixes into our production instalation. We noticed that the input field in cataloging only allows 255 characters and is an input field. We have currently resume fields that are much larger than that (about 3000 characters). We re trying to develop a way to make our templates aware of the fact that a field is large enough to be better input in a text field insted of an input field. We want to develop something more generic. An idea we have: having in the system preferences a list of fields that may contain large values. When we're about displaying an input field we should take into account the systems preferences. We will accept your fedback, we hope we can contribute this back to the CVS. Andres

4 6

Minutes of Website/Interface Design Meeting 2005-05-26
by Joshua Ferraro 26 May '05

26 May '05

Present at various times: NZ: rach, russ, irma GB: shaun and ben US: owen and kados ID: indradg 1/. koha.org Website a. Review of Russ's site map and ideas about _what_ koha.org is: 1. it's a place that appeals to those in libraries making decisions about library systems 2. it's a resource for developers b. we're going to put case studies up (DIY case studies like NPL as well as 'supported' case studies from software support providers: we'll have a framework for easily adding more case studies c. addressed tension between the koha.org site and the sites of companies who offer services around koha; we need to keep koha.org as vendor neutral as possible to attract more vendors d. "Showcase" section of koha.org will include links to 'official koha demos' as well as vendor demos (to showcase what vendor support can produce) e. We're going to have an official blog but for now we'll just aggrogate existing developer blogs from the homepage f. add links to presentations folks have done on Koha; as well as upcoming events where Koha will be showcased and links to arguments _for_ open source in libraries g. design process: rach and russ will rework the site map based on our meeting and will make some wireframes they will also. russ and shaun will get a 'content delivery schedule to divvy up the jobs. shaun will continue to work on his design. (Added later) Irma will proofread and do QA on the website. 2/. Koha Interfaces a. we will break up templates into 'programmer templates and 'official koha templates' b. programmer templates are just stripped-down templates for testing/demonstrating functionality c. 'official koha templates' are crafted using design principles: beauty, usability, standards compliance, accessibility (PDA, screen readers, etc.), color coordination, graphical appeal, etc. d. NPL templates will be used as design models, but default templates will be redesigned. e. there was some brief discussion of using kea for template design 3/. Roles a. rach and owen will be Interface Designers, rach will focus on design (graphically) and owen will do the coding; shaun and ben will work on an alternate set of templates and will contribute some improvements in wording, etc. b. LibLime can provide hosting for a staging server where the process can be observed c. Irma will do proofing and QA Complete log at: http://koha.org/cgi-bin/logs.pl?recall=recall&saved_query=Website%20Interfa… -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Koha ILS, Mambo Intranet, DiscrimiNet Filter jmf(a)liblime.com | Full Demos at http://liblime.com | 1(888)KohaILS

3 2