May 2006 - Koha-devel - lists.koha-community.org

Biblio.pm Choices IMPORTANT
by Joshua Ferraro 24 May '06

24 May '06

Hi folks, I've been looking at merging Biblio.pm from dev-week and HEAD and it looks like there are a few choices that need to be made ... I'd like to get your feedback on these. (BTW: Tumer, go ahead and commit all the Biblio.pm stuff you've got to HEAD, we'll deal with the conflicts after the fact ...) 1. in HEAD we have 'z3859_extended_services', a routine capable of handling all the Z3859 extended services offered by Zebra, as well as 'set_service_options', which sets the service options before calling z3859_extended_services. But in dev_week, it looks like that's handled with 'zebraop' and when zebraop fails it falls back on zebraopfiles ... So which of these routes shall we take, perhaps a combo of both approaches? 2. in HEAD we have no more bibid, is that going to cause any strange conflicts in dev_week code (didn't spot anything specific). 3. in HEAD we have 3 types of subs, REALXXX, NEWXXX, something_elseXXX ... can't we just have one type of sub? Any reason we can't just completely clean up this API once and for all? 4. char_decode (you knew it was coming :-). Tumer, can you check the mail I sent earlier about encoding and see if installing XML::SAX::Expat solves your problems with MARC::File::XML's handling of encoding? If we have to stick with char_decode, then so be it, but I'd really like to have a more standards-compliant way to handle encoding in both MARC21 and UNIMARC. What this means is that in addition to managing the proper encodings (to and from MARC-8 in MARC21, and to and from the appropriate encodings in UNIMARC), Koha should also handle the leader (for MARC21) and tag 100 (for UNIMARC), and properly set the encoding when all is said and done. Internally, I vote that we store everything as UTF-8 in all circumstances. That's all for now .. Cheers, -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Featuring Koha Open-Source ILS jmf(a)liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS

3 3

Navigation links in prog templates
by Chris Cormack 24 May '06

24 May '06

Hi All Bruno and I were talking and we were wondering if it is ok to add a general menu, maybe as an include, into the top of the prog templates. With just links to the main sections in koha, and a link back to the mainpage. Its quite easy to get a few pages into a section, and then have to hit the back button lots of times to get back to the main page so you can go somewhere else. Good idea, bad idea, no one cares? :) Chris -- Chris Cormack Programmer 027 4500 789 Katipo Communications Ltd chris(a)katipo.co.nz www.katipo.co.nz

5 4

Fixing my mistakes
by Chris Cormack 24 May '06

24 May '06

Hi all Just a quick thank you to Bruno for fixing my syntax errors that I snuck in when merging and adding version strings to some of the modules. Thanks Bruno, Ill try not to do it again :) Chris -- Chris Cormack Programmer 027 4500 789 Katipo Communications Ltd chris(a)katipo.co.nz www.katipo.co.nz

1 0

New Feature: Export in MARCXML
by Joshua Ferraro 23 May '06

23 May '06

Hi all, I've just committed a new command-line script for exporting bibliographic data from Koha to an ISO2709 _or_ MARCXML file. It currently handles MARC21 records correctly (leader management) and allows the user to specify the desired encoding of the resulting dump file (MARC-8 or UTF-8) as well as the export format (MARC or MARCXML). It needs some work to fully support UNIMARC, as soon as I have a spare moment I'll work on that ... OPTIONS: -format <format> MARC, MARCXML [MARC] -e <encoding> MARC-8 or UTF-8 [UTF-8] -ignoreerrors Ignore encoding errors -assumeunicode Assume Unicode when unsure of encoding -file <filename> Filename to store dump [koha.mrc] -h This file EXAMPLES: \$ ./export.pl --format=MARCXML --encoding=UTF-8 -file=2005-05-23-koha.mrc --ignoreerrors So now we can all boast about Koha's MARCXML export abilities :-) Cheers, -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Featuring Koha Open-Source ILS jmf(a)liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS

1 0

(no subject)
by paul.poulain＠free.fr 22 May '06

22 May '06

Hello all, we all agree that prog templates are for programmers basically. On devWeek, we investigated the possibility to have /includes/ directory as a system preference. That still sounds a good idea to me, i'll investigate it a little bit more later. For now, I would like to suggest a minor css property for 2 cases : - a message has to be clearly identified on the screen (like "the member card expire in 15 days", or "this book is reserved" in a book list, or many informations that we want the patron not to miss. - a message to warn the librarian that a problem has occured and the operation was impossible or failed for any reason (like "this barcode is unknown" or "this barcode already affected" in item adding, or "this book is reserved & can't be reserved again", or "this member is suspended or gna" ...) If programmers don't put a specific tag here, template designers will miss them for sure ! That's why I suggest to have 2 new properties : - warning - problem They will be useable in any tag (<div class="warning"> or <p class="problem"> ... Waiting for your comments -- Paul

3 2

Encoding Problems and Solutions (IMPORTANT!)
by Joshua Ferraro 20 May '06

20 May '06

Hi all, I've spent the better part of two days working on the encoding problems that several of the Koha developers have been experiencing and I think I've met with some success. First, software I'm running: Perl 5.8.4 XML::SAX 0.14 MARC::Record 2.0RC1 MARC::File::XML 0.83 MARC::Charset 0.95 There were two problems ... the first happens when you have MARC21 records with wrongly encoded characters. Let me be clear that I mean wrongly encoded by the MARC21 MARC-8 encoding. The LOC has defined all of the valid MARC-8 characters and mapped them to UTF-8 here: http://www.loc.gov/marc/specifications/codetables.xml (warning, very large file) This is the code table that MARC::Charset uses to convert MARC-8 encoded records into UTF-8 ... and that's what MARC::File::XML uses to convert a binary MARC file into MARCXML. By default, if MARC::Charset encounters a character that isn't in that code table, it drops the whole subfield and throws a warn like this: no mapping found at position 8 in Price : � 7.99; Inv.# B 476913; Date 06/03/98; Supplier : Dawson UK; Recd 20/03/98; Contents : 1. The problem : 1. Don't bargain over positions; 2. The method : 2. Separate the people from the problem; 3. Focus on interests, not positions; 4. Invent options for mutual gain; 5. Insist on using objective criteria; 3. Yes, but : 6. What if they are more powerful? 7. What if they won't play? 8. What if they use dirty tricks? 4. In conclusion; 5. Ten questions people ask about getting to yes; g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/local/share/perl/5.8.4/MARC/Charset.pm line 197. I don't know if you can see it or not, but before the 7.99 in the above dump is a \x9C character, which is an invalid MARC-8 character. The temporary solution is to add the following: use MARC::Charset; MARC::Charset->ignore_errors(1); to any script or module that you expect to encounter wrong encodings. This way, it will just drop the offending character rather than the whole subfield (it still throws the warn though). I'm not 100% happy with that solution, but I can deal with it until someone has better suggestion. The second problem is the more serious one ... several of us have tried to pass UTF-8 encoded XML records in to the new_from_xml() method and had the parser crash if there were 'combining characters' in the record. The problem seems to be with the PurePerl parser ... as soon as I installed XML::SAX::Expat it went away. I've attached a script to this email that you can use to test your system to make sure things are set up correctly. The script attempts to convert a binary MARC record in either UTF-8 or MARC-8 encoding, to MARC-XML (encoded as UTF-8) and then back to binary MARC (as UTF-8). You can test the first problem with the following record: http://liblime.com/public/badencoding.mrc And the second with this one: http://liblime.com/public/combiningchar.mrc Run the script like this: $ ./roundtrip.pl badencoding.mrc out.utf8.mrc dump Compare the original MARC record with the out.utf8.mrc ... the only difference should be in the encoding and possibly missing chars if your records had bad encoding. (you can edit roundtrip.pl to turn on/off the ignore_errors flag). And don't forget to install XML::SAX::Expat. So ... now that we've got the MARC21 encoding problems out of the way, we need to look at UNIMARC. Mike Rylander has already done some work on this but I think due to the second error above, we've not had success testing thusfar. Cheers, -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Featuring Koha Open-Source ILS jmf(a)liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS

1 0

NEW branch on rel_2_2 : authorities
by Henri-Damien LAURENT 16 May '06

16 May '06

Hi, I am working for a network which needs a pretty high standard stuff on authorities with multi hierarchical authorities and a good summary feature. We were planning to do it on head but I have a customer review on June, 29. and we fear authorities management in HEAD won't be fair enough by that time. But we still want Koha stable branches to remain stable. All the work I shall do will be on a new branch called hdl_auth. This will allow me to create two subs in Authorities called build_summary and build_hierarchy. Both will be called whenever an authority is changed and shall store datas into auth_header tables new fields. It seems to me that build a summary on the fly would be time consuming (and tümer seemed to agree to that principle) and building a hierarchy is a problem that requires many authorities_search for 1 item. So I will create a branch. I will propose you to test when development process is over. I shall have to get it work into version 3.0. Bonne journée. -- Henri-Damien LAURENT

2 1

smart bot on IRC
by Pierrick LE GALL 16 May '06

16 May '06

Hello koha-devel, This message is for chris but I'm sure it will be interesting for every #koha channel user... Chris, just as a reminder, can you install a "smart" bot on #koha. So that we could ask "smartbot, what about notforloan?" and the bot would tell all what he as registered concerning the loan prevention. Bye -- Pierrick LE GALL INEO media system

2 1

IBERMARC addition
by Damián Marqués Lluch 15 May '06

15 May '06

Last April, 24th I tried to join to this list and I posted a message. However, somehow, I got added to the sourceforge devel-list (apparently no longer in use) and probably my message was never read. I reproduce my message below. On the other hand, I'm not subscribed to the right mailing list, so I will not receive any reply... does anybody know how can I subscribe to the right mailing list? Dear all, I recently downloaded koha-2.2.5 from your website and started to play around with it. I'm not a librarian, I'm just a computer science guy who was looking a program to keep track of his personal library, photography and music CD collection. All in a single program. MARC format seems to be suitable. Since I'm spanish, I took the MARC21 structure as a basis and changed it to the IBERMARC format, the one in use by the Biblioteca Nacional de España (Spanish National Library). Furthermore, I also took the IBERMARC format for authority records and I feed it into the proper MySQL koha tables (auth*). Finally I made an spanish stopwords list. So, I've changed the koha-2.2.5.tar.gz file in my computer in the following way: added: koha-2.2.5/scripts/misc/marc_datas/ibermarc_es/structure_def.sql added: koha-2.2.5/scripts/misc/sql-datas/spanish/authorities_normal_ibermarc.sql added: koha-2.2.5/scripts/misc/sql-datas/spanish/authorities_normal_ibermarc.txt added: koha-2.2.5/scripts/misc/sql-datas/spanish/stopwords.sql added: koha-2.2.5/scripts/misc/sql-datas/spanish/stopwords.txt changed: koha-2.2.5/Install.pm The changes made to Install.pm allow the user to select option 3. IBERMARC so that the default framework is IBERMARC. All new files have been encoded as ISO-8859-1 in order for the spanish character set to be fully recognized (in particular Unicode UTF-8 was not working correctly in my system). Install.pm has been left Unicode UTF-8 All of this is working in my system, so I wonder if you could be interested in have a look to my changes. Maybe you could consider the posibility to add them to the official distribution after testing them. With kind regards, Damián

3 5

notforloan rule
by Pierrick LE GALL 15 May '06

15 May '06

Hi koha-devel, Working on C4::Reserves2 and associated scripts, I'm wondering what is the rule to say an item is not for loan? In the database, we have itemtypes.notforloan and items.notforloan. In the authorised values for category "STATUS" I have 5 values from 0 to 5 (with no 3). In my items table, most of my items have NULL as notforloan value. In my itemtypes table, notforloan is either "loanable" or "excluded from loan". Should I consider that if items.notforloan is null then the correct value is the itemtypes one? Bye -- Pierrick LE GALL INEO media system

2 2