[Koha-cvs] CVS: koha-doc/en/sysadmin/migration migrating_to_koha.html,NONE,1.1 migrating_to_koha.xml,NONE,1.1
skemotah
shedges at users.sourceforge.net
Tue Nov 9 02:30:40 CET 2004
Update of /cvsroot/koha/koha-doc/en/sysadmin/migration
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25216
Added Files:
migrating_to_koha.html migrating_to_koha.xml
Log Message:
Load Migrating to Koha guide
--- NEW FILE ---
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Migrating to Koha ver. 2.0.0</title><link rel="stylesheet" href="html.css" type="text/css"><meta name="generator" content="DocBook XSL Stylesheets V1.65.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="d0e1"></a>Migrating to Koha ver. 2.0.0</h2></div><div><div class="author"><h3 class="author"><span class="firstname">Stephen</span> <span class="surname">Hedges</span></h3><tt class="email"><<a href="mailto:shedges AT skemotah.com">shedges AT skemotah.com</a>></tt></div></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="3"><b>Revision History</b></th></tr><tr><td align="left">Revision 2.0.0</td><td align="left">2004-11-08</td><td align="left">sh</td></tr><tr><td align="left" colspan="3"><p>Initial XML markup of HTML document</p></td></tr></table><
/div></div></div><div></div><hr></div><div class="toc"><dl><dt><span class="section"><a href="#d0e28">1. Copyright and License</a></span></dt><dt><span class="section"><a href="#d0e44">2. Disclaimer</a></span></dt><dt><span class="section"><a href="#d0e51">3. Introduction</a></span></dt><dd><dl><dt><span class="section"><a href="#d0e54">3.1. What is Koha?</a></span></dt><dt><span class="section"><a href="#d0e96">3.2. Why Change to Koha?</a></span></dt></dl></dd><dt><span class="section"><a href="#d0e112">4. Preparations</a></span></dt><dd><dl><dt><span class="section"><a href="#d0e115">4.1. Prerequisites</a></span></dt><dd><dl><dt><span class="section"><a href="#d0e150">4.1.1. Item Types</a></span></dt><dt><span class="section"><a href="#d0e173">4.1.2. MARC tag structure</a></span></dt></dl></dd></dl></dd><dt><span class="section"><a href="#d0e247">5. Extracting and Loading the Data</a></span></dt><dd><dl><dt><span class="section"><a href="#d0e266">5.1. Bibliographic Records
</a></span></dt><dt><span class="section"><a href="#d0e406">5.2. User Information</a></span></dt><dt><span class="section"><a href="#d0e755">5.3. Transaction Records</a></span></dt><dt><span class="section"><a href="#d0e919">5.4. Additional Documentation</a></span></dt></dl></dd></dl></div><font color="red"><authorblurb><p>By Stephen Hedges, Skemotah Solutions</p></authorblurb></font><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e28"></a>1. Copyright and License</h2></div></div><div></div></div><p>This document, <span class="emphasis"><em>Migrating to Koha ver. 2.0.0</em></span>, is copyrighted (c) 2004 by <span class="emphasis"><em>Stephen Hedges</em></span> of <a href="http://www.skemotah.com" target="_top">Skemotah Solutions</a>. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version publ
ished by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at <a href="http://www.gnu.org/copyleft/fdl.html" target="_top">http://www.gnu.org/copyleft/fdl.html</a></p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e44"></a>2. Disclaimer</h2></div></div><div></div></div><p>No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies, that could be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility.</p><p>All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of par
ticular products or brands should not be seen as endorsements.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e51"></a>3. Introduction</h2></div></div><div></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e54"></a>3.1. What is Koha?</h3></div></div><div></div></div><p>Koha version 2.0.0 is an open source integrated library system (ILS) for automating a lending library. It has all of the basic features needed to run a library, handling:</p><div class="itemizedlist"><ul type="disc"><li><p>an online public access catalog (OPAC) of the library's holdings;</p></li><li><p>a database of library users;</p></li><li><p>issuing books to borrowers and returning books to the collection;</p></li><li><p>borrower requests for library items;</p></li><li><p>orders from vendors;</p></li><li><p>book budgets;</p></li><li><p>transfers between library branche
s;</p><p>and most other functions associated with operating a lending library.</p></li></ul></div><p>Koha ver.2.0.0 supports MARC21 and UNIMARC bibliographic records. While there are still a few ILS products that use their own format for storing bibliographic data, most commercial products have adopted some version of the venerable MARC (MAchine Readable Cataloging) record standard. Beginning with version 2.0.0, Koha supports MARC records, specifically records following the <a href="http://www.loc.gov/marc/" target="_top">MARC21</a> and <a href="http://www.ifla.org/VI/3/p1996-1/sec-uni.htm" target="_top">UNIMARC</a> standards.</p><p>Koha is currently in use in several small to medium size libraries with as many as 130,000 bibliographic records and 40,000 borrower records. At the time of this writing (early 2004), it lacks support for Library of Congress classification, using the Dewey Decimal System instead, meaning that it is best-suited for schools and public librari
es. The current code is maintained on <a href="http://sourceforge.net/projects/koha" target="_top">Sourceforge</a>.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e96"></a>3.2. Why Change to Koha?</h3></div></div><div></div></div><p>The main attraction of Koha is that it is free, since it is open source software. It is best to think of it as "free to change" however, rather than "free of cost" It is possible to set up a functioning Koha system without spending any money, but you will still spend more time on the implementation than you would with a commercial ILS (where you are paying the software vendor to set it up for you), and you will need to have a pretty fair knowledge of web server software (usually <a href="http://www.apache.org/" target="_top">Apache</a>) and the <a href="http://www.perl.org/" target="_top">Perl</a> programming language in order to configure some parts of Koha. Some
knowledge of <a href="http://www.mysql.com/" target="_top">MySQL</a> is also handy for database maintenance chores. Most libraries should probably plan on investing time, training, and some money before they have Koha up and running. That being said, Koha is still much less expensive than commercial library software, especially when you consider that there are no annual license fees.</p><p>The best reason to migrate from a commercial system to Koha is the fact that you will have complete control over your library software. Because the entire source code is open, a library can modify Koha to do things as the library wants them done. Commercial systems must aim their research and development for the "middle of the market," where most of their business can be found. But a library that wants to surpass the capabilities of most other libraries will find that Koha provides the freedom to try boldly imaginative innovations in library service.</p></div
></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e112"></a>4. Preparations</h2></div></div><div></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e115"></a>4.1. Prerequisites</h3></div></div><div></div></div><p>This document assumes that you have already installed a Koha system and have an existing ILS, and now wish to transfer your data from the old system to Koha. There are no instructions in this document, therefore, for installing and configuring the basic Koha parameters. (The most recent instruction set should be available at <a href="http://www.koha.org/manual" target="_top">koha.org/manual</a>. There is also a very good guide for installing Koha on Debian available from <a href="http://kados.org/LibraryScience/installing_koha.html" target="_top">Joshua Ferraro</a>.) Nor are there any instructions for entering
new data or using Koha's Z39.50 client to retrieve MARC records from a Z39.50 server. If you are planning to build such "new" data instead of transferring data from an existing system, however, parts of this document will still be helpful in explaining how Koha stores bibliographic and patron data.</p><p>Once you have installed the software and are able to run it, you will need to set up your system parameters on the administration page (<tt class="filename">cgi-bin/koha/admin-home.pl</tt>). Some of these, such as the book funds and budget, currencies, authorized values, etc., can be done if and when you choose to use those features of Koha. Others need to be done before you can import data into Koha and use it. These essential parameters are:</p><div class="itemizedlist"><ul type="disc"><li><p>Item types;</p></li><li><p>MARC tag structure (and Koha to MARC links);</p></li><li><p>Library branches;</p></li><li><p>Borrower categories;</p></li><li><p>Sto
p words; and</p></li><li><p>System preferences.</p></li></ul></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="d0e150"></a>4.1.1. Item Types</h4></div></div><div></div></div><p>The item types are the "categories" into which your library items fall. For instance, you probably want to have videocassettes in a different category from non-fiction books, and mysteries in a different category from children's picture books. If you already are using a commercial ILS, you almost certainly already have all of your materials divided up into such categories. Now you need to tell Koha what your categories are.</p><p>This is more important than it sounds. Much of Koha simply does not work right unless your item types are defined and every item has some type designated. Small things like loan periods and item counts become big headaches if you haven't set up your item types. There's good reason for this
being at the top of the Koha parameters page!</p><p>As you add item types to Koha, you should be aware that the type code is limited to four characters. This code is rarely displayed by Koha; instead the description of the type will be what users see. Loan length is specified as a number of (calendar) days and the rental charge is just what it sounds like, any amount you might charge to users for borrowing items of a certain type (like videos).</p><p>Renewals allowed is a yes/no question on the administration page (check the box if items of this type can be renewed). What actually happens when the item type is saved is that a number is saved in the <tt class="varname">renewalsallowed</tt> column in the <tt class="varname">itemtypes</tt> table of the Koha MySQL database: yes=1 and no=0. But this number is also used by the code to determine how many times an item can be renewed. If your library allows items to be renewed more than once, the
n you will have to manually edit the <tt class="varname">itemtypes</tt> table to reflect this. For example, if you allow some item types to be renewed twice, you need to manually change the value of <tt class="varname">renewalsallowed</tt> from 1 to 2.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="d0e173"></a>4.1.2. MARC tag structure</h4></div></div><div></div></div><p>Koha allows you to specify which MARC tags you want to use and which you want to ignore. When you downloaded and installed Koha, you also got the entire list of MARC21 tags and subfields in current use. Now you need to use the administration page to edit this list and tell Koha which tags you want to use and how you want to use them, or modify the tags to conform to UNIMARC standards.</p><p>If you are CERTAIN that you will never use a MARC tag, then you can delete it, but since this will not result in any appreciable improvement in p
erformance, it is probably better to leave it. There will be tags you want to add, however. If you are using older MARC tags that are not in the list of tags supplied with Koha, then use the MARC tag structure administration page (<tt class="filename">cgi-bin/koha/admin/marctagstructure.pl</tt>) to add them. Similarly, you will probably need to add the holdings tag you currently use, or at least check the subfield structure of the 852 tag if you use it for holdings.</p><p>Editing the subfields from the MARC tag structure page is very important (<tt class="filename">cgi-bin/koha/admin/marc_subfields_structure.pl</tt>). It is also a very time-consuming process, but one that is absolutely necessary if you want your Koha installation to behave as intended. As an ilustration, here is a possible subfield structure for the 245 "title statement" tag:</p><pre class="programlisting">SELECT tagsubfield,kohafield,tab FROM marc_subfield_structure WHERE tagfi
eld=245;
+-------------+-------------------------+------+
| tagsubfield | kohafield | tab |
+-------------+-------------------------+------+
| 6 | | -1 |
| 8 | | -1 |
| a | biblio.title | 1 |
| b | bibliosubtitle.subtitle | 1 |
| c | | 1 |
| f | | -1 |
| g | | -1 |
| h | biblio.abstract | 1 |
| k | | -1 |
| n | | 1 |
| p | | 1 |
| s | | -1 |
+-------------+-------------------------+------+</pre><p>To focus on the important information, the query above does not show the "lib for librarians" and "lib for OPAC" values (what the librarian or OPAC user will see as the subfield description) or the repeatable and mandatory values (always either yes or no, represented by 1 or 0 in the database). The standard MARC21 subfield descriptions and repeatabilty are supplied with Koha, and if you require that <span class="emphasis"><em>every one</em></span> of your MARC records have a particular subfield, then you should set that subfield to mandatory.</p><p>The "non-standard" information here is the Koha field and the tab. The <tt class="varname">kohafield</tt> column holds any mapping between the MARC tables in the database and the original Koha tables. MARC records are capable of holding much more information than the original, non-MARC Koha tables. For standard searching and display of brief bibliograp
hic information, however, the original Koha tables are usually adequate. Therefore, instead of rewriting most of the scripts and modules in Koha version 1.9 to make the MARC-compatible version 2.0, the developers set up a separate table (<tt class="varname">marc_subfield_value</tt>) to hold the MARC record data, and linked that data to the original Koha tables, so the original scripts and modules could still be used.</p><p>In the example above, the 245a MARC tag and subfield is linked (mapped) to the title column in the original Koha <tt class="varname">biblio</tt> table, and the 245b tag is linked to the <tt class="varname">subtitle</tt> column in the <tt class="varname">bibliosubtitle</tt> table. This means that every time a new MARC record is loaded into the database, any information that is in the 245a tag is also stored in a corresponding record in the <tt class="varname">biblio</tt> table, and anything in the 245b tag also gets stored in the <tt class="varn
ame">bibliosubtitle</tt> table.</p><p>We just said that the original Koha tables are "usually adequate" for storing enough bibliographic information to construct brief displays. In the example above, however, you can see a "work-around" for an instance in which the Koha tables were not adequate. In the particular installation of Koha from which this example was taken, it was decided that it was important to store the 245h "material designation" tag in the original Koha tables so it could be displayed in the first list of results of any search. The material designation would tell the searcher if the bibliographic item shown was something other than a book, such as a sound recording or a videorecording. Since there was no other place to store this information, an otherwise unused Koha column was selected (the abstract column in the biblio table), and the templates that build the search results displays were altered accordingly. (As w
ith Perl, with Koha there is always more than one way to do things!)</p><p>Once you have decided which MARC subfields to map to Koha columns, don't use the MARC tag structure page to do it. While it is possible to do it here, Koha has a separate <span class="bold"><b>Links Koha-MARC DB</b></span> page (<tt class="filename">cgi-bin/koha/admin/koha2marclinks.pl</tt>) that you can access from the parameters page (<tt class="filename">admin-home.pl</tt>). This interface is a much easier and safer way to set up your MARC to Koha mapping.</p><p>The tab column holds either a -1 or a number from 0 to 10. "Tab" is used here in the sense of a tab in a web browser, not a tab in a text processor. It controls how MARC tags and subfields are grouped and displayed when the librarian or cataloger is working with the MARC records. Any tags and subfields that you want to have available for editing or displaying in the MARC display pages need to have a value
between 0 and 9. A -1 value means the subfield is ignored when building such pages. While you could give all the other subfields (with the exception of subfields in your item information tag) the same value, all these subfields, whether empty or not, would display every time you wanted to edit or display your MARC record, making a very long web page! It is best to subdivide your MARC tags and subfields into logical groups that are then displayed on separate pages in their own "tabs." Here are a couple of examples to illustrate one way to do this:</p><pre class="programlisting">SELECT tagfield,tagsubfield,tab FROM marc_subfield_structure WHERE tab=0;
+----------+-------------+------+
| tagfield | tagsubfield | tab |
+----------+-------------+------+
| 010 | a | 0 |
| 010 | b | 0 |
| 020 | a | 0 |
| 022 | a | 0 |
| 024 | a | 0 |
| 024 | d | 0 |
| 028 | a | 0 |
| 028 | b | 0 |
| 035 | a | 0 |
| 082 | a | 0 |
| 082 | b | 0 |
| 100 | a | 0 |
| 100 | d | 0 |
| 110 | a | 0 |
+----------+-------------+------+
SELECT tagfield,tagsubfield,tab FROM marc_subfield_structure WHERE tab=1;
+----------+-------------+------+
| tagfield | tagsubfield | tab |
+----------+-------------+------+
| 245 | a | 1 |
| 245 | b | 1 |
| 245 | c | 1 |
| 245 | h | 1 |
| 245 | n | 1 |
| 245 | p | 1 |
| 246 | a | 1 |
| 246 | b | 1 |
| 246 | h | 1 |
| 250 | a | 1 |
| 250 | b | 1 |
| 256 | a | 1 |
| 260 | a | 1 |
| 260 | b | 1 |
| 260 | c | 1 |
| 300 | a | 1 |
| 300 | b | 1 |
| 300 | c | 1 |
| 300 | e | 1 |
| 300 | f | 1 |
| 300 | g | 1 |
| 440 | a | 1 |
| 440 | n | 1 |
| 440 | v | 1 |
| 490 | x | 1 |
| 490 | v | 1 |
| 490 | a | 1 |
+----------+-------------+------+</pre><p>...and so forth. The tab value 10 is reserved for the item information from your holdings tag subfield, since this information is always displayed on a separate page when editing or displaying MARC records.</p><p>There's one other chore you need to take care of before you are ready to load data, and that is checking to make sure the <tt class="varname">barcode</tt> column in the items table is big enough to hold your barcodes. If your barcodes are longer than the space allowed, you will need to <a href="http://dev.mysql.com/doc/mysql/en/ALTER_TABLE.html" target="_top">alter</a> the items table accordingly.</p></div></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e247"></a>5. Extracting and Loading the Data</h2></div></div><div></div></div><p>Obviously, if you have an existing ILS, you have a lot of valuable data that you have collected that y
ou will want to transfer into Koha. This process may be the most complex and costly part of migrating to Koha, and is unfortunately an area where this document will be of relatively little help. Each vendor's product handles data differently, with variations in database structure, database software, and exporting tools. You may well find that you will need top pay your current vendor to prepare your data for export, with no recourse to any other solution because their software source code is closed and proprietary.</p><p>Still, there are a few general comments about data extraction that will be helpful in planning and negotiating the process of exporting your data.</p><p>There are basically three types of data you will want to retrieve from your old system:</p><div class="itemizedlist"><ul type="disc"><li><p>bibliographic records;</p></li><li><p>user information; and</p></li><li><p>transaction records.</p></li></ul></div><div class="section" lang="en"><div
class="titlepage"><div><div><h3 class="title"><a name="d0e266"></a>5.1. Bibliographic Records</h3></div></div><div></div></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Since it is assumed that you are already using a commercial ILS and are migrating to the most current version of Koha, the following remarks pertain to preparing existing MARC records for import into Koha.</p></div><p>Koha version 2.0.0 makes extensive use of the <a href="http://marcpm.sourceforge.net/" target="_top">MARC::Record</a> Perl package for reading, checking, and processing records for import. This collection of Perl modules is designed to work with MARC data in transmission format, more accurately "ISO 2709 exchange format," which is the MARC data in a format that's easy for computers to pass back and forth, though not so easy for humans to read. As the name "ISO 2709" implies, this is an international standard for formating bi
bliographic records; your current ILS vendor should have tools available to export your bibliographic records in ISO 2709 format (or you should demand to know why they are not conforming to international standards!). Getting your records -- your complete records, including holdings information -- in ISO 2709 format is the essential first step in importing your bibliographic information into Koha. If for some strange reason your current ILS is incapable of dumping records in ISO 2709 format, but can do ASCII or some other widely recognized format, you may be lucky enough to be able to use some of the <a href="http://www.loc.gov/marc/marctools.html" target="_top">tools</a> listed by the Library of Congress for manipulating MARC records and do the conversion to ISO 2709 yourself. Most likely, however, you will find that your vendor can supply the tools you need for exporting your records in the standard exchange format, though they may ask for a fee before pro
viding them.</p><p>Once you have your records in ISO 2709 format, you will work from a command line on your Koha server and run the <tt class="filename">bulkmarcimport.pl</tt> script that comes with Koha to load your records. The command to use is: <b class="userinput"><tt>KohaFolder/intranet/scripts/misc/bulkmarcimport.pl -file YourISO2709fileName</tt></b> -- where <tt class="filename">KohaFolder</tt> is the name of the directory or folder where you installed Koha (such as <tt class="filename">/usr/local/koha</tt>) and <tt class="filename">YourISO2709fileName</tt> is the name (and location) of the file of MARC records dumped from your old ILS.</p><p>If you intend to make any changes to the structure of your MARC records (for example, you want to change your holdings tag structure), this should be done before you import the records. Again, the Library of Congress tools may be helpful. If you are reasonably adept at Perl programming, or are willing to
pay a small fee to someone who is, you can also use the MARC::Record Perl package to rebuild your records. Either way, the point is the same: make any changes <span class="emphasis"><em>before</em></span> you import your records, by making changes to the ISO 2709 file itself.</p><p>The <tt class="filename">bulkmarcimport.pl</tt> script is also a handy place to start if you want to write a script to import records on a regular basis. This might be the case if you use a different utility to generate new MARC records and then do regular imports of those records into Koha. Because of differences between MARC records and Koha records, you will find that some pieces of information do not import cleanly with the 'plain' <tt class="filename">bulkmarcimport.pl</tt> utility. For example, MARC publication dates often have a 'c' (for 'copyright') in front, so <tt class="filename">bulkmarcimport.pl</tt> will place 'c1999' in the Koha publication date column
in the database. (This is assuming that you have mapped MARC tag 260c to <tt class="varname">biblioitems.publicationyear</tt>.) Koha can select catalog records by publication year, but only if the publication year is a number, so entries like 'c1999' would not be selected. While it is easy enough to correct this after the initial import of your records by using the MySQL <a href="http://dev.mysql.com/doc/mysql/en/String_functions.html" target="_top">TRIM</a> function, you won't want to be doing this kind of 'clean-up' work on a regular basis, when you can modify the <tt class="filename">bulkmarcimport.pl</tt> script to do it when the records are loaded.</p><p>As of this writing, the Koha import process does not handle multiple subject entries very well. In general, while a MARC record may have many subject entries, only one currently gets loaded into the Koha subjects table (<tt class="varname">bibliosubject</tt>). Again, this can be remedied b
y modifying the <tt class="filename">bulkmarcimport.pl</tt> script if you are good at Perl. (Or, at the risk of being repetitious, you are willing to hire someone who is.)</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Since this document was first written, newer versions of Koha have been released which do catalog searches directly against the MARC record, instead of the old Koha tables, so loading of data into the subjects table is less important.</p></div><p>If you are getting ready to do some complex things with your records, you need to know a little about Koha's database tables for storing records:</p><pre class="programlisting">+-------------------------+
| Bibliographic Tables |
+-------------------------+
| additionalauthors |
| biblio |
| biblioanalysis |
| biblioitems |
| bibliosubject |
| bibliosubtitle |
| bibliothesaurus |
| catalogueentry |
| deletedbiblio |
| deletedbiblioitems |
| deleteditems |
| items |
| itemsprices |
| marc_biblio |
| marc_blob_subfield |
| marc_subfield_table |
| marc_word |
+-------------------------+</pre><p>Many of the table names are self-explanatory, but some are not. The three primary 'old-Koha' tables are; <tt class="varname">biblio</tt>, which holds the basic bibliographic information such as title, author, etc.; <tt class="varname">biblioitems</tt>, which holds more specific information such as format, publication year, item type, etc.; and <tt class="varname">items</tt>, which holds information about specific items, such as barcode number, price, date acquired, etc. These three tables are linked together by a common '<tt class="varname">biblionumber</tt>.' You may have a <tt class="varname">biblio</tt> entry for <i class="citetitle">The Two Towers</i>, several <tt class="varname">biblioitems</tt> entries that link to it (perhaps one for the book, one for the audio recording, one for the large print edition), and multiple <tt class="varname">items</tt> (copies) for each biblioitem. This structure mirrors som
e aspects of the International Federation of Library Associations' Functional Requirements for Bibliographic Records (<a href="http://www.ifla.org/VII/s13/frbr/frbr.htm" target="_top">FRBR</a>).</p><p>MARC records do not have such a structure, but instead have a basic record, which would contain the information that Koha would keep in the <tt class="varname">biblio</tt> and <tt class="varname">biblioitems</tt> tables, and a holdings tag, which would contain the information in Koha's <tt class="varname">items </tt>table. Koha stores MARC records in the <tt class="varname">marc_subfield_table</tt> table, which breaks the record down into tags and subfields and then links everything back together using a single '<tt class="varname">bibid</tt>' for all the pieces. The <tt class="varname">marc_biblio</tt> table keeps track of the relationship between the <tt class="varname">bibid</tt> of the MARC record and the <tt class="varname">biblionumber</tt> of the correspondin
g information that has been stored in the old-Koha tables.</p><p>There are also two general "keyword" tables that serve somewhat as indexes, storing each word in a record with the <tt class="varname">biblionumber</tt> or <tt class="varname">bibid</tt> of the record it came from. For items cataloged using the old-Koha interface, these words are stored in <tt class="varname">catalogueentry</tt>, while words from MARC records (either imported or cataloged using the MARC interface) are stored in <tt class="varname">marc_word</tt>.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e406"></a>5.2. User Information</h3></div></div><div></div></div><p>Unlike bibliographic records, there is as yet no commonly used standard for user records, though the relatively new <a href="http://www.niso.org/committees/committee_at.html" target="_top">NCIP</a> standard addresses some aspects of user information and may become as widespr
ead as MARC records someday. Meanwhile, this is an area where you will have to dig out as much information as possible from your existing database, using whatever means possible. If you are fortunate enough to have some sort of export utility with you current ILS, you just need to determine that it exports all the data you want. Most likely, however, you have no such utility and will have to either access your database directly and dump the data you need (if possible) or ask your current vendor to do it for you. In any case, you will want to retrieve the following information about your users:</p><div class="itemizedlist"><ul type="disc"><li><p>names (first names, last name, and perhaps title);</p></li><li><p>library card number;</p></li><li><p>user ID (if different from library card number);</p></li><li><p>street address;</p></li><li><p>mailing address, or any other alternate address you might have on file;</p></li><li><p>date of birth (if available)
;</p></li><li><p>telephone number;</p></li><li><p>user category code (e.g. 'A' for adult);</p></li><li><p>date enrolled (if available);</p></li><li><p>branch where enrolled (if available);</p></li><li><p>any status codes (e.g. blocked, bad address, etc.);</p></li><li><p>user password (if available in plain text);</p></li><li><p>preferred method of contact (if available);</p></li><li><p>any notes.</p></li></ul></div><p>This information will then need to be loaded into the Koha <tt class="varname">borrowers</tt> table. The columns in this table are a little confusing, since the table has been expanded at various times to accomodate the needs of libraries needing to store more information and therefore needing more columns in the table. Here is the current table structure, with some notes on what the columns can contain:</p><pre class="programlisting">SHOW COLUMNS FROM borrowers;
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| borrowernumber | int(11) | | MUL | NULL | auto_increment |
| cardnumber | varchar(9) | | MUL | | |
| surname | text | | | | |
| firstname | text | | | | |
| title | text | YES | | NULL | |
| othernames | text | YES | | NULL | |
| initials | text | | | | |
| streetaddress | text | | | | |
| suburb | text | YES | | NULL | |
| city | text | | | | |
| phone | text | | | | |
| emailaddress | text | YES | | NULL | |
| faxnumber | text | YES | | NULL | |
| textmessaging | text | YES | | NULL | |
| altstreetaddress | text | YES | | NULL | |
| altsuburb | text | YES | | NULL | |
| altcity | text | YES | | NULL | |
| altphone | text | YES | | NULL | |
| dateofbirth | date | YES | | NULL | |
| branchcode | varchar(4) | | | | |
| categorycode | char(2) | YES | | NULL | |
| dateenrolled | date | YES | | NULL | |
| gonenoaddress | tinyint(1) | YES | | NULL | |
| lost | tinyint(1) | YES | | NULL | |
| debarred | tinyint(1) | YES | | NULL | |
| studentnumber | text | YES | | NULL | |
| school | text | YES | | NULL | |
| contactname | text | YES | | NULL | |
| borrowernotes | text | YES | | NULL | |
| guarantor | int(11) | YES | | NULL | |
| area | char(2) | YES | | NULL | |
| ethnicity | varchar(50) | YES | | NULL | |
| ethnotes | varchar(255) | YES | | NULL | |
| sex | char(1) | YES | | NULL | |
| expiry | date | YES | | NULL | |
| altnotes | varchar(255) | YES | | NULL | |
| altrelationship | varchar(100) | YES | | NULL | |
| streetcity | text | YES | | NULL | |
| phoneday | varchar(50) | YES | | NULL | |
| preferredcont | char(1) | YES | | NULL | |
| physstreet | varchar(100) | YES | | NULL | |
| homezipcode | varchar(25) | YES | | NULL | |
| zipcode | varchar(25) | YES | | NULL | |
| userid | varchar(30) | YES | | NULL | |
| password | varchar(30) | YES | | NULL | |
| flags | int(11) | YES | | NULL | |
+------------------+--------------+------+-----+---------+----------------+</pre><div class="variablelist"><dl><dt><span class="term">borrowernumber</span></dt><dd><p>This value is automatically generated when a patron is added. To add patrons in bulk, however, you may need to generate this number manually. For instance, if you put your patron information into a comma-delimited file and use MySQL's "LOAD DATA INFILE" command to load your data (a very fast operation!), the first comma-delimited field should contain a unique borrowernumber, preferably sequential.</p></dd><dt><span class="term">cardnumber</span></dt><dd><p>the user's library card "number." It does not actually need to be a number.</p></dd><dt><span class="term">surname</span></dt><dd><p>the user's family name</p></dd><dt><span class="term">firstname</span></dt><dd><p>can actually be as many "first" names as necessary</p></dd><dt><span class="term">title</span></dt><dd><p>e.g. "Mr."</p></dd><dt>
<span class="term">othernames</span></dt><dd><p>names after the first name can be put here instead of in firstname</p></dd><dt><span class="term">initials</span></dt><dd><p>the person's initials, e.g. 'John Doe' = 'JD'; rarely used and relatively unimportant</p></dd><dt><span class="term">streetaddress</span></dt><dd><p>actually the mailing address; see also physstreet</p></dd><dt><span class="term">suburb</span></dt><dd><p>useful in some countries</p></dd><dt><span class="term">city</span></dt><dd><p>and also the state, etc., any other info needed for the mailing address other than the postal code</p></dd><dt><span class="term">phone</span></dt><dd><p>the telephone number</p></dd><dt><span class="term">emailaddress</span></dt><dd><p>the e-mail address</p></dd><dt><span class="term">faxnumber</span></dt><dd><p>the number of the person's fax machine</p></dd><dt><span class="term">textmessaging</span></dt><dd><p>contact information for any text messaging device</p></dd><dt><sp
an class="term">altstreetaddress</span></dt><dd><p>street address of the person's alternate contact (contactname)</p></dd><dt><span class="term">altsuburb</span></dt><dd><p>alternate contact's suburb</p></dd><dt><span class="term">altcity</span></dt><dd><p>alternate contact's city (and state/province AND postal code)</p></dd><dt><span class="term">altphone</span></dt><dd><p>alternate contact's telephone number</p></dd><dt><span class="term">dateofbirth</span></dt><dd><p>should be entered in MySQL format (yyyy-mm-dd), Koha will convert based on your parameters setting</p></dd><dt><span class="term">branchcode</span></dt><dd><p>your code for the branch where the person enrolled</p></dd><dt><span class="term">categorycode</span></dt><dd><p>your code for the type of member; 'C' is suggested for children, since this triggers some of the guarantor options</p></dd><dt><span class="term">dateenrolled</span></dt><dd><p>the date when the person enrolled, in MySQL format (yyyy-mm-dd).<
/p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>You should set this to '0000-00-00' if you have no enrollment date. Leaving it blank (NULL) leads to inaccurate dates when editing borrower records later.</p></div></dd><dt><span class="term">gonenoaddress</span></dt><dd><p>set to '1' if the person has moved without leaving a forwarding address</p></dd><dt><span class="term">lost</span></dt><dd><p>set to '1' if the person's library card has been reported lost</p></dd><dt><span class="term">debarred</span></dt><dd><p>set to '1' if the person has been "debarred" or blocked from receiving library service for excessive overdues or fines, etc.</p></dd><dt><span class="term">studentnumber</span></dt><dd><p>a student ID number, but could be used for any ID number (e.g. Social Security number in the USA)</p></dd><dt><span class="term">school</span></dt><dd><p>the student's school (for use with student number)</p></dd><dt><span c
lass="term">contactname</span></dt><dd><p>the name of the person's alternate contact</p></dd><dt><span class="term">borrowernotes</span></dt><dd><p>any notes or messages about the person</p></dd><dt><span class="term">guarantor</span></dt><dd><p>for children, this is the borrowernumber of the adult parent or custodian</p></dd><dt><span class="term">area</span></dt><dd><p>a two-letter code set by the library to identify the service area in which the person resides (may be omitted)</p></dd><dt><span class="term">ethnicity</span></dt><dd><p>useful for libraries that are required to track service to ethnic minorities, but easily ignored</p></dd><dt><span class="term">ethnotes</span></dt><dd><p>any notes related to ethnicity</p></dd><dt><span class="term">sex</span></dt><dd><p>'M' or 'F' (male or female); 'M' is the default</p></dd><dt><span class="term">expiry</span></dt><dd><p>the date when the person's membership expires, in MySQL format (yyyy-mm-dd)</p></dd><dt><span class="t
erm">altnotes</span></dt><dd><p>any notes or messages about the alternate contact</p></dd><dt><span class="term">altrelationship</span></dt><dd><p>the nature of the alternate contact, can be 'workplace', 'relative','friend' or 'neighbour'</p></dd><dt><span class="term">streetcity</span></dt><dd><p>goes with physstreet if the person lives in a city other than their mailing address</p></dd><dt><span class="term">phoneday</span></dt><dd><p>the daytime or "work" telephone number</p></dd><dt><span class="term">preferredcont</span></dt><dd><p>a one-letter code indicating the person's preferred contact method, e.g. 'E' for e-mail</p></dd><dt><span class="term">physstreet</span></dt><dd><p>the actual street address, if the person has a different mailing address (e.g. post office box)</p></dd><dt><span class="term">homezipcode</span></dt><dd><p>the postal code of the person's physical street address</p></dd><dt><span class="term">zipcode</span></dt><dd><p>the postal code of the perso
n's mailing address</p></dd><dt><span class="term">userid</span></dt><dd><p>this is the username the person has chosen to use to gain access to their account through the OPAC</p></dd><dt><span class="term">password</span></dt><dd><p>this is the password the person has chosen to use to gain access to their account through the OPAC; it cannot be plain text, but must be encrypted using MD5 (md5_base64) before importing</p></dd><dt><span class="term">flags</span></dt><dd><p>these are the person's privileges, which can be set when editing their record. The privileges are stored as the decimal representation of a binary number, with each binary digit indicating whether the privilege is allowed ('1') or not allowed ('0'). Typically, you will want to give imported borrowers the privileges to borrow books and reserve books for themselves, which can be set by inserting the integer 384 in this column.</p></dd></dl></div><p>The most important pieces of information are the bo
rrowernumber and the cardnumber (required), the person's surname and firstname, their mailing address, and the branch at which they enrolled. All of the other information can be useful but may be omitted without affecting Koha's operation significantly.</p><p>Probably the easiest and fastest way to load the borrower data into Koha is to use a simple Perl script or a spreadsheet utility to put the information into a comma-separated value (csv) file, with the same structure as the Koha borrowers file, and then use MySQL's <a href="http://dev.mysql.com/doc/mysql/en/LOAD_DATA.html" target="_top">LOAD DATA INFILE</a> utility to load the data. If using a spreadsheet utility to do this, for example, you would create a spreadsheet with a column corresponding to each column in the borrowers table, put your borrower data into these columns, and save the whole thing in csv format. If you used "borrowers.csv" for the name of this file, you would then go into MySQL and
do: <b class="userinput"><tt>mysql> LOAD DATA INFILE "borrowers.csv" INTO TABLE borrowers FIELDS TERMINATED BY ',' ENCLOSED BY '"';</tt></b></p><p>This process is fast, loading tens of thousands of borrower records in only a couple of seconds.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e755"></a>5.3. Transaction Records</h3></div></div><div></div></div><p>Obviously, once you have loaded your bibliographic records and your borrower records, you need to know which borrowers have borrowed which items, which borrowers have reserves on which items, which borrowers owe fines for which items, etc. In other words, you need your transaction records, which relate your books to your borrowers.</p><p>As with borrower records, you will have to dig out as much information as possible from your existing database, using whatever means possible. Most likely, you will have to either access your database directly and dump the d
ata you need (if possible) or ask your current vendor to do it for you. In any case, you will probably want to retrieve the following transaction information (at least):</p><div class="itemizedlist"><ul type="disc"><li><p>items on loan, and to whom they are (and were) loaned;</p></li><li><p>borrowers who are blocked from using the library (and why);</p></li><li><p>borrowers who have bad addresses;</p></li><li><p>charges owed by borrowers;</p></li><li><p>items reserved by borrowers.</p></li></ul></div><p>Remember, because transaction records bascially involve some sort of relationship between borrowers and items, you want to be sure you have already loaded both your bibliographic and user records <span class="emphasis"><em>before</em></span> you load transaction data.</p><p>Here are the Koha database tables that store transaction data:</p><pre class="programlisting">+-------------------------+
| Transaction Tables |
+-------------------------+
| accountlines |
| accountoffsets |
| issues |
| reserves |
+-------------------------+</pre><p>Some of the columns in the <tt class="varname">borrowers</tt> table also hold some transaction-type data, namely <tt class="varname">gonenoaddress</tt> (set to "1" if the borrower has moved without leaving a new address), <tt class="varname">lost</tt> (set to "1" if the borrower has lost their library card), <tt class="varname">debarred</tt> (set to "1" if the borrower's library privileges have been suspended), and <tt class="varname">borrowernotes</tt>, which can hold any textual information about the borrower.</p><p>The <tt class="varname">accountlines</tt> and <tt class="varname">accountoffsets</tt> tables hold information about charges which the borrower owes the library:</p><pre class="programlisting">SHOW COLUMNS FROM accountlines;
+-------------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+---------------+------+-----+---------+-------+
| borrowernumber | int(11) | | MUL | 0 | |
| accountno | smallint(6) | | | 0 | |
| itemnumber | int(11) | YES | | NULL | |
| date | date | YES | | NULL | |
| amount | decimal(28,6) | YES | | NULL | |
| description | text | YES | | NULL | |
| dispute | text | YES | | NULL | |
| accounttype | varchar(5) | YES | | NULL | |
| amountoutstanding | decimal(28,6) | YES | | NULL | |
| timestamp | timestamp(14) | YES | MUL | NULL | |
+-------------------+---------------+------+-----+---------+-------+</pre><pre class="programlisting"> SHOW COLUMNS FROM accountoffsets;
+----------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------------+------+-----+---------+-------+
| borrowernumber | int(11) | | | 0 | |
| accountno | smallint(6) | | | 0 | |
| offsetaccount | smallint(6) | | | 0 | |
| offsetamount | decimal(28,6) | YES | | NULL | |
| timestamp | timestamp(14) | YES | | NULL | |
+----------------+---------------+------+-----+---------+-------+</pre><p>For migration purposes, <tt class="varname">accountlines</tt> is the table that needs to be loaded with your existing data about charges owed by users. The <tt class="varname">accountoffsets</tt> table stores information about payments made against the account. For example, if a borrower owes the library $20.00 for a lost book, that information would get loaded into the <tt class="varname">accountlines</tt> table's <tt class="varname">borrowernumber</tt>, <tt class="varname">itemnumber</tt>, <tt class="varname">amount</tt> and <tt class="varname">amountoutstanding</tt> columns. Initially, <tt class="varname">amount</tt> and <tt class="varname">amountoutstanding</tt> are the same. If the borrower then makes a $10.00 payment toward this account, the payment amount is stored in <tt class="varname">accountoffsets.offsetamount</tt> and a new value for <tt class="varname">accountlines.amountoutstanding</tt> i
s calculated.</p><p>Probably the most important data you want to move from your existing ILS to Koha is the record of which books each borrower has. This data gets loaded into the <tt class="varname">issues</tt> table:</p><pre class="programlisting">SHOW COLUMNS FROM issues;
+-----------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+---------------+------+-----+---------+-------+
| borrowernumber | int(11) | | MUL | 0 | |
| itemnumber | int(11) | | MUL | 0 | |
| date_due | date | YES | | NULL | |
| branchcode | char(4) | YES | | NULL | |
| issuingbranch | char(18) | YES | | NULL | |
| returndate | date | YES | | NULL | |
| lastreneweddate | date | YES | | NULL | |
| return | char(4) | YES | | NULL | |
| renewals | tinyint(4) | YES | | NULL | |
| timestamp | timestamp(14) | YES | | NULL | |
+-----------------+---------------+------+-----+---------+-------+</pre><p>The column names in this table are mostly self-explanatory. The most important for the migration process are the <tt class="varname">borrowernumber</tt> (which, you'll remember, is <span class="emphasis"><em>not</em></span> the same as the library card number), the <tt class="varname">itemnumber</tt> (not the same as the barcode number) and the <tt class="varname">date_due</tt>. If you do not know the <tt class="varname">branch</tt> where the item was issued, that's OK -- you'll still have a record of who has what and when it is due.</p><p>All the dates should be loaded in the yyyy-mm-dd format expected by MySQL. The <tt class="varname">returndate</tt> column will remain empty until the item is returned. Records are <span class="emphasis"><em>not</em></span> deleted from the <tt class="varname">issues</tt> table when the item is returned -- Koha knows the item has been returned by checking for a return
date.</p><p>Finally, you'll need to load information about reserved items into the <tt class="varname">reserves</tt> table:</p><pre class="programlisting">SHOW COLUMNS FROM reserves;
+------------------+---------------+------+-----+------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------+---------------+------+-----+------------+-------+
| borrowernumber | int(11) | | | 0 | |
| reservedate | date | | | 0000-00-00 | |
| biblionumber | int(11) | | | 0 | |
| constrainttype | char(1) | YES | | NULL | |
| branchcode | varchar(4) | YES | | NULL | |
| notificationdate | date | YES | | NULL | |
| reminderdate | date | YES | | NULL | |
| cancellationdate | date | YES | | NULL | |
| reservenotes | text | YES | | NULL | |
| priority | smallint(6) | YES | | NULL | |
| found | char(1) | YES | | NULL | |
| timestamp | timestamp(14) | YES | | NULL | |
| itemnumber | int(11) | YES | | NULL | |
+------------------+---------------+------+-----+------------+-------+</pre><p>Again, most of the column names are self-explanatory. The <tt class="varname">constrainttype</tt> column is used to indicate whether the reserve has been placed on "any" copy (in which case the <tt class="varname">constrainttype</tt> will be "a" and the <tt class="varname">biblionumber</tt> will be set) or "one" specific copy (in which case the <tt class="varname">constrainttype</tt> will be "o" and the <tt class="varname">itemnumber</tt> will be set). The <tt class="varname">found</tt> column value is set once the item has been returned and is ready to be checked out to the borrower or to be sent to the pick-up branch (which is recorded in the <tt class="varname">branchcode</tt> column). The <tt class="varname">priority</tt> column keeps track of who is next on the list for the item. Once a reserve has been filled, the priority is set to "0."</p></div><div class="section" lang="en"><div class="tit
lepage"><div><div><h3 class="title"><a name="d0e919"></a>5.4. Additional Documentation</h3></div></div><div></div></div><p>Many of the tools you will use in the process of writing scripts and manipulating data have web documentation:</p><div class="variablelist"><dl><dt><span class="term"><a href="http://dev.mysql.com/doc/" target="_top">MySQL</a></span></dt><dd><p>the basic commands you will need to work directly with your database.</p></dd><dt><span class="term"><a href="http://www.perldoc.com/" target="_top">Perl</a></span></dt><dd><p>the programming language used for Koha.</p></dd><dt><span class="term"><a href="http://dbi.perl.org/docs/" target="_top">DBI</a></span></dt><dd><p>the Perl database interface that allows you to include MySQL commands in your Perl scripts and retrieve data for use in scripts.</p></dd><dt><span class="term"><a href="http://directory.google.com/Top/Computers/Programming/Internet/CGI/Tutorials/?tc=1" target="_top">CGI</a></span></dt><dd><p>
the Common Gateway Interface that allows Perl scripts to send data to, and retrieve data from, a webpage.</p></dd><dt><span class="term"><a href="http://marcpm.sourceforge.net/" target="_top">MARC::Record</a></span></dt><dd><p>the Perl module that allows you to manipulate MARC records.</p></dd><dt><span class="term"><a href="http://html-template.sourceforge.net/" target="_top">HTML::Template</a></span></dt><dd><p>the Perl module that allows you to set up webpage templates to display Koha data, or modify the default templates to suit your library.</p></dd></dl></div><p>Additional help is also available by sending an e-mail to the <a href="http://www.koha.org/mailing/" target="_top">Koha mailing list</a>.</p></div></div></div></body></html>
--- NEW FILE ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
<title>Migrating to Koha ver. 2.0.0</title>
<articleinfo>
<date>2004-11-08</date>
<author>
<firstname>Stephen</firstname>
<surname>Hedges</surname>
<email>shedges AT skemotah.com</email>
</author>
<revhistory>
<revision>
[...1421 lines suppressed...]
</varlistentry>
<varlistentry>
<term><ulink
url="http://html-template.sourceforge.net/">HTML::Template</ulink></term>
<listitem>
<para>the Perl module that allows you to set up webpage
templates to display Koha data, or modify the default templates
to suit your library.</para>
</listitem>
</varlistentry>
</variablelist></para>
<para>Additional help is also available by sending an e-mail to the
<ulink url="http://www.koha.org/mailing/">Koha mailing
list</ulink>.</para>
</section>
</section>
</article>
More information about the Koha-cvs
mailing list