[Koha-cvs] CVS: koha-doc/en/sysadmin/migration CDS_ISIS_to_Koha.html,NONE,1.1 CDS_ISIS_to_Koha.xml,NONE,1.1
skemotah
shedges at users.sourceforge.net
Tue Nov 16 13:08:17 CET 2004
Update of /cvsroot/koha/koha-doc/en/sysadmin/migration
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4271
Added Files:
CDS_ISIS_to_Koha.html CDS_ISIS_to_Koha.xml
Log Message:
Load CDS/ISIS to Koha guide
--- NEW FILE ---
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Migrating a CDS/ISIS based system to Koha</title><link rel="stylesheet" href="html.css" type="text/css"><meta name="generator" content="DocBook XSL Stylesheets V1.65.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="d0e1"></a>Migrating a CDS/ISIS based system to Koha</h2></div><div><div class="author"><h3 class="author"><span class="firstname">Baiju</span> <span class="surname">M</span></h3><tt class="email"><<a href="mailto:baijum81 AT hailmail.net">baijum81 AT hailmail.net</a>></tt></div></div><div><p class="pubdate">2004-11-16</p></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="3"><b>Revision History</b></th></tr><tr><td align="left">Revision 2.0.0p1</td><td align="left">2004-11-15</td><td align="left">bm</td></tr><tr><td align="left" colspa
n="3"><p>Small additions by BM.</p></td></tr><tr><td align="left">Revision 2.0.0</td><td align="left">2004-04-01</td><td align="left">bm</td></tr><tr><td align="left" colspan="3"><p>Initial XML markup (by Stephen Hedges) of a wiki page posted at <a href="http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha" target="_top">http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha</a> on 2004-04-01, with links to program listings expanded.</p></td></tr></table></div></div></div><div></div><hr></div><div class="toc"><dl><dt><span class="section"><a href="#d0e41">1. Copyright and License</a></span></dt><dt><span class="section"><a href="#d0e69">2. Disclaimer</a></span></dt><dt><span class="section"><a href="#d0e76">3. Introduction</a></span></dt><dt><span class="section"><a href="#d0e87">4. Migrating Book Data</a></span></dt><dt><span class="section"><a href="#d0e130">5. Migrating Member Data</a></span></dt></dl></div><font
color="red"><authorblurb><p>By Baiju M</p></authorblurb></font><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e41"></a>1. Copyright and License</h2></div></div><div></div></div><p>Copyright 2004 Baiju M <tt class="email"><<a href="mailto:baijum81 AT hailmail.net">baijum81 AT hailmail.net</a>></tt></p><p>This document is related to Koha and is licensed to you under the GNU General Public License version 2 or later (<a href="http://www.gnu.org/licenses/gpl.html" target="_top">http://www.gnu.org/licenses/gpl.html</a>).</p><p>Koha-related documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies.</p><p>You may create a derivative work and distribute it provided that you:</p><div class="orderedlist"><ol type="1"><li><p>License the derivative work with this same license, or the Linux Docume
ntation Project License (<a href="???" target="_top">http://www.tldp.org/COPYRIGHT.html</a>). Include a copyright notice and at least a pointer to the license used.</p></li><li><p>Give due credit to previous authors and major contributors.</p></li></ol></div><p>Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e69"></a>2. Disclaimer</h2></div></div><div></div></div><p>No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies that could be damaging to your system. Although this is highly unlikely, proceed with caution; the author(s) do not take any responsibility.</p><p>All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in
this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e76"></a>3. Introduction</h2></div></div><div></div></div><p>I am doing this migration from CDS/ISIS (version : 1.5) system to Koha for St. Joseph's College, Devagiri, Calicut. <a href="http://devagiricollege.org/" target="_top">http://devagiricollege.org/</a></p><p>In the current system it is possible to do circulation and other basic functionalities of the library.</p><p>The work is in progress, so give any pointers if you have.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e87"></a>4. Migrating Book Data</h2></div></div><div></div></div><div class="itemizedlist"><ul type="disc"><li><p>First I expor
ted Books data into XML using CDS/ISIS export tool (available in the ISIS Utils menu).</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Users of previous CDS/ISIS versions (i.e, 1.3 or before), export documents data from CDS/ISIS and save it as Common Communication Format (CCF), which is a standard based on ISO 2709. Import this ISO data into CDS/ISIS version 1.4 or later. This step is necessary to export the data as XML, the old versions of ISIS do not have support for exporting document data as XML directly. <i><span class="remark">FIXME: ISIS 1.3.x support exporting XML?</span></i></p></div></li><li><p>Then I wrote a python script (shown below) to convert XML to tab delimited text format.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>This step is not necessary, but you can do some corrections in data quickly.</p></div><div class="tip" style="margin-left: 0.5in; margin-
right: 0.5in;"><h3 class="title">Tip</h3><p>Use Emacs macro or Vim recording for quick parsing.</p></div><pre class="programlisting">## ciddtf.py : Convert ISIS Document Database to Text File
##Before Parsing :
##Change RECORDX\xc3\x91_5\xc3\xaf\x03\x12 to RECORD
##Change & to and
##fix problems of <Tag_97> manualy.
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
# Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/docs.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
record_file = open('../outs/docs.txt', 'w')
tag = ['Tag_4', 'Tag_7', 'Tag_11', 'Tag_12', 'Tag_13', 'Tag_18', 'Tag_19', 'Tag_20', 'Tag_21', 'Tag_24', 'Tag_38', 'Tag_10', 'Tag_114', 'Tag_115', 'Tag_110', 'Tag_5', 'Tag_122', 'Tag_97', 'Tag_151', 'Tag_153', 'Tag_123']
for j in record:
if j.has_key('Tag_691'):
record_file.write(j['Tag_691'])
##Check docs without 114 tag
## if j.has_key('Tag_114'):
## pass
## else:
## print j['Tag_691']
else:
record_file.write('_B_L_A_N_K_')
for h in tag:
if j.has_key(h):
record_file.write('\t'+j[h])
else:
record_file.write('\t_B_L_A_N_K_')
record_file.write('\n')</pre><p>Here is a sample XML:</p><div class="literallayout"><p><?xml version="1.0" encoding="ISO-8859-1"?><br>
<!DOCTYPE DATABASE_SJCLDB SYSTEM "SJCLDB.dtd"><br>
<DATABASE_SJCLDB><br>
<RECORD><br>
<Tag_691>43009</Tag_691><br>
<Tag_4>Chiriyute pithamahan: V K N Krutikalekkurichulla paddhanangal</Tag_4><br>
<Tag_11>DC Books</Tag_11><br>
<Tag_12>Kottayam</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Roy</Tag_18><br>
<Tag_19>P C</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>134p</Tag_10><br>
<Tag_114>894.37 Roy</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>GL-M3</Tag_110><br>
<Tag_122>2000-03-28</Tag_122><br>
<Tag_151>Rs.40.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>43018</Tag_691><br>
<Tag_4>Edanadan pattu</Tag_4><br>
<Tag_11>Current Books</Tag_11><br>
<Tag_12>Kottayam</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Sajitha</Tag_18><br>
<Tag_19>K R</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>160p</Tag_10><br>
<Tag_114>398.87 Saj</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>GL-3</Tag_110><br>
<Tag_122>2000-03-28</Tag_122><br>
<Tag_151>Rs.50.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>38980</Tag_691><br>
<Tag_4>Plays</Tag_4><br>
<Tag_11>EWP</Tag_11><br>
<Tag_12>New York</Tag_12><br>
<Tag_13>1989</Tag_13><br>
<Tag_18>O'Neill</Tag_18><br>
<Tag_19>Eugene</Tag_19><br>
<Tag_114>O7,2 One</Tag_114><br>
<Tag_115>M9.1</Tag_115><br>
<Tag_122>1996-06-27</Tag_122><br>
</RECORD><br>
<RECORD><br>
<Tag_691>44579</Tag_691><br>
<Tag_4>Arangukaanatha Nadan</Tag_4><br>
<Tag_11>Current</Tag_11><br>
<Tag_12>Thrissur</Tag_12><br>
<Tag_13>1998</Tag_13><br>
<Tag_18>Thikkodian</Tag_18><br>
<Tag_10>396p</Tag_10><br>
<Tag_114>894.8 Thi/Ara</Tag_114><br>
<Tag_115>N8</Tag_115><br>
<Tag_110>GL-M8</Tag_110><br>
<Tag_122>2001-06-19</Tag_122><br>
<Tag_151>Rs.135.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>16469</Tag_691><br>
<Tag_4>Sartre: A collection of critical essays</Tag_4><br>
<Tag_11>Prentice Hall</Tag_11><br>
<Tag_12>USA</Tag_12><br>
<Tag_13>1965</Tag_13><br>
<Tag_18>Kern</Tag_18><br>
<Tag_19>Edith</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>180p</Tag_10><br>
<Tag_114>O-,4 M79</Tag_114><br>
<Tag_115>J5</Tag_115><br>
<Tag_110>Twentieth century views</Tag_110><br>
<Tag_122>1998-09-26</Tag_122><br>
</RECORD><br>
<RECORD><br>
<Tag_691>42068</Tag_691><br>
<Tag_4>Internet for beginners</Tag_4><br>
<Tag_11>Icon Books</Tag_11><br>
<Tag_12>Australia</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Brunner</Tag_18><br>
<Tag_19>laurel</Tag_19><br>
<Tag_20>Jevtic</Tag_20><br>
<Tag_21>Zoran</Tag_21><br>
<Tag_10>175p</Tag_10><br>
<Tag_115>N7</Tag_115><br>
<Tag_122>1998/09/26</Tag_122><br>
<Tag_123>CPYR</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>42367</Tag_691><br>
<Tag_4>Text book of Biochemistry with clinical correlations</Tag_4><br>
<Tag_7>Ed.4</Tag_7><br>
<Tag_11>Wiley-Liss/John Wiley and Sons</Tag_11><br>
<Tag_12>New York</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Devlin</Tag_18><br>
<Tag_19>Thomas M</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>1186p</Tag_10><br>
<Tag_114>612.015 Dev/KD</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>Zoology</Tag_110><br>
<Tag_122>1998-10-08</Tag_122><br>
<Tag_97>$.30.95</Tag_97><br>
</RECORD><br>
</DATABASE_SJCLDB> </p></div></li><li><p>Now parse the tab separated file using a perl script (shown below) and convert it as MARC 21 or UNIMARC.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Before doing this last step, please study Stephen Hedges' migration guide here: <a href="http://www.skemotah.com/Migrating_to_Koha.html" target="_top">http://www.skemotah.com/Migrating_to_Koha.html</a></p></div><pre class="programlisting">package cmftf;
## Create MARC From Text File
## Copyright 2004 Baiju M <baijum81 at hailmail.net>
## This program is licensed under GNU GPL.
use strict;
use MARC::Record;
my $input_file = "../outs/docs.txt";
my $output_file = "../outs/docs.out";
my $repeated_items = " ";
open(INFILE, $input_file) or die "Can't open $input_file: $!";
open(OUTFILE, ">>", $output_file) or die "Can't open $output_file: $!";
my $c = 0;
while (<INFILE>) {
$c++;
my $biblionumber_ftf_691_090_c, my $title_ftf_4_245_a, my $edition_ftf_7_250_a, my $publisher_ftf_11_260_b, my $publishcountry_ftf_12_260_a, my $publishyear_ftf_13_260_c, my $author1ee_ftf_18_100_a, my $author1se_ftf_19_100_a, my $author2ee_ftf_20_700_a, my $author2se_ftf_21_700_a, my $adiauthors_ftf_24_700_a, my $role_ftf_38_590_a, my $phydescr_ftf_10_300_a, my $classno_ftf_114_852_k, my $classno_ftf_115_852_h, my $keyword_ftf_5_520_a, my $voldetails_ftf_110_440_v, my $dateofentry_ftf_122_952_v, my $notes_ftf_97_500_a, my $pubprice_151_952_r, my $currency_153_590_b, my $reccreated_123_508_a, my $remainder;
($biblionumber_ftf_691_090_c, $title_ftf_4_245_a, $edition_ftf_7_250_a, $publisher_ftf_11_260_b, $publishcountry_ftf_12_260_a, $publishyear_ftf_13_260_c, $author1ee_ftf_18_100_a, $author1se_ftf_19_100_a, $author2ee_ftf_20_700_a, $author2se_ftf_21_700_a, $adiauthors_ftf_24_700_a, $role_ftf_38_590_a, $phydescr_ftf_10_300_a, $classno_ftf_114_852_k, $classno_ftf_115_852_h, $keyword_ftf_5_520_a, $voldetails_ftf_110_440_v, $dateofentry_ftf_122_952_v, $notes_ftf_97_500_a, $pubprice_151_952_r, $currency_153_590_b, $reccreated_123_508_a, $remainder) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c !~ /[^\d]/) and ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) and ($repeated_items !~ /$biblionumber_ftf_691_090_c/) ) {
$repeated_items .= " $biblionumber_ftf_691_090_c";
my @biblionumber_array;
open(INF, $input_file) or die "Can't open $input_file: $!";
my $i = 1;
while (<INF>) {
$i++;
if ($i >= $c) {
my $biblionumber_ftf_691_090_c2, my $title_ftf_4_245_a2, my $edition_ftf_7_250_a2, my $publisher_ftf_11_260_b2, my $publishcountry_ftf_12_260_a2, my $publishyear_ftf_13_260_c2, my $author1ee_ftf_18_100_a2, my $author1se_ftf_19_100_a2, my $author2ee_ftf_20_700_a2, my $author2se_ftf_21_700_a2, my $adiauthors_ftf_24_700_a2, my $role_ftf_38_590_a2, my $phydescr_ftf_10_300_a2, my $classno_ftf_114_852_k2, my $classno_ftf_115_852_h2, my $keyword_ftf_5_520_a2, my $voldetails_ftf_110_440_v2, my $dateofentry_ftf_122_952_v2, my $notes_ftf_97_500_a2, my $pubprice_151_952_r2, my $currency_153_590_b2, my $reccreated_123_508_a2, my $remainder2;
($biblionumber_ftf_691_090_c2, $title_ftf_4_245_a2, $edition_ftf_7_250_a2, $publisher_ftf_11_260_b2, $publishcountry_ftf_12_260_a2, $publishyear_ftf_13_260_c2, $author1ee_ftf_18_100_a2, $author1se_ftf_19_100_a2, $author2ee_ftf_20_700_a2, $author2se_ftf_21_700_a2, $adiauthors_ftf_24_700_a2, $role_ftf_38_590_a2, $phydescr_ftf_10_300_a2, $classno_ftf_114_852_k2, $classno_ftf_115_852_h2, $keyword_ftf_5_520_a2, $voldetails_ftf_110_440_v2, $dateofentry_ftf_122_952_v2, $notes_ftf_97_500_a2, $pubprice_151_952_r2, $currency_153_590_b2, $reccreated_123_508_a2, $remainder2) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c2 !~ /[^\d]/) and ($title_ftf_4_245_a2 !~ /_B_L_A_N_K_/) and ($title_ftf_4_245_a2 eq $title_ftf_4_245_a) and ($author1ee_ftf_18_100_a2 eq $author1ee_ftf_18_100_a)){
print "$biblionumber_ftf_691_090_c:$biblionumber_ftf_691_090_c2\t";
$repeated_items .= " $biblionumber_ftf_691_090_c2";
push @biblionumber_array, $biblionumber_ftf_691_090_c2;
}
}
}
print "\n";
close(INF);
my $record = MARC::Record->new();
if ($author1ee_ftf_18_100_a =~ /_B_L_A_N_K_/) {
$author1ee_ftf_18_100_a = "";
}
if ($author1se_ftf_19_100_a =~ /_B_L_A_N_K_/) {
$author1se_ftf_19_100_a = "";
}
my $author1_ftf = join(" ", $author1se_ftf_19_100_a, $author1ee_ftf_18_100_a);
if ($author1_ftf ne "") {
my $author1 = MARC::Field->new(
'100','1','',
a => $author1_ftf
);
$record->add_fields($author1);
}
if ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) {
my $title = MARC::Field->new(
'245','1','2',
a => $title_ftf_4_245_a
);
$record->add_fields($title);
}
if ($edition_ftf_7_250_a !~ /_B_L_A_N_K_/) {
my $edition = MARC::Field->new(
'250','','',
a => $edition_ftf_7_250_a
);
$record->add_fields($edition);
}
if ($publishcountry_ftf_12_260_a !~ /_B_L_A_N_K_/) {
my $publishcountry = MARC::Field->new(
'260','','',
a => $publishcountry_ftf_12_260_a
);
$record->add_fields($publishcountry);
}
if ($publisher_ftf_11_260_b =~ /_B_L_A_N_K_/) {
$publisher_ftf_11_260_b = "";
}
if ($publishyear_ftf_13_260_c =~ /_B_L_A_N_K_/) {
$publishyear_ftf_13_260_c = "";
}
my $publisher = MARC::Field->new(
'260','','',
b => $publisher_ftf_11_260_b,
c => $publishyear_ftf_13_260_c
);
$record->add_fields($publisher);
if ($phydescr_ftf_10_300_a =~ /_B_L_A_N_K_/) {
$phydescr_ftf_10_300_a = "";
}
my $phydescr = MARC::Field->new(
'300','','',
a => $phydescr_ftf_10_300_a,
f => 'BOOK'
);
$record->append_fields($phydescr);
if ($voldetails_ftf_110_440_v !~ /_B_L_A_N_K_/) {
my $voldetails = MARC::Field->new(
'440','','3',
v => $voldetails_ftf_110_440_v
);
$record->add_fields($voldetails);
}
if ($notes_ftf_97_500_a !~ /_B_L_A_N_K_/) {
my $notes = MARC::Field->new(
'500','','',
a => $notes_ftf_97_500_a
);
$record->add_fields($notes);
}
if ($reccreated_123_508_a !~ /_B_L_A_N_K_/) {
my $reccreated = MARC::Field->new(
'508','','',
a => $reccreated_123_508_a
);
$record->add_fields($reccreated);
}
if ($keyword_ftf_5_520_a !~ /_B_L_A_N_K_/) {
my $keyword = MARC::Field->new(
'520','','',
a => $keyword_ftf_5_520_a
);
$record->add_fields($keyword);
}
if ($role_ftf_38_590_a =~ /_B_L_A_N_K_/) {
$role_ftf_38_590_a = '';
}
if ($currency_153_590_b =~ /_B_L_A_N_K_/) {
$currency_153_590_b = '';
}
my $role = MARC::Field->new(
'590','','',
a => $role_ftf_38_590_a,
b => $currency_153_590_b
);
$record->add_fields($role);
if ($author2ee_ftf_20_700_a =~ /_B_L_A_N_K_/) {
$author2ee_ftf_20_700_a = "";
}
if ($author2se_ftf_21_700_a =~ /_B_L_A_N_K_/) {
$author2se_ftf_21_700_a = "";
}
if ($adiauthors_ftf_24_700_a =~ /_B_L_A_N_K_/) {
$adiauthors_ftf_24_700_a = "";
}
my $adiauthors_ftf = join(" ", $author2se_ftf_21_700_a, $author2ee_ftf_20_700_a, $adiauthors_ftf_24_700_a);
if ($adiauthors_ftf ne "") {
my $adiauthors = MARC::Field->new(
'700','1','',
a => $adiauthors_ftf
);
$record->add_fields($adiauthors);
}
if ($classno_ftf_114_852_k =~ /_B_L_A_N_K_/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
my $classno = MARC::Field->new(
'852','','',
k => '999.9999'
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
my $classno = MARC::Field->new(
'852','','',
k => '999.9998',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
else {
if ($classno_ftf_114_852_k =~ /^\d\d\d/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_114_852_k,
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h,
h => $classno_ftf_114_852_k
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => '999.9997',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
if ($pubprice_151_952_r =~ /_B_L_A_N_K_/) {
$pubprice_151_952_r = '';
}
if ($dateofentry_ftf_122_952_v =~ /_B_L_A_N_K_/) {
$dateofentry_ftf_122_952_v = '';
}
my @biblionumber_array_tmp = sort @biblionumber_array;
foreach (@biblionumber_array_tmp) {
my $biblionumber = $_;
my $barcode = MARC::Field->new(
'952','','',
b => "MAIN",
d => "MAIN",
p => $biblionumber,
r => $pubprice_151_952_r,
u => $biblionumber,
v => $dateofentry_ftf_122_952_v
);
$record->add_fields($barcode);
}
#print "\n";
print OUTFILE $record->as_usmarc();
}
else {
#print $_;
}
}
close(OUTFILE);</pre></li><li><p>And finally use koha's <tt class="filename">bulkmarcimport.pl</tt> script to convert this to SQL.</p></li></ul></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e130"></a>5. Migrating Member Data</h2></div></div><div></div></div><div class="itemizedlist"><ul type="disc"><li><p>To convert member details to Koha, first I exported member database into XML, then wrote a python script to convert it to tab separated text file:</p><pre class="programlisting">##cimdtf.py : Convert ISIS Member Database to Text File
##Before Parsing :
##Change & to and
##fix problems of <Tag_*> manualy.
##tag = ['Tag_201', 'Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
##Tag_201 : Name -> firstname
##Tag_202 : Membership number -> cardnumber
##Tag_203 : Division -> streetaddress
##Tag_216 : Status (UG,PG, etc.) -> categorycode
##Tag_217 : Joining date -> dateenrolled
## CREATE TABLE borrowers (
##01 borrowernumber int(11) NOT NULL auto_increment,
##02 cardnumber varchar(16) NOT NULL default '', ##Tag_202
##03 surname text NOT NULL, ##Tag_201
##04 firstname text NOT NULL, ##' '
##05 title text, ##' '
##06 othernames text, ##''
##07 initials text NOT NULL, ##''
##08 streetaddress text NOT NULL, ##Tag_203
##09 suburb text, ##NULL
##10 city text NOT NULL, ##Tag_203 (Use this?)
##11 phone text NOT NULL, ##''
##12 emailaddress text, ##''
##13 faxnumber text, ##''
##14 textmessaging text, ##''
##15 altstreetaddress text, ##NULL
##16 altsuburb text, ##NULL
##17 altcity text, ##NULL
##18 altphone text, ##''
##19 dateofbirth date default NULL, ##'0000-00-00'
##20 branchcode varchar(4) NOT NULL default '', ##'MAIN'
##21 categorycode char(2) default NULL, ##Tag_216
##22 dateenrolled date default NULL, ##Tag_217
##23 gonenoaddress tinyint(1) default NULL, ##NULL
##24 lost tinyint(1) default NULL, ##NULL
##25 debarred tinyint(1) default NULL, ##NULL
##26 studentnumber text, ##NULL
##27 school text, #NULL
##28 contactname text, ##' '
##29 borrowernotes text, ##''
##30 guarantor int(11) default NULL, ##NULL
##31 area char(2) default NULL, ##'' or NULL
##32 ethnicity varchar(50) default NULL, ##'' or NULL
##33 ethnotes varchar(255) default NULL, ##'' or NULL
##34 sex char(1) default NULL, ##NULL
##35 expiry date default NULL, ##'0000-00-00'
##36 altnotes varchar(255) default NULL, ##''
##37 altrelationship varchar(100) default NULL, ##'workplace'
##38 streetcity text, ##''
##39 phoneday varchar(50) default NULL, ##''
##40 preferredcont char(1) default NULL, ##NULL
##41 physstreet varchar(100) default NULL, ##''
##42 password varchar(30) default NULL, ##NULL
##43 flags int(11) default NULL, ##NULL
##44 userid varchar(30) default NULL, ##NULL
##45 homezipcode varchar(25) default NULL, ##''
##46 zipcode varchar(25) default NULL, ##''
##47 UNIQUE KEY cardnumber (cardnumber),
##48 KEY borrowernumber (borrowernumber)
## ) ;
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
## Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/members.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
output_file = open('../outs/members.txt', 'w')
tag = ['Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
for j in record:
if j.has_key('Tag_202') and j.has_key('Tag_201') and j.has_key('Tag_216'):
output_file.write('\t') ##01
output_file.write(j['Tag_202']+'\t') ##02
output_file.write(j['Tag_201']+'\t') ##03
output_file.write(' \t') ##04
output_file.write(' \t') ##05
output_file.write('\t') ##06
output_file.write('\t') ##07
if j.has_key('Tag_203'): ##08
output_file.write(j['Tag_203']+'\t')
else:
output_file.write('\t')
output_file.write('NULL\t') ##09
output_file.write('\t') ##10
output_file.write('\t') ##11
output_file.write('\t') ##12
output_file.write('\t') ##13
output_file.write('\t') ##14
output_file.write('NULL\t') ##15
output_file.write('NULL\t') ##16
output_file.write('NULL\t') ##17
output_file.write('\t') ##18
output_file.write('0000-00-00\t') ##19
output_file.write('MAIN\t') ##20
if j.has_key('Tag_216'): ##21
output_file.write(j['Tag_216']+'\t')
else:
output_file.write('OT\t')
pass
if j.has_key('Tag_217'): ##22
output_file.write(j['Tag_217']+'\t')
else:
output_file.write('0000-00-00\t')
pass
output_file.write('NULL\t') ##23
output_file.write('NULL\t') ##24
output_file.write('NULL\t') ##25
output_file.write('NULL\t') ##26
output_file.write('NULL\t') ##27
output_file.write(' \t') ##28
output_file.write('\t') ##29
output_file.write('NULL\t') ##30
output_file.write('NULL\t') ##31
output_file.write('NULL\t') ##32
output_file.write('NULL\t') ##33
output_file.write('M\t') ##34
output_file.write('0000-00-00\t') ##35
output_file.write('NULL\t') ##36
output_file.write('workplace\t') ##37
output_file.write('NULL\t') ##38
output_file.write('NULL\t') ##39
output_file.write('NULL\t') ##40
output_file.write('NULL\t') ##41
output_file.write('NULL\t') ##42
output_file.write('NULL\t') ##43
output_file.write('NULL\t') ##44
output_file.write('NULL\t') ##45
output_file.write('NULL\t') ##46
output_file.write('\n')</pre></li><li><p>Later to load it into MySQL:</p><p><b class="userinput"><tt>LOAD DATA LOCAL INFILE "members.txt" INTO TABLE borrowers;</tt></b></p></li></ul></div></div></div></body></html>
--- NEW FILE ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
<title>Migrating a CDS/ISIS based system to Koha</title>
<articleinfo>
<pubdate>2004-11-16</pubdate>
<author>
<firstname>Baiju</firstname>
<surname>M</surname>
<email>baijum81 AT hailmail.net</email>
</author>
<revhistory>
<revision>
<revnumber>2.0.0p1</revnumber>
<date>2004-11-15</date>
<authorinitials>bm</authorinitials>
<revdescription>
<para>Small additions by BM.</para>
</revdescription>
</revision>
<revision>
<revnumber>2.0.0</revnumber>
<date>2004-04-01</date>
<authorinitials>bm</authorinitials>
<revdescription>
<para>Initial XML markup (by Stephen Hedges) of a wiki page posted
at <ulink
url="http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha">http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha</ulink>
on 2004-04-01, with links to program listings expanded.</para>
</revdescription>
</revision>
</revhistory>
</articleinfo>
<authorblurb>
<para>By Baiju M</para>
</authorblurb>
<section>
<title id="copyright">Copyright and License</title>
<para>Copyright 2004 Baiju M <email>baijum81 AT
hailmail.net</email></para>
<para>This document is related to Koha and is licensed to you under the
GNU General Public License version 2 or later (<ulink
url="http://www.gnu.org/licenses/gpl.html">http://www.gnu.org/licenses/gpl.html</ulink>).</para>
<para>Koha-related documents may be reproduced and distributed in whole or
in part, in any medium physical or electronic, as long as this copyright
notice is retained on all copies.</para>
<para>You may create a derivative work and distribute it provided that
you:</para>
<orderedlist>
<listitem>
<para>License the derivative work with this same license, or the Linux
Documentation Project License (<ulink
url="???">http://www.tldp.org/COPYRIGHT.html</ulink>). Include a
copyright notice and at least a pointer to the license used.</para>
</listitem>
<listitem>
<para>Give due credit to previous authors and major
contributors.</para>
</listitem>
</orderedlist>
<para>Commercial redistribution is allowed and encouraged; however, the
author would like to be notified of any such distributions.</para>
</section>
<section>
<title id="disclaimer">Disclaimer</title>
<para>No liability for the contents of this document can be accepted. Use
the concepts, examples and information at your own risk. There may be
errors and inaccuracies that could be damaging to your system. Although
this is highly unlikely, proceed with caution; the author(s) do not take
any responsibility.</para>
<para>All copyrights are held by their respective owners, unless
specifically noted otherwise. Use of a term in this document should not be
regarded as affecting the validity of any trademark or service mark.
Naming of particular products or brands should not be seen as
endorsements.</para>
</section>
<section>
<title>Introduction</title>
<para>I am doing this migration from CDS/ISIS (version : 1.5) system to
Koha for St. Joseph's College, Devagiri, Calicut. <ulink
url="http://devagiricollege.org/">http://devagiricollege.org/</ulink></para>
<para>In the current system it is possible to do circulation and other
basic functionalities of the library.</para>
<para>The work is in progress, so give any pointers if you have.</para>
</section>
<section>
<title>Migrating Book Data</title>
<itemizedlist>
<listitem>
<para>First I exported Books data into XML using CDS/ISIS export tool
(available in the ISIS Utils menu).</para>
<note>
<para>Users of previous CDS/ISIS versions (i.e, 1.3 or before),
export documents data from CDS/ISIS and save it as Common
Communication Format (CCF), which is a standard based on ISO 2709.
Import this ISO data into CDS/ISIS version 1.4 or later. This step
is necessary to export the data as XML, the old versions of ISIS do
not have support for exporting document data as XML directly.
<remark>FIXME: ISIS 1.3.x support exporting XML?</remark></para>
</note>
</listitem>
<listitem>
<para>Then I wrote a python script (shown below) to convert XML to tab
delimited text format.<note>
<para>This step is not necessary, but you can do some corrections
in data quickly.</para>
</note><tip>
<para>Use Emacs macro or Vim recording for quick parsing.</para>
</tip><programlisting>## ciddtf.py : Convert ISIS Document Database to Text File
##Before Parsing :
##Change RECORDX\xc3\x91_5\xc3\xaf\x03\x12 to RECORD
##Change & to and
##fix problems of <Tag_97> manualy.
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
# Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/docs.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
record_file = open('../outs/docs.txt', 'w')
tag = ['Tag_4', 'Tag_7', 'Tag_11', 'Tag_12', 'Tag_13', 'Tag_18', 'Tag_19', 'Tag_20', 'Tag_21', 'Tag_24', 'Tag_38', 'Tag_10', 'Tag_114', 'Tag_115', 'Tag_110', 'Tag_5', 'Tag_122', 'Tag_97', 'Tag_151', 'Tag_153', 'Tag_123']
for j in record:
if j.has_key('Tag_691'):
record_file.write(j['Tag_691'])
##Check docs without 114 tag
## if j.has_key('Tag_114'):
## pass
## else:
## print j['Tag_691']
else:
record_file.write('_B_L_A_N_K_')
for h in tag:
if j.has_key(h):
record_file.write('\t'+j[h])
else:
record_file.write('\t_B_L_A_N_K_')
record_file.write('\n')</programlisting></para>
<para>Here is a sample XML:<literallayout><?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE DATABASE_SJCLDB SYSTEM "SJCLDB.dtd">
<DATABASE_SJCLDB>
<RECORD>
<Tag_691>43009</Tag_691>
<Tag_4>Chiriyute pithamahan: V K N Krutikalekkurichulla paddhanangal</Tag_4>
<Tag_11>DC Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Roy</Tag_18>
<Tag_19>P C</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>134p</Tag_10>
<Tag_114>894.37 Roy</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-M3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.40.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>43018</Tag_691>
<Tag_4>Edanadan pattu</Tag_4>
<Tag_11>Current Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Sajitha</Tag_18>
<Tag_19>K R</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>160p</Tag_10>
<Tag_114>398.87 Saj</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.50.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>38980</Tag_691>
<Tag_4>Plays</Tag_4>
<Tag_11>EWP</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1989</Tag_13>
<Tag_18>O'Neill</Tag_18>
<Tag_19>Eugene</Tag_19>
<Tag_114>O7,2 One</Tag_114>
<Tag_115>M9.1</Tag_115>
<Tag_122>1996-06-27</Tag_122>
</RECORD>
<RECORD>
<Tag_691>44579</Tag_691>
<Tag_4>Arangukaanatha Nadan</Tag_4>
<Tag_11>Current</Tag_11>
<Tag_12>Thrissur</Tag_12>
<Tag_13>1998</Tag_13>
<Tag_18>Thikkodian</Tag_18>
<Tag_10>396p</Tag_10>
<Tag_114>894.8 Thi/Ara</Tag_114>
<Tag_115>N8</Tag_115>
<Tag_110>GL-M8</Tag_110>
<Tag_122>2001-06-19</Tag_122>
<Tag_151>Rs.135.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>16469</Tag_691>
<Tag_4>Sartre: A collection of critical essays</Tag_4>
<Tag_11>Prentice Hall</Tag_11>
<Tag_12>USA</Tag_12>
<Tag_13>1965</Tag_13>
<Tag_18>Kern</Tag_18>
<Tag_19>Edith</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>180p</Tag_10>
<Tag_114>O-,4 M79</Tag_114>
<Tag_115>J5</Tag_115>
<Tag_110>Twentieth century views</Tag_110>
<Tag_122>1998-09-26</Tag_122>
</RECORD>
<RECORD>
<Tag_691>42068</Tag_691>
<Tag_4>Internet for beginners</Tag_4>
<Tag_11>Icon Books</Tag_11>
<Tag_12>Australia</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Brunner</Tag_18>
<Tag_19>laurel</Tag_19>
<Tag_20>Jevtic</Tag_20>
<Tag_21>Zoran</Tag_21>
<Tag_10>175p</Tag_10>
<Tag_115>N7</Tag_115>
<Tag_122>1998/09/26</Tag_122>
<Tag_123>CPYR</Tag_123>
</RECORD>
<RECORD>
<Tag_691>42367</Tag_691>
<Tag_4>Text book of Biochemistry with clinical correlations</Tag_4>
<Tag_7>Ed.4</Tag_7>
<Tag_11>Wiley-Liss/John Wiley and Sons</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Devlin</Tag_18>
<Tag_19>Thomas M</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>1186p</Tag_10>
<Tag_114>612.015 Dev/KD</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>Zoology</Tag_110>
<Tag_122>1998-10-08</Tag_122>
<Tag_97>$.30.95</Tag_97>
</RECORD>
</DATABASE_SJCLDB> </literallayout></para>
</listitem>
<listitem>
<para>Now parse the tab separated file using a perl script (shown
below) and convert it as MARC 21 or UNIMARC.<note>
<para>Before doing this last step, please study Stephen Hedges'
migration guide here: <ulink
url="http://www.skemotah.com/Migrating_to_Koha.html">http://www.skemotah.com/Migrating_to_Koha.html</ulink></para>
</note><programlisting>package cmftf;
## Create MARC From Text File
## Copyright 2004 Baiju M <baijum81 at hailmail.net>
## This program is licensed under GNU GPL.
use strict;
use MARC::Record;
my $input_file = "../outs/docs.txt";
my $output_file = "../outs/docs.out";
my $repeated_items = " ";
open(INFILE, $input_file) or die "Can't open $input_file: $!";
open(OUTFILE, ">>", $output_file) or die "Can't open $output_file: $!";
my $c = 0;
while (<INFILE>) {
$c++;
my $biblionumber_ftf_691_090_c, my $title_ftf_4_245_a, my $edition_ftf_7_250_a, my $publisher_ftf_11_260_b, my $publishcountry_ftf_12_260_a, my $publishyear_ftf_13_260_c, my $author1ee_ftf_18_100_a, my $author1se_ftf_19_100_a, my $author2ee_ftf_20_700_a, my $author2se_ftf_21_700_a, my $adiauthors_ftf_24_700_a, my $role_ftf_38_590_a, my $phydescr_ftf_10_300_a, my $classno_ftf_114_852_k, my $classno_ftf_115_852_h, my $keyword_ftf_5_520_a, my $voldetails_ftf_110_440_v, my $dateofentry_ftf_122_952_v, my $notes_ftf_97_500_a, my $pubprice_151_952_r, my $currency_153_590_b, my $reccreated_123_508_a, my $remainder;
($biblionumber_ftf_691_090_c, $title_ftf_4_245_a, $edition_ftf_7_250_a, $publisher_ftf_11_260_b, $publishcountry_ftf_12_260_a, $publishyear_ftf_13_260_c, $author1ee_ftf_18_100_a, $author1se_ftf_19_100_a, $author2ee_ftf_20_700_a, $author2se_ftf_21_700_a, $adiauthors_ftf_24_700_a, $role_ftf_38_590_a, $phydescr_ftf_10_300_a, $classno_ftf_114_852_k, $classno_ftf_115_852_h, $keyword_ftf_5_520_a, $voldetails_ftf_110_440_v, $dateofentry_ftf_122_952_v, $notes_ftf_97_500_a, $pubprice_151_952_r, $currency_153_590_b, $reccreated_123_508_a, $remainder) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c !~ /[^\d]/) and ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) and ($repeated_items !~ /$biblionumber_ftf_691_090_c/) ) {
$repeated_items .= " $biblionumber_ftf_691_090_c";
my @biblionumber_array;
open(INF, $input_file) or die "Can't open $input_file: $!";
my $i = 1;
while (<INF>) {
$i++;
if ($i >= $c) {
my $biblionumber_ftf_691_090_c2, my $title_ftf_4_245_a2, my $edition_ftf_7_250_a2, my $publisher_ftf_11_260_b2, my $publishcountry_ftf_12_260_a2, my $publishyear_ftf_13_260_c2, my $author1ee_ftf_18_100_a2, my $author1se_ftf_19_100_a2, my $author2ee_ftf_20_700_a2, my $author2se_ftf_21_700_a2, my $adiauthors_ftf_24_700_a2, my $role_ftf_38_590_a2, my $phydescr_ftf_10_300_a2, my $classno_ftf_114_852_k2, my $classno_ftf_115_852_h2, my $keyword_ftf_5_520_a2, my $voldetails_ftf_110_440_v2, my $dateofentry_ftf_122_952_v2, my $notes_ftf_97_500_a2, my $pubprice_151_952_r2, my $currency_153_590_b2, my $reccreated_123_508_a2, my $remainder2;
($biblionumber_ftf_691_090_c2, $title_ftf_4_245_a2, $edition_ftf_7_250_a2, $publisher_ftf_11_260_b2, $publishcountry_ftf_12_260_a2, $publishyear_ftf_13_260_c2, $author1ee_ftf_18_100_a2, $author1se_ftf_19_100_a2, $author2ee_ftf_20_700_a2, $author2se_ftf_21_700_a2, $adiauthors_ftf_24_700_a2, $role_ftf_38_590_a2, $phydescr_ftf_10_300_a2, $classno_ftf_114_852_k2, $classno_ftf_115_852_h2, $keyword_ftf_5_520_a2, $voldetails_ftf_110_440_v2, $dateofentry_ftf_122_952_v2, $notes_ftf_97_500_a2, $pubprice_151_952_r2, $currency_153_590_b2, $reccreated_123_508_a2, $remainder2) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c2 !~ /[^\d]/) and ($title_ftf_4_245_a2 !~ /_B_L_A_N_K_/) and ($title_ftf_4_245_a2 eq $title_ftf_4_245_a) and ($author1ee_ftf_18_100_a2 eq $author1ee_ftf_18_100_a)){
print "$biblionumber_ftf_691_090_c:$biblionumber_ftf_691_090_c2\t";
$repeated_items .= " $biblionumber_ftf_691_090_c2";
push @biblionumber_array, $biblionumber_ftf_691_090_c2;
}
}
}
print "\n";
close(INF);
my $record = MARC::Record->new();
if ($author1ee_ftf_18_100_a =~ /_B_L_A_N_K_/) {
$author1ee_ftf_18_100_a = "";
}
if ($author1se_ftf_19_100_a =~ /_B_L_A_N_K_/) {
$author1se_ftf_19_100_a = "";
}
my $author1_ftf = join(" ", $author1se_ftf_19_100_a, $author1ee_ftf_18_100_a);
if ($author1_ftf ne "") {
my $author1 = MARC::Field->new(
'100','1','',
a => $author1_ftf
);
$record->add_fields($author1);
}
if ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) {
my $title = MARC::Field->new(
'245','1','2',
a => $title_ftf_4_245_a
);
$record->add_fields($title);
}
if ($edition_ftf_7_250_a !~ /_B_L_A_N_K_/) {
my $edition = MARC::Field->new(
'250','','',
a => $edition_ftf_7_250_a
);
$record->add_fields($edition);
}
if ($publishcountry_ftf_12_260_a !~ /_B_L_A_N_K_/) {
my $publishcountry = MARC::Field->new(
'260','','',
a => $publishcountry_ftf_12_260_a
);
$record->add_fields($publishcountry);
}
if ($publisher_ftf_11_260_b =~ /_B_L_A_N_K_/) {
$publisher_ftf_11_260_b = "";
}
if ($publishyear_ftf_13_260_c =~ /_B_L_A_N_K_/) {
$publishyear_ftf_13_260_c = "";
}
my $publisher = MARC::Field->new(
'260','','',
b => $publisher_ftf_11_260_b,
c => $publishyear_ftf_13_260_c
);
$record->add_fields($publisher);
if ($phydescr_ftf_10_300_a =~ /_B_L_A_N_K_/) {
$phydescr_ftf_10_300_a = "";
}
my $phydescr = MARC::Field->new(
'300','','',
a => $phydescr_ftf_10_300_a,
f => 'BOOK'
);
$record->append_fields($phydescr);
if ($voldetails_ftf_110_440_v !~ /_B_L_A_N_K_/) {
my $voldetails = MARC::Field->new(
'440','','3',
v => $voldetails_ftf_110_440_v
);
$record->add_fields($voldetails);
}
if ($notes_ftf_97_500_a !~ /_B_L_A_N_K_/) {
my $notes = MARC::Field->new(
'500','','',
a => $notes_ftf_97_500_a
);
$record->add_fields($notes);
}
if ($reccreated_123_508_a !~ /_B_L_A_N_K_/) {
my $reccreated = MARC::Field->new(
'508','','',
a => $reccreated_123_508_a
);
$record->add_fields($reccreated);
}
if ($keyword_ftf_5_520_a !~ /_B_L_A_N_K_/) {
my $keyword = MARC::Field->new(
'520','','',
a => $keyword_ftf_5_520_a
);
$record->add_fields($keyword);
}
if ($role_ftf_38_590_a =~ /_B_L_A_N_K_/) {
$role_ftf_38_590_a = '';
}
if ($currency_153_590_b =~ /_B_L_A_N_K_/) {
$currency_153_590_b = '';
}
my $role = MARC::Field->new(
'590','','',
a => $role_ftf_38_590_a,
b => $currency_153_590_b
);
$record->add_fields($role);
if ($author2ee_ftf_20_700_a =~ /_B_L_A_N_K_/) {
$author2ee_ftf_20_700_a = "";
}
if ($author2se_ftf_21_700_a =~ /_B_L_A_N_K_/) {
$author2se_ftf_21_700_a = "";
}
if ($adiauthors_ftf_24_700_a =~ /_B_L_A_N_K_/) {
$adiauthors_ftf_24_700_a = "";
}
my $adiauthors_ftf = join(" ", $author2se_ftf_21_700_a, $author2ee_ftf_20_700_a, $adiauthors_ftf_24_700_a);
if ($adiauthors_ftf ne "") {
my $adiauthors = MARC::Field->new(
'700','1','',
a => $adiauthors_ftf
);
$record->add_fields($adiauthors);
}
if ($classno_ftf_114_852_k =~ /_B_L_A_N_K_/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
my $classno = MARC::Field->new(
'852','','',
k => '999.9999'
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
my $classno = MARC::Field->new(
'852','','',
k => '999.9998',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
else {
if ($classno_ftf_114_852_k =~ /^\d\d\d/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_114_852_k,
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h,
h => $classno_ftf_114_852_k
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => '999.9997',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
if ($pubprice_151_952_r =~ /_B_L_A_N_K_/) {
$pubprice_151_952_r = '';
}
if ($dateofentry_ftf_122_952_v =~ /_B_L_A_N_K_/) {
$dateofentry_ftf_122_952_v = '';
}
my @biblionumber_array_tmp = sort @biblionumber_array;
foreach (@biblionumber_array_tmp) {
my $biblionumber = $_;
my $barcode = MARC::Field->new(
'952','','',
b => "MAIN",
d => "MAIN",
p => $biblionumber,
r => $pubprice_151_952_r,
u => $biblionumber,
v => $dateofentry_ftf_122_952_v
);
$record->add_fields($barcode);
}
#print "\n";
print OUTFILE $record->as_usmarc();
}
else {
#print $_;
}
}
close(OUTFILE);</programlisting></para>
</listitem>
<listitem>
<para>And finally use koha's <filename>bulkmarcimport.pl</filename>
script to convert this to SQL.</para>
</listitem>
</itemizedlist>
</section>
<section>
<title>Migrating Member Data</title>
<itemizedlist>
<listitem>
<para>To convert member details to Koha, first I exported member
database into XML, then wrote a python script to convert it to tab
separated text file:<programlisting>##cimdtf.py : Convert ISIS Member Database to Text File
##Before Parsing :
##Change & to and
##fix problems of <Tag_*> manualy.
##tag = ['Tag_201', 'Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
##Tag_201 : Name -> firstname
##Tag_202 : Membership number -> cardnumber
##Tag_203 : Division -> streetaddress
##Tag_216 : Status (UG,PG, etc.) -> categorycode
##Tag_217 : Joining date -> dateenrolled
## CREATE TABLE borrowers (
##01 borrowernumber int(11) NOT NULL auto_increment,
##02 cardnumber varchar(16) NOT NULL default '', ##Tag_202
##03 surname text NOT NULL, ##Tag_201
##04 firstname text NOT NULL, ##' '
##05 title text, ##' '
##06 othernames text, ##''
##07 initials text NOT NULL, ##''
##08 streetaddress text NOT NULL, ##Tag_203
##09 suburb text, ##NULL
##10 city text NOT NULL, ##Tag_203 (Use this?)
##11 phone text NOT NULL, ##''
##12 emailaddress text, ##''
##13 faxnumber text, ##''
##14 textmessaging text, ##''
##15 altstreetaddress text, ##NULL
##16 altsuburb text, ##NULL
##17 altcity text, ##NULL
##18 altphone text, ##''
##19 dateofbirth date default NULL, ##'0000-00-00'
##20 branchcode varchar(4) NOT NULL default '', ##'MAIN'
##21 categorycode char(2) default NULL, ##Tag_216
##22 dateenrolled date default NULL, ##Tag_217
##23 gonenoaddress tinyint(1) default NULL, ##NULL
##24 lost tinyint(1) default NULL, ##NULL
##25 debarred tinyint(1) default NULL, ##NULL
##26 studentnumber text, ##NULL
##27 school text, #NULL
##28 contactname text, ##' '
##29 borrowernotes text, ##''
##30 guarantor int(11) default NULL, ##NULL
##31 area char(2) default NULL, ##'' or NULL
##32 ethnicity varchar(50) default NULL, ##'' or NULL
##33 ethnotes varchar(255) default NULL, ##'' or NULL
##34 sex char(1) default NULL, ##NULL
##35 expiry date default NULL, ##'0000-00-00'
##36 altnotes varchar(255) default NULL, ##''
##37 altrelationship varchar(100) default NULL, ##'workplace'
##38 streetcity text, ##''
##39 phoneday varchar(50) default NULL, ##''
##40 preferredcont char(1) default NULL, ##NULL
##41 physstreet varchar(100) default NULL, ##''
##42 password varchar(30) default NULL, ##NULL
##43 flags int(11) default NULL, ##NULL
##44 userid varchar(30) default NULL, ##NULL
##45 homezipcode varchar(25) default NULL, ##''
##46 zipcode varchar(25) default NULL, ##''
##47 UNIQUE KEY cardnumber (cardnumber),
##48 KEY borrowernumber (borrowernumber)
## ) ;
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
## Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/members.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
output_file = open('../outs/members.txt', 'w')
tag = ['Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
for j in record:
if j.has_key('Tag_202') and j.has_key('Tag_201') and j.has_key('Tag_216'):
output_file.write('\t') ##01
output_file.write(j['Tag_202']+'\t') ##02
output_file.write(j['Tag_201']+'\t') ##03
output_file.write(' \t') ##04
output_file.write(' \t') ##05
output_file.write('\t') ##06
output_file.write('\t') ##07
if j.has_key('Tag_203'): ##08
output_file.write(j['Tag_203']+'\t')
else:
output_file.write('\t')
output_file.write('NULL\t') ##09
output_file.write('\t') ##10
output_file.write('\t') ##11
output_file.write('\t') ##12
output_file.write('\t') ##13
output_file.write('\t') ##14
output_file.write('NULL\t') ##15
output_file.write('NULL\t') ##16
output_file.write('NULL\t') ##17
output_file.write('\t') ##18
output_file.write('0000-00-00\t') ##19
output_file.write('MAIN\t') ##20
if j.has_key('Tag_216'): ##21
output_file.write(j['Tag_216']+'\t')
else:
output_file.write('OT\t')
pass
if j.has_key('Tag_217'): ##22
output_file.write(j['Tag_217']+'\t')
else:
output_file.write('0000-00-00\t')
pass
output_file.write('NULL\t') ##23
output_file.write('NULL\t') ##24
output_file.write('NULL\t') ##25
output_file.write('NULL\t') ##26
output_file.write('NULL\t') ##27
output_file.write(' \t') ##28
output_file.write('\t') ##29
output_file.write('NULL\t') ##30
output_file.write('NULL\t') ##31
output_file.write('NULL\t') ##32
output_file.write('NULL\t') ##33
output_file.write('M\t') ##34
output_file.write('0000-00-00\t') ##35
output_file.write('NULL\t') ##36
output_file.write('workplace\t') ##37
output_file.write('NULL\t') ##38
output_file.write('NULL\t') ##39
output_file.write('NULL\t') ##40
output_file.write('NULL\t') ##41
output_file.write('NULL\t') ##42
output_file.write('NULL\t') ##43
output_file.write('NULL\t') ##44
output_file.write('NULL\t') ##45
output_file.write('NULL\t') ##46
output_file.write('\n')</programlisting></para>
</listitem>
<listitem>
<para>Later to load it into MySQL:</para>
<para><userinput>LOAD DATA LOCAL INFILE "members.txt" INTO TABLE
borrowers;</userinput></para>
</listitem>
</itemizedlist>
</section>
</article>
More information about the Koha-cvs
mailing list