data import question & 008 use in Koha
Hello all, We're in the middle of migrating MARC data to Koha. We tried importing serials data into Koha and a record that had 1900+ items attached did not import. I found the record and tested importing it, removing items until it would import and found out that 1238 was the maximum number of items I could import with one MARC record. Can anyone tell me if there is a way to configure the import to accept more items? I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny. Also, our old ILS has a database issue that results in incomplete export of records created from original cataloging. These records are missing 008 fields. We're working out a method of exporting the data from the back end and splicing it back into the MARC before import, but the data imports into Koha without announced or noticeable errors. I wonder if we're wasting our time. Does Koha use the 008 field for anything? -- Cheers, Christopher Curry Assistant Technical Librarian / Assistant IT Officer American Philosophical Society 105 South Fifth Street Philadelphia, PA 19106-3386 Tel. (215) 599-4299 ccurry@amphilsoc.org <mailto:ccurry@amphilsoc.org> *For technical support, please use helpdesk@amphilsoc.org <mailto:helpdesk@amphilsoc.org>* Main Library number: (215)440-3400 APS website: http://www.amphilsoc.org
Hi,
Also, our old ILS has a database issue that results in incomplete export of records created from original cataloging. These records are missing 008 fields. We're working out a method of exporting the data from the back end and splicing it back into the MARC before import, but the data imports into Koha without announced or noticeable errors. I wonder if we're wasting our time. Does Koha use the 008 field for anything?
I answer on the question about missing 008. I think you can use import without 008 but you loose many indexes based on 008 values. In fact 008 is an important point in a good Marc21 record. If you use Zebra, the definition of indexes are in the file etc/zebradb/marc_defs/marc21/biblios/record.abs The indexes connect with 008 are: date-entered-on-file:n:range(data,0,5),date-entered-on-file:s:range(data,0,5), pubdate:w:range(data,7,4),pubdate:n:range(data,7,4),pubdate:y:range(data,7,4),pubdate:s:range(data,7,4), pl:w:range(data,15,3), ta:w:range(data,22,1), ff8-23:w:range(data,23,1),ff8-29:w:range(data,29,1), lf:w:range(data,33,1), bio:w:range(data,34,1), ln:n:range(data,35,3), ctype:w:range(data,24,4), Record-source:w:range(data,39,0) So I strongly suggest you to insert records with 008. But for test you can insert also data without. Bye Zeno Tajoli CILEA - Segrate (MI) tajoliAT_SPAM_no_prendiATcilea.it (Indirizzo mascherato anti-spam; sostituisci quanto tra AT con @)
2009/11/3 Christopher Curry <ccurry@amphilsoc.org>:
Hello all,
We're in the middle of migrating MARC data to Koha. We tried importing serials data into Koha and a record that had 1900+ items attached did not import. I found the record and tested importing it, removing items until it would import and found out that 1238 was the maximum number of items I could import with one MARC record. Can anyone tell me if there is a way to configure the import to accept more items?
I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.
I think you are running into one of the limitations that the MARC format inhertited from the days when magnetic tape was the cool new thing... "Record length (character positions 00-04 [in the leader]), contains a five-character ASCII numeric string equal to the length of the entire record, including itself and the record terminator. The five-character numeric string is right justified and unused positions contain zeroes (zero fill). ***The maximum length of a record is 99999 octets***." (My attempt at emphasis.) http://www.loc.gov/marc/specifications/specrecstruc.html#genrec And if you have enough items you reach that limit in size. I have this same problem with one of the records i tried to import for my client #1 - my plan is to try and transform the record to MARCXML, which does not impose arbitrary limits on the size of a record, and import that with the "-m MARCXML" option for bulkmarcimport.pl. Regards, Magnus Enger libriotech.no
Thanks, Magnus & Zeno, for your very useful replies. Cheers, Christopher Curry Assistant Technical Librarian / Assistant IT Officer American Philosophical Society 105 South Fifth Street Philadelphia, PA 19106-3386 Tel. (215) 599-4299 ccurry@amphilsoc.org <mailto:ccurry@amphilsoc.org> *For technical support, please use helpdesk@amphilsoc.org <mailto:helpdesk@amphilsoc.org>* Main Library number: (215)440-3400 APS website: http://www.amphilsoc.org Magnus Enger wrote:
2009/11/3 Christopher Curry <ccurry@amphilsoc.org>:
Hello all,
We're in the middle of migrating MARC data to Koha. We tried importing serials data into Koha and a record that had 1900+ items attached did not import. I found the record and tested importing it, removing items until it would import and found out that 1238 was the maximum number of items I could import with one MARC record. Can anyone tell me if there is a way to configure the import to accept more items?
I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.
I think you are running into one of the limitations that the MARC format inhertited from the days when magnetic tape was the cool new thing...
"Record length (character positions 00-04 [in the leader]), contains a five-character ASCII numeric string equal to the length of the entire record, including itself and the record terminator. The five-character numeric string is right justified and unused positions contain zeroes (zero fill). ***The maximum length of a record is 99999 octets***." (My attempt at emphasis.) http://www.loc.gov/marc/specifications/specrecstruc.html#genrec
And if you have enough items you reach that limit in size.
I have this same problem with one of the records i tried to import for my client #1 - my plan is to try and transform the record to MARCXML, which does not impose arbitrary limits on the size of a record, and import that with the "-m MARCXML" option for bulkmarcimport.pl.
Regards, Magnus Enger libriotech.no
participants (3)
-
Christopher Curry -
Magnus Enger -
Zeno Tajoli