[Koha-devel] New KOHA API at HEAD

[Tümer Garip]

I have started committing a new API to head and here is what is
implemented and what is foreseen.
The new DB: changes since 2.2.6;
1-All the information related to biblios,  items, and authorities are
now held in separate MARC records (currently both MARC and MARCXML and
will may drop MARC in due time as well.We have to discuss this!).
2- Biblioitems table completely depreceated.
3- Items table  has
itemnumber-biblionumber-barcode-marc-marxml-timestamp only
4-Biblio table has
biblionumber-title-author-itemtype-isbn-issn-frameworkcode-marc-marcxml-
timestamp only
5-auth_header table has authid-authtypecode-marc-datecreated
6- marc_subfield_structure renamed biblios_subfield_structure and
kohafield dropped,hidden is tiny int(3)
7- marc_tag_structure renamed biblios_tag_structure
8- biblio_framework renamed biblios_framework
9-New tables added:
holdings_tag_structure,holdings_subfield_structure,koha_attr,zebraqueue

Concepts behind;
1- Items (holdings) is now a complete record in itself. This provides
more flexibility with storing data that was supposed to fit in one field
in the early version. Now a whole record. But all the sql fields are
gone. So anything you want to put it you have to store them in MARC.
Thats why those fields with with %9% in them are designed for.
2- Faster circulation. Only update items record.
3- We can hide the items record from a union catalog in Z3950. Private
data in items is protected.
4- Frameworks handling is only for record input and display purposes.
You can define different frameworks for biblios and authorities only.
Holdings frameworks  you can design but cannot rename. A framework name
is given at biblios level which creates matching frameworks for biblios
and holdings. That is if you create a framework called 'Music Records'
then you set it up one for biblios and one for holdings. (Its simpler
than it sounds)
5- Visibility flag in frameworks is more verbose. You select
"Show","Show collapsed" or "Hide" separately for "OPAC" "Intranet" and
"Editor" 
6- A new koha_attr table provides the field matching between DB fields
and MARC record. Apart from a few fields like biblionumber, itembumber
etc. that KOHA requires (and have to be defined to work) the rest is
upto the system designer what to match. 
7- The same table matches Zebra indexing terms to koha terms or field
names. I say kohaterms because most of the fields now do not exist. They
are pseudo field names so that we can match 'dewey' to some marc field.
In the example koha_attr table provided (for MARC21) all bibliographic
fields are matched to Bib-1 attributes to make it universally
searchable, while all holdings data is private matches
8- Same table allows creating search screens with field labels and not
bothering about what the PQF attibute is for it. All is internally taken
care of. You have a show in intranet and show in opac flag. A sort flag
is added so that any field that sorts automatically gets added to search
screens.
9- zebraqueue table is for zebraupdates. Zebra updating cannot keep up
with large libraries circulation demands. So this puts updating in a
queue and works asynchronosly so that KOHA can go on doing other things.
It has a system preference time switch how long to wait between updates
from (0 to forever in seconds). Or a batchMode preference to use shadow
files of zebra and not to commit until told so. I personally found
zebraqueu to be more reliable than batchMode.

New Zebra indexing and KOHA meta record:
1- Having separated holdings record from bibliographic earlier
zebraindexing was of no use. Zebra is not a relational DB. (I am
surprised at people commenting on this subject at koha-devel !!!).
2- A new KOHA meta record is designed based on MARCXML. Currently it
holds one bibliographic and many holdings records in marxml but can be
extended to hold authorities records as well.
3- Each individual record is fullfledged MARCXML.
4- The design is <kohacollection><koharecord/></kohacollection>. A koha
collection can hold many koha meta records. Its usefull for batch
indexing of ZEBRA. Yes we can now batch index XML records with ZEBRA!
5- A koha meta record is
<koharecord><record/><holdings><record/></holdings></koharecord>
Each <record/> is a MARCXML record. The first one is the bibliographic
and any record withing the holdings tag is considered a MARCXML item
record.Holdings tag can hold 0 to many records.
6- Zebra can display either the whole koharecord, only the bibliographic
or the holdings record.
Using yaz-proxy an outside connection to Z3950 server (a union catalogue
may be) can be limited to see only the bibliographic record.

API and whats next;
1- Due to the urgency of our needed to have the system working ASAP so
the current committed API is a mixture of 2_2_6 and 3.0 and working ie
it uses both MARC iso2709 and MARCXML. The display and search should be
written in xsl stysheets for performace. Already tested it with LibXML
perl module and results are promising.
2- Using ZOOM the dependency on Event and Net::Z3950 is depreceated. No
more deamons for Z3950 searches. This way KOHA may claim full Windows
and IE compatibility. The concept is changed so that cataloger may
choose servers on the fly and any record he/she retrieves is not a new
addition of biblio but just replaces the old one if he/she was modifying
one.
3- HTML::Template is replaced with HTML::Template::Pro - for UTF8 issues
4- A top layer UTF8DBI by Pavel Kudinov http://search.cpan.org/~kudinov/
is implemented as part of API - for UTF8 issues
5- KOHA is now fully UTF8 compliant.
6- Repeated fields and adding authorities bug in addbiblio.pl fixed
7- Using hardcoded subfield 9  for linking authorities to biblios
removed. A pseudofield called auth_linkid is defined in koha_attr table
which can map to anything with the notation '0009' to use subfield 9 for
any tag.
 
Thats generally what it is. I will try to provide an upgrade script from
2_2_6 to HEAD soon.
In the meantime once all the commit is finished next week you should be
able to play with it with new data.

Cheers
Tumer