[Koha-devel] RFC 3.2: stop copying item data to MARC and MARCXML

[Ryan Higgins]

Oops -- hit send instead of save :/   to complete my thought :

On Tue, Sep 30, 2008 at 1:36 PM, Ryan Higgins <ryan.higgins at liblime.com>wrote:

>
> Yes, this will be a significant change, but one I think is necessary.
> It will only be more difficult to test later when more features are added.
>
> Serial records become unmanageable when they
>
... reach a couple hundred items.  We had a couple of  bugs caused by
the duplication of the data in early 3.0 that were subsequently fixed, but
caused
the data in marc vs items table to become out of sync.

I think this move will improve performance quite a bit, as it not only
affects
many-item records, but our availability and circ status.  Some quick
profiling
reveals that xml parsing is by far the greatest component of most operations
in Koha.  Removing the need to parse the full bib record on every circ
transaction
should be a good step in improving performance imo.


Ryan


On Tue, Sep 30, 2008 at 8:26 AM, Paul POULAIN <paul.poulain at free.fr> wrote:

> Galen Charlton a écrit :
>
> > Consequently, I propose the following changes, in order of priority:
> >
> > 1. Take item data out of biblioitems.marc*.  Instead, whenever item
> > records need to be embedded in a MARC bib record (primarily during
> > Zebra indexing or record export), do it on the fly.  If a bib record
> > has lots of items, this will still be an expensive operation, but it
> > will take place in a batch job or indexing daemon.
> > 2. Separate the item field mapping from the rest of the MARC
> > framework.  Allow multiple item field mappings, each identified by a
> > code, and add the ability to specify a mapping to be used during
> > record import, export, or indexing.
> > 3. Switch to Zebra's DOM filter and supply item data for indexing
> > outside of the MARC bib record; instead, define an XML schema to wrap
> > bib, item, and summary records without force the last two to be
> > embedded in the MARC.  This is similar to an idea that I believe Tümer
> > Garip proposed back in 2006.
>
> this is a good idea. BUT : it means making heavy changes in Koha, as
> it's the core of most features we have. Even if we have nice packages &
> a stable API, I think the changes will be heavy. I spoke of that with
> hdl this morning, he fully agree.
>
> So I think this change must be carefully estimated, and double checked
> if we can include that in 3.2 with the timeline you've announced (and
> that I fully agree with)
>
> --
> Paul POULAIN
> http://www.biblibre.com
> Expert en Logiciels Libres pour l'info-doc
> NOUVEAU TELEPHONE : 04 91 81 35 08
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha.org
> http://lists.koha.org/mailman/listinfo/koha-devel
>



-- 
> Ryan Higgins
>
> LibLime * Open-Source Solutions for Libraries
> Featuring KohaZOOM ILS
> 888-564-2457 x704
>



-- 
Ryan Higgins

LibLime * Open-Source Solutions for Libraries
Featuring KohaZOOM ILS
888-564-2457 x704
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20080930/8387f37a/attachment-0003.htm>