[Koha-devel] MARC record size limit

[Fouts, Clay]

That is substantially faster, and more in line with my expectation of what
the performance difference would be. I had thought I was using the XS
module, but clearly something was amiss! This is good news.

I would think implementing within MARC::Record would be the winner between
those two approaches, rather than coming up with an intermediary format.
However, I had been imagining utilizing the YAML for caching rather than as
persistent data, making the accommodation of internal changes to
MARC::Record a matter of flushing the cache rather than regenerating the
serialized records.

Cheers,
Clay


2010/10/26 Frederic Demians <frederic at tamil.fr>

>
>  I did some (very limited) testing on storing and retrieving MARC in YAML.
>> The results were not encouraging. IIRC, I just did a direct conversion of
>> the MARC::Record object into YAML and back. Perhaps there's a way to
>> optimize the formatting that would improve performance, but my testing
>> showed sometimes even worse performance than XML.
>>
>
> Did you use YAML or YAML::XS? My tests with YAML::XS shows a very
> significant improvement with YAML: see attached file. Of course, we should
> define an serialization format independent from MARC::Record object if we
> don't want to break the process when MARC::Record internal data structure
> ever change.
>
>
>  MARCXML is a performance killer at this point, but there's no other
>> apparent way to handle large bib records. The parsing is the issue, not the
>> data transfer load. Perhaps cached BSON-formatted MARC::Record objects are a
>> way out of this.
>>
>
>  Benchmark should be done with all available serialization formats.
>
> We also could implement serialization/deserialization logic directly into
> MARC::Record library, as ISO2709 and XML format, in order gain control.
> --
> Frédéric
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20101027/75237e3d/attachment.htm>