[Koha-devel] MARC record size limit

[Frederic Demians]

> I did some (very limited) testing on storing and retrieving MARC in 
> YAML. The results were not encouraging. IIRC, I just did a direct 
> conversion of the MARC::Record object into YAML and back. Perhaps 
> there's a way to optimize the formatting that would improve 
> performance, but my testing showed sometimes even worse performance 
> than XML.

Did you use YAML or YAML::XS? My tests with YAML::XS shows a very 
significant improvement with YAML: see attached file. Of course, we 
should define an serialization format independent from MARC::Record 
object if we don't want to break the process when MARC::Record internal 
data structure ever change.

> MARCXML is a performance killer at this point, but there's no other 
> apparent way to handle large bib records. The parsing is the issue, 
> not the data transfer load. Perhaps cached BSON-formatted MARC::Record 
> objects are a way out of this.

   Benchmark should be done with all available serialization formats.

We also could implement serialization/deserialization logic directly 
into MARC::Record library, as ISO2709 and XML format, in order gain 
control.
--
Frédéric
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test-marc-deserial
URL: </pipermail/koha-devel/attachments/20101027/71c1fe6a/attachment-0001.asc>