https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37020 --- Comment #38 from Thomas Klausner <domm@plix.at> ---
I have had a look at the XML::Twig patch and now that the records are again streamed and not loaded all at once I don't see why we should add a new dependency and increase code complexity with very little to gain in terms of memory consumption.
I agree. Usimg XML::Twig makes little sense, esp as XML::LibXML (and thus XML::LibXML::Reader are already available)
I may be wrong, but memory consumbed by XML::LibXML should be released a record has been read, so there should not really be any significant difference.
The old code (prior to pushing all the records onto an array) had no memory problem. But I would still like to move away from using MARC::Batch when reading XML files, because the code there is crazy. But maybe this could be a second, later patch / bug.
With regard to the patch by Kyle M there is a bug introduced where the main loop can be terminated without indexing the last batch of records, since the conditions for indexing the last batch will not be met. There needs to be an additional check for if we have run out of records, and after that a check whether to terminate the loop.
Yeah, reading that patch was also quite hard... Anyway, I think what we should do is: * review your patch * merge it if it works * start another bugzilla issue to refactor bulkmarcimport to use XML::LibXML::Reader for streaming XML input files -- You are receiving this mail because: You are watching all bug changes.