https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23766 Bug ID: 23766 Summary: Allow MARC record importer to handle large batches without causing memory issues Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: kyle@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Right now, no matter how large a marc batch file is, the marc record stager loads all the records into memory ( see http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=C4/ImportBatch.pm;... ). We should break this up and process chunks of records at a time to keep the memory footprint low for servers without enough RAM to handle this memory load. My proposal is to modify process_batch in stage_file to loop over the records processing X number of records at a time. X should be configurable via a syspref. In addition, this will require modifying C4::ImporBatch::RecordsFromMARCXMLFile to accept an offset and a count such that it will start at the offset and return the given count of records. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.