[Bug 38427] New: (38408 follow-up) Additional fixes in rebuild_zebra.pl and koha-rebuild-zebra
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 Bug ID: 38427 Summary: (38408 follow-up) Additional fixes in rebuild_zebra.pl and koha-rebuild-zebra Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: koha-bugs@lists.koha-community.org Reporter: m.de.rooy@rijksmuseum.nl QA Contact: testopia@bugs.koha-community.org CC: robin@catalyst.net.nz See base report 38408. We need to adjust koha-rebuild-zebra now to make the forks change more visible. Fixing an issue with offset and length for biblios. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |38408 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38408 [Bug 38408] Add parallel exporting of MARC records to Zebra rebuild/reindex -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |m.de.rooy@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 --- Comment #1 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- TODO Remove obsoleted variable during dev process: +my $chunk_size = 100000; For the purists: + $offset ||= 0; + $length ||= ( $record_type eq 'biblio' ? Koha::Biblios->count : Koha::Authorities->count ); We should probably better do //= here. This will make -length 0 do nothing (which is useless, but anyway..). Check on a negative offset too? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 --- Comment #2 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- TODO + export_marc_records_from_sth( $record_type, $sth, "$directory", $nosanitize, $seq ); + $pm->finish; # exit for child only + } + $offset += $chunk_size; + $num_records_exported += $chunk_size; Do not assume chunk_size to be equal to exported record count. Here we need to pass back info from the child process to the parent. Since this complicates the base report, we should do it later. In daily life this wont hurt. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 --- Comment #3 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- TODO Resolve inconsistency in current code when using length/offset sub select_all_authorities { $strsth.=qq{ LIMIT $length } if ($length && !$offset); $strsth.=qq{ LIMIT $offset,$length } if ($length && $offset); sub select_all_biblios { $strsth.=qq{ LIMIT $length } if ($length && !$offset); $strsth.=qq{ LIMIT $offset,$length } if ($offset); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #3)
TODO Resolve inconsistency in current code when using length/offset
sub select_all_authorities { $strsth.=qq{ LIMIT $length } if ($length && !$offset); $strsth.=qq{ LIMIT $offset,$length } if ($length && $offset);
sub select_all_biblios { $strsth.=qq{ LIMIT $length } if ($length && !$offset); $strsth.=qq{ LIMIT $offset,$length } if ($offset);
Awful but passing an offset without a limit requires something like 2^64-1: SELECT * FROM somewhere LIMIT 18446744073709551615 OFFSET 5 Surely we can pass the total number of records here like: + $length ||= ( $record_type eq 'biblio' ? Koha::Biblios->count : Koha::Authorities->count ); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38427 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- Seems that some of this would address some of my comments on bug 38408 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org