[Koha-bugs] [Bug 24123] bulkmarcimport.pl doesn't support UTF-8 encoded MARCXML records

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24123

Katrin Fischer <katrin.fischer at bsz-bw.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |katrin.fischer at bsz-bw.de

--- Comment #8 from Katrin Fischer <katrin.fischer at bsz-bw.de> ---
(In reply to Martin Renvoize from comment #6)
> Happy with this pragmatic approach to a fix..
> 
> It has however, bugged me for a long time that we have both bulkmarcimport
> and the stage + commit command-line scripts to do the same process in
> reality.  I'd love to see that tidied up some time.
> 
> Also.. I'm really not sure why we need MARC::Batch at all.. my understanding
> is that its intended use case is for multiple files, but we're only loading
> a single file using bulkmarcimport aren't we?
> 
> Either way, pushing :)

While both import, they do it quite differently. Bulkmarcimport is faster as it
does not use the staging tables, but imports directly into the database.
Matching works differently (not using the matching rules system) and it can't
be undone.

The stage and commit scripts could probably be combined into one thing, but as
they are they match the fact it's also 2 tools in Koha. They use the matching
rules and import tables, which can mean create quite a lot of data db size wise
as everything is stored twice after importing and cleaning the batch I think
doesn't really make the db shrink. But you can undo them and use complex
matching rules. When you want to have 'versioning' for your records, it's also
helpful to keep that data.

We noticed that loading big files via stage/commit created quite a big load on
the servers - I wonder if instead of multiple files a possible use could be to
break up big files and work on them sequentially?

-- 
You are receiving this mail because:
You are watching all bug changes.