[Koha-bugs] [Bug 21828] Improve efficiency of C4::Biblio::LinkBibHeadingsToAuthorities

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21828

Andreas Roussos <a.roussos at dataly.gr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                URL|                            |misc/link_bibs_to_authoriti
                   |                            |es.pl

--- Comment #2 from Andreas Roussos <a.roussos at dataly.gr> ---
bulkmarcimport.pl seems to be the only script that is affected
the most by this, as it can potentially make an excessive number
of calls to LinkBibHeadingsToAuthorities().

Also, if the AutoLinkBiblios system preference is set to 'Do',
then AddBiblio() and ModBiblio() both call BiblioAutoLink()
which in turn calls LinkBibHeadingsToAuthorities() to do the
heavy-lifting. So, I believe my patch might bring an improvement
in the overall responsiveness of the Staff interface when saving
new bibliographic records or modifying existing ones, especially
for records with many many fields. However, the big win here is the
significantly lower execution times for any helper scripts that
call these functions. These benefits apply to UNIMARC only ;-)

To put the above claim to the test, I profiled the DBI calls made
while using link_bibs_to_authorities.pl and also timed how long
it took to run the script. The exact command line used is shown
below. My database had 8,241 authority and 23,142 biblio records.

                 +---------------+------------+------------+
  DBI calls made | without patch | with patch | difference |
                 +---------------+------------+------------+
                 |    14,732,858 |    626,090 |    -95.75% |
                 +---------------+------------+------------+

                 +---------------+------------+------------+
  execution time | without patch | with patch | difference |
                 +---------------+------------+------------+
                 |     1,125 sec |    186 sec |    -83.47% |
                 +---------------+------------+------------+

Command line used:

$ DBI_PROFILE=1 ./misc/link_bibs_to_authorities.pl -t

I include the "raw" profiling data from my test runs below:

Bib authority heading linking report
=======================================================
Linker module:                          C4::Linker::Default
Run started at:                         08/01/23 20:10:39
Run ended at:                           08/01/23 20:29:24
Total run time:                         1124977 ms
Number of bibs checked:                 23142
Number of bibs modified:                0
Number of bibs with errors:             0
Number of headings linked:              32316
Number of headings unlinked:            15793
Number of headings fuzzily linked:      0

****  Ran in test mode only  ****
DBI::Profile: 880.455353s 78.05% (14732858 calls) link_bibs_to_authorities.pl @
2023-08-01 20:29:24

Bib authority heading linking report
=======================================================
Linker module:                          C4::Linker::Default
Run started at:                         08/01/23 20:40:06
Run ended at:                           08/01/23 20:43:12
Total run time:                         185965 ms
Number of bibs checked:                 23142
Number of bibs modified:                0
Number of bibs with errors:             0
Number of headings linked:              32316
Number of headings unlinked:            15793
Number of headings fuzzily linked:      0

****  Ran in test mode only  ****
DBI::Profile: 29.444538s 15.58% (626090 calls) link_bibs_to_authorities.pl @
2023-08-01 20:43:12

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.