[Koha-devel] link_bibs_to_authorities.pl can corrupt records

Chris Nighswonger cnighswonger at foundations.edu
Sat Jan 7 14:27:37 CET 2012 

Just what are probably obvious observations here:

1. *Always* run *any* script (stable or otherwise) which works on
data, on a test db (read "one into which zero records are being added)
*first and before* running it on a live (in *any* sense of the term)
db.

2. If my catalogers were doing 100+ bibs per day, I'd be running a
mysqldump on the db while they were at lunch and when they quit for
the day (and probably anytime they took a coffee break, too).

Just my $0.02 worth.

Kind Regards,
Chris


2012/1/6 Paul <paul.a at aandc.org>:
> Chris,
>
> Many thanks - picking up on you points below:
>
>
> At 09:57 AM 1/7/2012 +1300, Chris Cormack wrote:
>
> I have successfully use that script to link authorities to biblio records
> after doing a migration. But the way it works is to remove all links and
> recreate.
>
>
> Not sure about the "recreate" bit...
>
>
> Course I didn't do this on a server people were using, which it sounds like
> you are, to have lost 100 biblios.
>
>
> "People" are using it.  After nearly a year of using Koha 3.2, 3.6.1 is up
> and running (kudos to the development team, average LAN response time is
> down from 4.7 secs to well < 1 sec.)  I asked for assistance with zebra for
> authorities, and the assistance from the community allowed me to resolve the
> difficulties.  Then I asked our volunteers to start cataloguing on 3.6.1 to
> sort out any remaining "update" glitches -- hence the ~100 biblios in the
> last 24 hours (if you want the nitty-gritty on why we've got to do so many
> per day, I'll oblige off-list)
>
>
> Its included for that use case. Jared is working on a much better linking
> script and he gave you the information how to change it.
>
>
> Respectfully, it's included in the "current stable release" with no warnings
> (3.6.1 was "current" when I installed it -- I haven't verified 3.6.2.) and
> Jared's information came after "horses and barn doors."
>
>
> Its not a total screwup script, and it didn't demolish the whole db, I think
> you would find people more willing to help you, in their own free time, in
> their weekends if you didn't come across so aggressive and accusatory in
> your tone.
>
>
> I humbly apologize to anyone who finds me "aggressive and accusatory"  --
> it's probably the way I was taught to write (concisely, briefly, etc) rather
> than my feelings.  Most people who know me well find me to be a cuddly
> rather than a grizzly bear.  Again my apologies.
>
> Best - Paul
>
>
> Chris
>
> On 7 Jan 2012 09:48, "Archives and Collections Society" <info at aandc.org>
> wrote:
> At 03:02 PM 1/6/2012 -0500, Ian Walls wrote:
>
> Paul,
> The link_bibs_to_authorities.pl script is hardcoded to ALWAYS erase manual
> links.  I'm not sure what the reasoning is behind that.  But, every
> time you run it, any link you've done manually that the script can't
> automatically figure out will be removed.  Nature of the beast (for now,
> at least).
>
>
> What is "any link you've done manually"?, please.  More than 90% of our
> input is Z39.50 and "default" item cataloguing.
>
> OK - a total screw-up that is included without any warning whatsoever in the
> standard distribution of 3.6.1 ???  From the bug report
> <http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5683>
>
> Chris Cormack 2011-06-01 21:13:23 UTC
> Pushed to master
>
> Jared Camins-Esakov 2011-07-08 14:37:02 UTC
> Seems to be working now. Closing.
>
> If it's "nature of the beast" why isn't the "beast" mentioned somewhere?Â
> What on earth is the reason for this .pl to be included in any distribution?
>
> More...
>
> 2012/1/6 Chris Cormack <chris at bigballofwax.co.nz> Im assuming you did this
> on your testing/staging server right? Id restore that from backup, then try
> applying the patch and see if you get better behaviour. If so, then do it on
> your production machine.
>
>
> what "patch"???
>
> And yes -- I can restore from a previous sqldump, and only lose +/- 100
> biblios.   We're in pre-production and when we got the zebra indexing
> working properly, I told our volunteers to start inputting to 3.6.1 *only*
> (not duplicate on 3.2) so that I could look into any difficulties they
> experienced as cataloguers.
>
>
> I had no way at all of knowing that a pl script included in the "latest
> stable distribution" could possibly demolish the whole db.
>
> Again, what "patch" and where do I find it?
>
> Might I suggest that someone re-opens the bug (I have attempted to annotate
> it) and removes this dangerous script from public distribution (or at least
> includes a very visible warning.)
>
> Best - Paul
>
> Chris
> On 7 Jan 2012 08:47, "Paul" <paul.a at aandc.org> wrote:
> Bug 5683 (link_bibs_to_authorities.pl can corrupt records) was signed off
> some months ago, but is perhaps a little too cryptic for me to follow in
> detail. Â Can someone help? As far as I can see, without remedial action,
> this is catastrophic.
> Following difficulties we have had migrating 3.2 to 3.6.1, someone on the
> users mailing list suggested we should run link_bibs_to_authorities.pl --
> (db has ~10k authorities and ~15k biblios) -- so I did just that, and have
> APPARENTLY DEMOLISHED *EVERY* LINK IN THE DB. Â From my notes:
> paul at nelson:/usr/share/koha$ ./bin/link_bibs_to_authorities.pl
> Bib authority heading linking report
> ------------------------------------ Number of bibs checked:     ÂÂ
> 14911 Number of bibs modified:      14886 Number of bibs with
> errors:   0
> but - the staff interface no longer correctly finds authorities !!!
> [1] Manually link (staff client) auth "Lubbock" to bib "Arctic Whalers" --
> now linked, 35 other Lubbock bibs not linked. Â Is this a question of zebra
> re-indexing?
> So: Â KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib
> ./bin/migration_tools/rebuild_zebra.pl -a -r -v and Â
> KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib
> ./bin/migration_tools/rebuild_zebra.pl -b -r -v -x
> koha at nelson:/usr/share/koha$ ./bin/link_bibs_to_authorities.pl ... processed
> 100 records / ... / ... processed 14900 records
> Bib authority heading linking report
> ------------------------------------ Number of bibs checked:     ÂÂ
> 14911 Number of bibs modified:      1 Number of bibs with errors:
>   0
> koha at nelson:/usr/share/koha$
> All this did was to *un-link" the bib/auth link that I had manually entered
> at [1] above. Could someone involved with signing off bug 5683 please
> explain why link==unlink?
> Help - please.
> Paul
> _______________________________________________ Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website
> : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs :
> http://bugs.koha-community.org/
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
>
>
>
> --
> Ian Walls
> Lead Development Specialist
> ByWater Solutions
> ALA Midwinter Booth #2048
> Phone # (888) 900-8944
> http://bywatersolutions.com
> ian.walls at bywatersolutions.com
> Twitter: @sekjal
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
>
> ---
> Archives and Collections (ACS) Society
> 205, Main Street, Picton, Ontario, K0K 2T0, Canada
> http://www.AandC.org
> Canadian Charitable Organization 88721 9921 RR0001
> Dedicated to maritime conservation and education.
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
> ---
> Maritime heritage and history, preservation and conservation,
> research and education through the written word and the arts.
> <http://UltraMarine.ca>, <http://AandC.org> and <http://MarDoc.ca>
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/

More information about the Koha-devel mailing list