[Koha-devel] link_bibs_to_authorities.pl can corrupt records

Paul paul.a at aandc.org
Fri Jan 6 22:45:29 CET 2012 

Chris,

Many thanks - picking up on you points below:

At 09:57 AM 1/7/2012 +1300, Chris Cormack wrote:

>I have successfully use that script to link authorities to biblio records 
>after doing a migration. But the way it works is to remove all links and 
>recreate.

Not sure about the "recreate" bit...

>Course I didn't do this on a server people were using, which it sounds 
>like you are, to have lost 100 biblios.

"People" are using it.  After nearly a year of using Koha 3.2, 3.6.1 is up 
and running (kudos to the development team, average LAN response time is 
down from 4.7 secs to well < 1 sec.)  I asked for assistance with zebra for 
authorities, and the assistance from the community allowed me to resolve 
the difficulties.  Then I asked our volunteers to start cataloguing on 
3.6.1 to sort out any remaining "update" glitches -- hence the ~100 biblios 
in the last 24 hours (if you want the nitty-gritty on why we've got to do 
so many per day, I'll oblige off-list)

>Its included for that use case. Jared is working on a much better linking 
>script and he gave you the information how to change it.

Respectfully, it's included in the "current stable release" with no 
warnings (3.6.1 was "current" when I installed it -- I haven't verified 
3.6.2.) and Jared's information came after "horses and barn doors."

>Its not a total screwup script, and it didn't demolish the whole db, I 
>think you would find people more willing to help you, in their own free 
>time, in their weekends if you didn't come across so aggressive and 
>accusatory in your tone.

I humbly apologize to anyone who finds me "aggressive and 
accusatory"  --  it's probably the way I was taught to write (concisely, 
briefly, etc) rather than my feelings.  Most people who know me well find 
me to be a cuddly rather than a grizzly bear.  Again my apologies.

Best - Paul


>Chris
>On 7 Jan 2012 09:48, "Archives and Collections Society" 
><<mailto:info at aandc.org>info at aandc.org> wrote:
>At 03:02 PM 1/6/2012 -0500, Ian Walls wrote:
>>Paul,
>>The <http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl 
>>script is hardcoded to ALWAYS erase manual links.  I'm not sure what 
>>the reasoning is behind that.  But, every time you run it, any link 
>>you've done manually that the script can't automatically figure out will 
>>be removed.  Nature of the beast (for now, at least).
>
>What is "any link you've done manually"?, please.  More than 90% of our 
>input is Z39.50 and "default" item cataloguing.
>
>OK - a total screw-up that is included without any warning whatsoever in 
>the standard distribution of 3.6.1 ???  From the bug report 
><<http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5683>http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5683>
>
>Chris Cormack 2011-06-01 21:13:23 UTC
>Pushed to master
>
>Jared Camins-Esakov 2011-07-08 14:37:02 UTC
>Seems to be working now. Closing.
>
>If it's "nature of the beast" why isn't the "beast" mentioned 
>somewhere?  What on earth is the reason for this .pl to be included in 
>any distribution?
>
>More...
>>2012/1/6 Chris Cormack 
>><<mailto:chris at bigballofwax.co.nz>chris at bigballofwax.co.nz>
>>Im assuming you did this on your testing/staging server right?
>>Id restore that from backup, then try applying the patch and see if you 
>>get better behaviour.
>>If so, then do it on your production machine.
>
>what "patch"???
>
>And yes -- I can restore from a previous sqldump, and only lose +/- 100 
>biblios.   We're in pre-production and when we got the zebra indexing 
>working properly, I told our volunteers to start inputting to 3.6.1 *only* 
>(not duplicate on 3.2) so that I could look into any difficulties they 
>experienced as cataloguers.
>
>I had no way at all of knowing that a pl script included in the "latest 
>stable distribution" could possibly demolish the whole db.
>
>Again, what "patch" and where do I find it?
>
>Might I suggest that someone re-opens the bug (I have attempted to 
>annotate it) and removes this dangerous script from public distribution 
>(or at least includes a very visible warning.)
>
>Best - Paul
>
>>Chris
>>On 7 Jan 2012 08:47, "Paul" <<mailto:paul.a at aandc.org>paul.a at aandc.org> 
>>wrote:
>>Bug 5683 (<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl 
>>can corrupt records) was signed off some months ago, but is perhaps a 
>>little too cryptic for me to follow in detail. Â Can someone help? As 
>>far as I can see, without remedial action, this is catastrophic.
>>Following difficulties we have had migrating 3.2 to 3.6.1, someone on the 
>>users mailing list suggested we should run 
>><http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl -- (db 
>>has ~10k authorities and ~15k biblios) -- so I did just that, and have 
>>APPARENTLY DEMOLISHED *EVERY* LINK IN THE DB. Â From my notes:
>>paul at nelson:/usr/share/koha$ 
>>./bin/<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl
>>Bib authority heading linking report
>>------------------------------------
>>Number of bibs checked:       14911
>>Number of bibs modified:      14886
>>Number of bibs with errors:   0
>>but - the staff interface no longer correctly finds authorities !!!
>>[1] Manually link (staff client) auth "Lubbock" to bib "Arctic Whalers" 
>>-- now linked, 35 other Lubbock bibs not linked. Â Is this a question of 
>>zebra re-indexing?
>>So: Â KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib 
>>./bin/migration_tools/<http://rebuild_zebra.pl>rebuild_zebra.pl -a -r -v
>>and  KOHA_CONF=/etc/koha/koha-conf.xml PERL5LIB=/usr/share/koha/lib 
>>./bin/migration_tools/<http://rebuild_zebra.pl>rebuild_zebra.pl -b -r -v -x
>>koha at nelson:/usr/share/koha$ 
>>./bin/<http://link_bibs_to_authorities.pl>link_bibs_to_authorities.pl
>>... processed 100 records
>>/ ... /
>>... processed 14900 records
>>Bib authority heading linking report
>>------------------------------------
>>Number of bibs checked:       14911
>>Number of bibs modified:      1
>>Number of bibs with errors:   0
>>koha at nelson:/usr/share/koha$
>>All this did was to *un-link" the bib/auth link that I had manually 
>>entered at [1] above. Could someone involved with signing off bug 5683 
>>please explain why link==unlink?
>>Help - please.
>>Paul
>>_______________________________________________
>>Koha-devel mailing list
>><mailto:Koha-devel at lists.koha-community.org>Koha-devel at lists.koha-community.org 
>>
>>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>website : <http://www.koha-community.org/>http://www.koha-community.org/
>>git : <http://git.koha-community.org/>http://git.koha-community.org/
>>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/
>>
>>
>>_______________________________________________
>>Koha-devel mailing list
>><mailto:Koha-devel at lists.koha-community.org>Koha-devel at lists.koha-community.org
>>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>website : <http://www.koha-community.org/>http://www.koha-community.org/
>>git : <http://git.koha-community.org/>http://git.koha-community.org/
>>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/
>>
>>
>>
>>
>>--
>>Ian Walls
>>Lead Development Specialist
>>ByWater Solutions
>>ALA Midwinter Booth #2048
>>Phone # (888) 900-8944
>><http://bywatersolutions.com>http://bywatersolutions.com
>><mailto:ian.walls at bywatersolutions.com>ian.walls at bywatersolutions.com
>>Twitter: @sekjal
>>_______________________________________________
>>Koha-devel mailing list
>><mailto:Koha-devel at lists.koha-community.org>Koha-devel at lists.koha-community.org
>>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>website : <http://www.koha-community.org/>http://www.koha-community.org/
>>git : <http://git.koha-community.org/>http://git.koha-community.org/
>>bugs : <http://bugs.koha-community.org/>http://bugs.koha-community.org/
>
>---
>Archives and Collections (ACS) Society
>205, Main Street, Picton, Ontario, K0K 2T0, Canada
><http://www.aandc.org/>http://www.AandC.org
>Canadian Charitable Organization 88721 9921 RR0001
>Dedicated to maritime conservation and education.
>
>_______________________________________________
>Koha-devel mailing list
>Koha-devel at lists.koha-community.org
>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>website : http://www.koha-community.org/
>git : http://git.koha-community.org/
>bugs : http://bugs.koha-community.org/

---
Maritime heritage and history, preservation and conservation,
research and education through the written word and the arts.
<http://UltraMarine.ca>, <http://AandC.org> and <http://MarDoc.ca>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20120106/caa54d97/attachment.htm>

More information about the Koha-devel mailing list