December 2002 - Koha-devel - lists.koha-community.org

RE: [oss4lib-discuss] MARC -> RDBMS
by Pat Eyler 12 Dec '02

12 Dec '02

On Wed, 11 Dec 2002, Brian Cassidy wrote: > Hello again, > > Thanks to everyone for their quick replies! Here's what I've done since > yesterday: > > I looked into the 'Koha' library software. It looks interesting, but the > line "It is currently possible to import MARC records, one at a time." > turned me off of it, due to the fact that I have several repositories > with thousands of records each. It sounds like we need to do a better job of documenting something then. THere are a couple of flavors of bulk-importing scripts at this point. Other flavors can be derived from the existing scripts at need. (It's all a smop, right?) -pate > > I looked into Michael Doran's concerns, but the powers that be want me > to steer away from a Z39.50 interface. So, that's as far as I got with > that. :/ > > After looking at the document > (http://www.openisis.org/openisis/doc/RdbConv) provided by Ferran Jorba, > I created a simple DB structure (pardon my ASCII :) > > .---------. .-----------. > | item | | marc_data | > ----------- ---- ------------- > | item_id | | item_id | > ----------- | data | > | ------------- > | > /|\ > .-------------. > | tag_data | > --------------- > | item_id | > | tag_order | > | tag | > | indicator_1 | > | indicator_2 | > | subfield | > | data | > --------------- > > Note: marc_data.data is just a 'blob' of the MARC record. > > I then used MARC::Record (Hi Ed.) to quickly write a couple perl scripts > to generate some SQL and run the queries. My database now has 2002 > (MARC) records in a somewhat useable structure. For instance, I can now > say > > SELECT > item_id > FROM > tag_data > WHERE > tag = '008' AND > data LIKE '%fre__' > > to get all of the IDs of the French records. > > I'm now at the point where I need to create a data crosswalk (as Art > mentioned) so I can display the data in a human readable way, as well as > facilitate easy searching. The Dublin Core crosswalk > (http://www.loc.gov/marc/marc2dc.html) on the loc.gov website (hat tip: > Art) seems like a great standard to follow. The problem that I'm now > facing is that frequently, data in a tag is split up into many > subfields. For instance: > > SELECT > item_id, > subfield, > data > FROM > ust_tag_data > WHERE > tag = '245' -- Title > ORDER BY > item_id > > One of my results is: > > item_id | subfield | data > ------------------------------------------------------------- > ... > 2 | a | Keeping Alberta competitive : > 2 | b | success through workplace learning. -- > ... > > Now, it's totally doable to loop through the results (in perl, ASP, etc) > and concatenate the data together. This COULD, however, get tedious > depending on the number of fields I need to look at. This is probably > the wrong forum to ask, but would anyone have any suggestions on how to > proceed (or perhaps takes some steps back and attack it differently)? > > Thanks again for all your help! > > -Brian Cassidy (brian at nald dot ca) > > > http://www.gordano.com - Messaging for educators. > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > > oss4lib-discuss(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss > see also http://www.oss4lib.org/ >

3 2

RE: [oss4lib-discuss] MARC -> RDBMS (fwd)
by Pat Eyler 11 Dec '02

11 Dec '02

The section on Koha is particularily interesting to us going forward I think. -pate ---------- Forwarded message ---------- Date: Wed, 11 Dec 2002 16:02:26 -0400 From: Brian Cassidy <brian(a)nald.ca> To: oss4lib-discuss(a)lists.sourceforge.net Subject: RE: [oss4lib-discuss] MARC -> RDBMS Hello again, Thanks to everyone for their quick replies! Here's what I've done since yesterday: I looked into the 'Koha' library software. It looks interesting, but the line "It is currently possible to import MARC records, one at a time." turned me off of it, due to the fact that I have several repositories with thousands of records each. I looked into Michael Doran's concerns, but the powers that be want me to steer away from a Z39.50 interface. So, that's as far as I got with that. :/ After looking at the document (http://www.openisis.org/openisis/doc/RdbConv) provided by Ferran Jorba, I created a simple DB structure (pardon my ASCII :) .---------. .-----------. | item | | marc_data | ----------- ---- ------------- | item_id | | item_id | ----------- | data | | ------------- | /|\ .-------------. | tag_data | --------------- | item_id | | tag_order | | tag | | indicator_1 | | indicator_2 | | subfield | | data | --------------- Note: marc_data.data is just a 'blob' of the MARC record. I then used MARC::Record (Hi Ed.) to quickly write a couple perl scripts to generate some SQL and run the queries. My database now has 2002 (MARC) records in a somewhat useable structure. For instance, I can now say SELECT item_id FROM tag_data WHERE tag = '008' AND data LIKE '%fre__' to get all of the IDs of the French records. I'm now at the point where I need to create a data crosswalk (as Art mentioned) so I can display the data in a human readable way, as well as facilitate easy searching. The Dublin Core crosswalk (http://www.loc.gov/marc/marc2dc.html) on the loc.gov website (hat tip: Art) seems like a great standard to follow. The problem that I'm now facing is that frequently, data in a tag is split up into many subfields. For instance: SELECT item_id, subfield, data FROM ust_tag_data WHERE tag = '245' -- Title ORDER BY item_id One of my results is: item_id | subfield | data ------------------------------------------------------------- ... 2 | a | Keeping Alberta competitive : 2 | b | success through workplace learning. -- ... Now, it's totally doable to loop through the results (in perl, ASP, etc) and concatenate the data together. This COULD, however, get tedious depending on the number of fields I need to look at. This is probably the wrong forum to ask, but would anyone have any suggestions on how to proceed (or perhaps takes some steps back and attack it differently)? Thanks again for all your help! -Brian Cassidy (brian at nald dot ca) http://www.gordano.com - Messaging for educators. ------------------------------------------------------- This sf.net email is sponsored by: With Great Power, Comes Great Responsibility Learn to use your power at OSDN's High Performance Computing Channel http://hpc.devchannel.org/ _______________________________________________ oss4lib-discuss(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://www.oss4lib.org/

1 0

Auth.pm
by Finlay Thompson 10 Dec '02

10 Dec '02

Hi Everyone, I have just commited into the main branch the substantially rewritten Auth.pm authentification module from the rel-1-2 branch (as in use at HLT). The biggest change is the get_template_and_user () sub. This combines a whole lot of code that I had at the top of every script. It gets the correct template, and authenticates the user, loading the users details into the template (in a loop variable USER_INFO). This means that you dont need to "use C4::Output;" in the scripts, just "use C4::Auth;" In addition I have put in some code that checks the systempreferences for a "timeout" time in seconds, otherwise it defaults to 120 sec (2 mins). The login prompt is now templated as well (important for new opac). This is found at opac-auth.tmpl for the opac and auth.tmpl for the intranet. It uses Context as all good main branch modules and scripts should. It also uses Output to get the template objects. It uses Circulation::Circ2 to get the borrower information which is a little weird. It would be good to have a Users.pm module that Circ and everything else could use that accessed all the users information and returned it in a uniform way. Maybe even OO? Anyway, thats the new Auth.pm Finlay

3 3

Open Source Award
by Pat Eyler 10 Dec '02

10 Dec '02

Right now, Linux world is taking submissions for their open source awards. This is a chance to speak up and nominat Koha ... If you think Koha is an outstanding project, or that it deserves more recognition, please consider dropping by http://www.linuxworldexpo.com/linuxworldny03/V33/index.cvn?ID=10147 and making a nomination. (If you *really* think someone else deserves it, I guess you could nominate them.) -pate Pat Eyler Kaitiaki/manager migrant Linux sys admin the Koha project ruby, shell, and perl geek http://www.koha.org http://pate.eylerfamily.org

1 0

Module List
by Chris Cormack 10 Dec '02

10 Dec '02

Hi All The module list is finished at http://www.saas.nsw.edu.au/wiki/index.php?page=ModuleList Please add comments, fix mistakes .. add thoughts etc. Im hoping that we can use this list to plan and then clean up the C4 directory. Chris -- Chris Cormack Programmer 025 500 789 Katipo Communications Ltd chris(a)katipo.co.nz www.katipo.co.nz

1 0

non-simple publication years?
by Ambrose Li [CCCGT] 10 Dec '02

10 Dec '02

Hi, I wonder if some other library places a non-simple value in the publication year field. Here we occasionally place things like "2000 [民89]" in the year of publication field (book lists date of publication in "Republic" date year 89, equivalent to A.D. 2000; we record both). biblio.copyrightdate is a smallint, which makes sense but won't exactly work in our case. I wonder what is the correct way to do this. Regards, -- Ambrose Li Chinese Cultural Centre of Greater Toronto | http://www.cccgt.org/

5 4

sample_only_param_tables.sql
by David Strainchamps 10 Dec '02

10 Dec '02

Hi, there is a trouble to insert itemtypes param with sample_only_param_tables.sql, because there are 6 fields but only 5 in the file David

2 1

marc_subfield_structure table
by David Strainchamps 10 Dec '02

10 Dec '02

Hi I have recreate all my koha base with koha.mysql and updatedatabase and i see a differrence with the structure that Paul send to me in marc_subfield_structure table the five last fields must be able to be null altough when i rebuild my db it's not the case and i can use structure_def.sql for unimarc David

2 1

FW: [Koha-devel] Re: non-simple publication years?
by Anthony W. Youngman 10 Dec '02

10 Dec '02

And you have to watch out with Western dates too ... Take Pepys diary. While you would probably have no desire to store these dates, those between January and March every year belong to which year? (New Year was March 25th). The only safe way to store these things (if you do want to future-proof it in any way) is to store all dates as a Julian/Conversion pair. Julian as in an explicit offset from a fixed date (eg the Unix ?1/1/1970? day 1) and a conversion that says Julian/Gregorian/Republic/Japanese Imperial/whatever. Not a trivial conversion if you haven't planned that in in your original design :-( Cheers, Wol -----Original Message----- From: news-misc(a)ada.dhs.org [mailto:news-misc@ada.dhs.org] Sent: 10 December 2002 04:55 To: koha-devel(a)lists.sourceforge.net Subject: [Koha-devel] Re: non-simple publication years? In article <200212042234.54111.fmmarzoa(a)gmx.net>, Francisco M. Marzoa Alonso <fmmarzoa(a)gmx.net> wrote: >I think that's a matter of representation, so database >structure could be as is (with that sort integer) and scripts >could be modified to support different dating standards simply >making conversions. Although I think this does need change some >scripts, not just templates, because as it seems to be your >case, sometimes the matter is an input value on a form. > >The same is applicable to date formats on different countries >or using different standards. (i.e. some countries use >DD-MM-AAAA while others use MM-DD-AAAA). I think there may be >some kind of support for this in koha so it should be not very >difficult to adapt it for your chinesse years, but I'm not >sure. I guess then this would be something for Koha 3.0 :-) In our library not all dates are in "Chinese" format ("Republic" years, now used only in the ROC); we only lists the Republic dates if the book lists them. I wondered whether this is just a peculiarity of our library, so I went to another library (http://mdx.tpml.edu.tw/v2.3/tables/01a.htm) and see how they do it. They do it just the same; not all books are listed in Republic dates, just some (presumably also just the books which lists dates in this format). So, sticking with the principle that Koha should not dictate how users use the system, we cannot go the conversion route :-) Also, if we go a conversion route, it will also be quite complicated, because one A.D. year may correspond to two or more years in the "foreign" calendar. (For example, the Chinese and Japanese dates are based on the reign of kings, and the king has the say what the new epoch is called (so we can never "future-proof" Koha if we go the conversion route); so one A.D. year can correspond to two Japanese years if, e.g., they get a new Japanese emperor :-) It could get messier if the foreign calendar system is not solar :-( [not the case with modern Chinese and Japanese dates, though] -- Ambrose Li <a.c.li(a)ieee.org> http://ada.dhs.org/~acli/cmcc/ http://www.cccgt.org/ DRM is theft - We are the stakeholders ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Koha-devel mailing list Koha-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/koha-devel This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system. Telephone numbers for ECA International offices are: Sydney +61 (0)2 9911 7799, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

1 0

Re: [Koha-devel] full text search
by paul POULAIN 10 Dec '02

10 Dec '02

Anthony W. Youngman a écrit: > Have you read the MySQL licence ... > > I don't know whether you're that bothered (Koha being gpl and all > that), but the MySQL licence specifically splits systems built on top > of it into ones that *can* *use* MySQL as one back end among many, and > others that *need* *MySQL*. > > As I say, since Koha is gpl maybe you don't care, but maybe others > might ... > > And if you're going to do this, for $DEITY's sake make it an > *optional* module, so that should people not want MySQL, they can just > leave this module out and then they're still independent ... > In fact, i think that the "need" mySQL refers to licence constraint : if your licence disable the opportunity for the user to use something else, then mySQL must be buyed. For us, as it's gpl, the "must" is a technical constraint, which is not covered by the mySQL constraint. MANY softwares works only on mySQL (1 "replace", which is mySQL-only, and your software is mysql dependant. And almost everybody uses replace when developping on mySQL) -- Paul POULAIN Consultant indépendant en logiciels libres responsable francophone de koha (SIBGB libre http://fr.koha.org)

4 4