From bibliwho@gmail.com Tue Nov  6 15:32:47 2012
From: Cab Vinton <bibliwho@gmail.com>
To: koha-devel@lists.koha-community.org
Subject: [Koha-devel] Searching for garbage characters
Date: Tue, 06 Nov 2012 09:32:02 -0500
Message-ID:
 <CABW43UQp5MW0YKhP-B0Z_o9o1z9b+S+s=5LoA3CDz0nH3twVSA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============4542549198912153481=="

--===============4542549198912153481==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

We've managed to import a number of MARC records with corrupted
diacritics and my attempts to retrieve these with a report haven't met
w/ success. (Sample records in this list:
http://catalog.splnh.com/cgi-bin/koha/opac-shelves.pl?viewshelf=3D8.)

My thought is to search for 100, 700, etc. tags containing any
characters outside of the ASCII 32 through 126 range, but my regex
skills aren't up to the task. To wit:

SELECT CONCAT('<a
href=3D\"/cgi-bin/koha/catalogue/detail.pl?biblionumber=3D',biblionumber,'\">=
',biblionumber,'</a>')
AS bibnumber, pname
FROM
(SELECT biblionumber,
ExtractValue(marcxml,'//datafield[@tag=3D"100"]/subfield[@code>=3D"a"]')
AS pname FROM biblioitems)
AS authors
WHERE pname REGEXP '[\W]'

These attempts also didn't seem to be getting me any closer:

WHERE pname REGEXP '[^a-z]'

WHERE pname LIKE '%[^a-zA-Z0-9]%'

WHERE PATINDEX('%[^a-zA-Z0-9]%',pname) > 1

Any thoughts on how to write this report? Have tried the folks over on
the MarcEdit list, but no solution as yet.

Many thanks,

Cab Vinton, Director
Sanbornton Public Library
Sanbornton, NH

Life is short. Read fast!

--===============4542549198912153481==--


From mtj@kohaaloha.com Tue Nov  6 21:03:21 2012
From: Mason James <mtj@kohaaloha.com>
To: koha-devel@lists.koha-community.org
Subject: Re: [Koha-devel] Searching for garbage characters
Date: Wed, 07 Nov 2012 09:03:01 +1300
Message-ID: <889AF8AE-0E02-40C4-8863-E48B6D2A9AFA@kohaaloha.com>
In-Reply-To:
 <CABW43UQp5MW0YKhP-B0Z_o9o1z9b+S+s=5LoA3CDz0nH3twVSA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2196645394443743476=="

--===============2196645394443743476==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable


On 2012-11-7, at 3:32 AM, Cab Vinton wrote:

> We've managed to import a number of MARC records with corrupted
> diacritics and my attempts to retrieve these with a report haven't met
> w/ success. (Sample records in this list:
> http://catalog.splnh.com/cgi-bin/koha/opac-shelves.pl?viewshelf=3D8.)
>=20
> My thought is to search for 100, 700, etc. tags containing any
> characters outside of the ASCII 32 through 126 range, but my regex
> skills aren't up to the task. To wit:
>=20


consider dumping your bibs out to a file, and use your favorite text editor a=
nd some regexs to help find those bad chars, too

--===============2196645394443743476==--