http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédérick Capovilla <fcapovilla@live.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 |PATCH-Sent Patch Status|--- |Needs Signoff -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédérick Capovilla <fcapovilla@live.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #4892|0 |1 is obsolete| | --- Comment #3 from Frédérick Capovilla <fcapovilla@live.ca> 2011-10-13 19:01:41 UTC --- Comment on attachment 4892 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=4892 Remove Koha-specific fields from records imported via Z39.50 A bug was found in this patch : If a field contains only one subfield and this subfield is $9, the $9 gets deleted, but the field isn't deleted. This can cause the record to become invalid MARC (empty field). Patch marked as obsolete. Keeping the bug open. A new, corrected version of this patch can be sent when it is adapted for Koha version 3.6 . -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #4 from Frédérick Capovilla <fcapovilla@live.ca> 2011-12-23 15:34:08 UTC --- Created attachment 6929 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=6929 Corrected patch, ported to Koha 3.6 Here is a corrected version of this patch. Empty fields should now be deleted. The patch was also ported to Koha 3.6 -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #5 from Frédérick Capovilla <fcapovilla@live.ca> 2011-12-23 15:35:19 UTC --- Patch sponsored by the CCSR ( http://www.ccsr.qc.ca ) -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chris@bigballofwax.co.nz Patch Status|Needs Signoff |Failed QA --- Comment #6 from Chris Cormack <chris@bigballofwax.co.nz> 2011-12-31 06:58:02 UTC --- Hmm it looks to me like this only does MARC21, and it will try to do it for UNIMARC and NORMARC probably making a mess of the record. I think it needs to check the syspref for MARC Flavour and do the right thing for each flavour. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédérick Capovilla <fcapovilla@live.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch Status|Failed QA |Needs Signoff -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch Status|Needs Signoff |Signed Off --- Comment #9 from Chris Cormack <chris@bigballofwax.co.nz> 2012-01-13 03:48:23 UTC --- Signed off, this now has the added benefit of fixing neworderempty.pl which used to blindly delete 952, wether the record was UNIMARC, NORMARC or MARC21. A follow up to add removing the authority id's for UNIMARC and NORMARC would be good. But this introduces no regressions, adds a feature and fixes a bug so should not have to wait for a follow up. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #11 from Frédérick Capovilla <fcapovilla@live.ca> --- Having never worked with unimarc or normarc records and Z39.50 servers, we are not equipped to easily adapt our patch for it to work with these formats. Could anyone improve this patch so it becomes flexible enough for inclusion on master? -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #12 from simith.doliveira@inlibro.com --- Created attachment 30795 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30795&action=edit This patch update the code and fix MARC flavour problem by removing all subfields $9 from the imported record. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 simith.doliveira@inlibro.com changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|PATCH-Sent (DO NOT USE) |P2 Status|Failed QA |Signed Off Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #15 from simith.doliveira@inlibro.com --- Created attachment 31181 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31181&action=edit Bug 6681 - When importing a biblio record via Z39.50, authority links and items also get imported. This patch removes the content of subfield 9 for all authority headings and items in biblios imported from a Z39.50 server. Changes: C4/Biblio.pm - new function RemoveItemsAndAuthidsFromRecord added acqui/neworderempty.pl - calls new function cataloguing/addbiblio.pl - calls new function t/Biblio.t - new tests added Testing: I Before applying the patch 0) Add a koha z39.50 server (Home › Administration › Z39.50 servers); 1) in Home > Cataloging click <<New from z39.50>>; 2) Import a record with an autority link (with zone 100 subzone 9 for example); 3) Validate: The record imported have zone 100, subzone 9; II After applying the patch 1) in Home > Cataloging click <<New from z39.50>>; 2) Import a record with an autority link (with zone 100 subzone 9 for example); 3) Validate: The record imported don't have the subzone 9 in the zone 100; -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 simith.doliveira@inlibro.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |tomascohen@gmail.com --- Comment #17 from Tomás Cohen Arazi <tomascohen@gmail.com> --- I'm not sure about the approach. Comments: 1) We should fix Koha's public server configuration so that it doesn't share information we don't want it to. Maybe applying an XSLT transformation. 2) We still might use Z39.50 targets that give us wrong data in their records (not Koha servers for example), so a way of defining what to remove is needed. This is harder to implement, but it can be achieved Z39.50 server-wise, using YAML or the usual | to separate fields$subfields. 3) This should be an opt-in, people who wants stuff removed should be able to choose to do it, not a forced record munging. So, if you continue with the current approach, please introduce a way to control it. 4) I like that you capture the warnings on the unit tests, and the fact that there are unit tests :-D But we need functional tests, in the sense that the behaviour should be tested. I can help if you need advise on the tests writing. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #31181|0 |1 is obsolete| | --- Comment #19 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 33177 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33177&action=edit Bug 6681: introduce XSLT scripts to remove items and links This patch adds the following sample files: - Del952.xsl - Del995.xsl - Del9LinksExcept952.xsl - Del9LinksExcept995.xsl Thay can be used on the new SRU/Z39.50 XSLT processing feature. At the same time they can be used to solve this bug. To test: - Have an SRU/Z39.50 target configured - Have a search that returns at least a record with the following properties: * It contains $9 links * It contains items (952 or 995 fields in MARC21/NORMARC and UNIMARC respectively). MARC21/NORMARC test: - Do a Z39.50/SRU cataloguing search that retrieves the desired record - Verify it includes $9 and 952 field(s) by clicking on the MARC link - Edit your Z39.50 target and add Del952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view of the imported record doesn't contain the 952 field - Edit your Z39.50 target and add Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view contains the 952 field (including the $9 subfield), and all other $9 subfields have been removed - Edit your Z39.50 target and add Del952.xsl, Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field (both, comma separated) - Re-run the search => SUCCESS: the MARC view doesn't contain $9 subfields nor items. - Repeat for UNIMARC, replacing 952 for 995. - Sign off :-D Regards Tomas Sponsored-by: Universidad Nacional de Cordoba -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |frederic@tamil.fr --- Comment #20 from Frédéric Demians <frederic@tamil.fr> --- It works well. This would be a good thing to have those XSL files in Koha. And at the first place, bug 6536 offers new interesting perspectives. But aren't this functionality (XSL transfo of z3950 search result) and those specific XSL files under-documented? -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33407|0 |1 is obsolete| | --- Comment #24 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 33410 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33410&action=edit Bug 6681: introduce XSLT scripts to remove items and links This patch adds the following sample files: - Del952.xsl - Del995.xsl - Del9LinksExcept952.xsl - Del9LinksExcept995.xsl Thay can be used on the new SRU/Z39.50 XSLT processing feature. At the same time they can be used to solve this bug. To test: - Have an SRU/Z39.50 target configured - Have a search that returns at least a record with the following properties: * It contains $9 links * It contains items (952 or 995 fields in MARC21/NORMARC and UNIMARC respectively). MARC21/NORMARC test: - Do a Z39.50/SRU cataloguing search that retrieves the desired record - Verify it includes $9 and 952 field(s) by clicking on the MARC link - Edit your Z39.50 target and add Del952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view of the imported record doesn't contain the 952 field - Edit your Z39.50 target and add Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view contains the 952 field (including the $9 subfield), and all other $9 subfields have been removed - Edit your Z39.50 target and add Del952.xsl, Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field (both, comma separated) - Re-run the search => SUCCESS: the MARC view doesn't contain $9 subfields nor items. - Repeat for UNIMARC, replacing 952 for 995. - Sign off :-D Regards Tomas Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Tested under MARC21. Works fine. (Additionally verified 995 with xsltproc.) As noted on Bugzilla, we could still improve documentation but imo no need to block these example transformations. Signed-off-by: Frederic Demians <f.demians@tamil.fr> I can't resist to put my own signature on this patch, confirming it works as described with z39.50 Unimarc targets. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #25 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 33425 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33425&action=edit Bug 6681: (qa followup) document the existence of the sample files As requestedby people testing the patch, I add references to the new xslt files on the Z39.50/SRU servers help. Example usages are also provided. Test: - Apply the patch - Go to the help page on the 'Z39.50/SRU Servers' page => SUCCESS: Notice there's a section documenting XSLT file(s) usage and provides some examples that cover the introduced files. - Sign off Thanks Tomas Sponsored-by: Universidad Nacional de Cordoba -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33425|0 |1 is obsolete| | --- Comment #26 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 33427 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33427&action=edit Bug 6681: (qa followup) document the existence of the sample files As requestedby people testing the patch, I add references to the new xslt files on the Z39.50/SRU servers help. Example usages are also provided. Test: - Apply the patch - Go to the help page on the 'Z39.50/SRU Servers' page => SUCCESS: Notice there's a section documenting XSLT file(s) usage and provides some examples that cover the introduced files. - Sign off Thanks Tomas Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Frederic Demians <f.demians@tamil.fr> Help page does shed some light to the XSLT usage. Enough to my taste. -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 --- Comment #28 from David Cook <dcook@prosentient.com.au> --- (In reply to Frédéric Demians from comment #27)

For the future, a better way for storing XSL could be designed. In the disorder: - a z3950severs table new field, - a new db table for XSL, - a centralized repository, - a link to a Koha community page (like for SQL report library).

I've been thinking about this for a little while as well. I've been working again on an OAI harvester which uses XSLTs to filter out the 952, 999, and $9, and I figure "koha-tmpl\intranet-tmpl\prog\en\xslt" isn't always the best place to put these things. I'm intrigued by the idea of using a db field or db table. If we allowed uploads, that would make it easier for libraries to add their own XSLTs (for better or worse) without meddling around on the file system. Actually, a db table could be interesting. We could have the blob, the filename, an id... and then have a db field in z3950servers (or whatever other server config table) that has a foreign key to the primary key id in the XSLT table. I'd kind of like the ability to set defaults as well though... so you could set up XSLT profiles to apply on import (rather than having a string of filenames). Of course, we'd probably need a linking table then for profiles to files. I think that'd be a good way to go about setting up defaults though. You could have a "Default Z3950" profile (and maybe a "Default OAI" table). -- Actually, as I work on this OAI client, I start thinking about ways to integrate code and storage of Z3950 and OAI targets. I'm trying to think if there are other protocols we use for importing records as well... and how instead of making more tables, we could make some tables more generic. I wonder too about using XSLTs when importing records via the batch import, although that would probably slow things down a lot as you'd be going from Iso Marc to Marc Object to MARCXML to XSLT to MARCXML. I suppose I like the idea of making storage and code more generalizable... </ramble> -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 M. de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=12973 -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #33410|0 |1 is obsolete| | Attachment #33427|0 |1 is obsolete| | --- Comment #30 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 33540 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=33540&action=edit [PASSED QA] Bug 6681: introduce XSLT scripts to remove items and links This patch adds the following sample files: - Del952.xsl - Del995.xsl - Del9LinksExcept952.xsl - Del9LinksExcept995.xsl Thay can be used on the new SRU/Z39.50 XSLT processing feature. At the same time they can be used to solve this bug. To test: - Have an SRU/Z39.50 target configured - Have a search that returns at least a record with the following properties: * It contains $9 links * It contains items (952 or 995 fields in MARC21/NORMARC and UNIMARC respectively). MARC21/NORMARC test: - Do a Z39.50/SRU cataloguing search that retrieves the desired record - Verify it includes $9 and 952 field(s) by clicking on the MARC link - Edit your Z39.50 target and add Del952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view of the imported record doesn't contain the 952 field - Edit your Z39.50 target and add Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field - Re-run the search => SUCCESS: the MARC view contains the 952 field (including the $9 subfield), and all other $9 subfields have been removed - Edit your Z39.50 target and add Del952.xsl, Del9LinksExcept952.xsl to the "XSLT File(s) for transforming results:" field (both, comma separated) - Re-run the search => SUCCESS: the MARC view doesn't contain $9 subfields nor items. - Repeat for UNIMARC, replacing 952 for 995. - Sign off :-D Regards Tomas Sponsored-by: Universidad Nacional de Cordoba Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Tested under MARC21. Works fine. (Additionally verified 995 with xsltproc.) As noted on Bugzilla, we could still improve documentation but imo no need to block these example transformations. Signed-off-by: Frederic Demians <f.demians@tamil.fr> I can't resist to put my own signature on this patch, confirming it works as described with z39.50 Unimarc targets. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6681 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|When importing a biblio |Provide a way for removing |record via Z39.50, |authority links and |authority links and items |holdings on Z39.50 record |also get imported |import Severity|major |enhancement -- You are receiving this mail because: You are the QA Contact for the bug. You are watching all bug changes.