http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #1 from Frédérick Capovilla <fcapovilla@live.ca> --- Created attachment 23145 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=23145&action=edit Add support for custom Z39.50 attributes to authority searches. Second patch to add support for custom PQF attributes to authority Z39.50 searches -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 Frédérick Capovilla <fcapovilla@live.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|Small patch |Medium patch --- Comment #2 from Frédérick Capovilla <fcapovilla@live.ca> --- Setting patch complexity to medium because of small DB changes. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 Manos PETRIDIS <egpetridis@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |egpetridis@yahoo.com -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #4 from Frédérick Capovilla <fcapovilla@live.ca> --- Unfortunately, I cannot provide a use-case you could reproduce because we encountered our search problems on a private server requiring a subscription with a username and a password. In our case, the BANQ server would not return results if we did not specify a value for "@attr 4". With this patch, we can set default PQF attributes for a all searches on this Z39.50 server without affecting search results on other Z39.50 servers. This is useful if we want to do a single Z39.50 search on multiple servers at the same time, each with their own supported or required attributes. For example, specifying "@attr 4=1" on server 1 will not cause an error on server 2 if this one doesn't support "@attr 4" attributes. -- Finally, what you are describing affects "Use" (@attr1) attributes. The main use of this patch is to set the other attributes which won't change depending of the used search terms. However, the same methods could be used to implement a "mapping table". This patch changes the PQF query generation technique to use an Array instead of a String, so each part of the query can easily be overwritten to fit each server's requirements. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #6 from Frédérick Capovilla <fcapovilla@live.ca> ---

Yes, I know that I was describing "Use" attributes. I'm fairly familiar with Bib-1 attributes. I was referring to configuring the "Use" attributes at the server level (since different servers sometimes use different "Use" attributes for the same access point), as you were mentioning setting "Structure" attributes at the server level. In theory, you would map attributes to access points. In this case, maybe one would map @attr 1=4 @attr 4=1 to "Title" so that the default Z39.50 "Title" search would be a "Title Phrase" search. You could also do something like @attr 1=1016 @attr 4=6 to do a "Keyword" search. Since the mappings would be at the server level, you wouldn't have to worry about servers that couldn't support "Structure" attributes.

I understood what you were saying in your first reply, and it would be a great to add this feature in Koha. I was just explaining that the attached patch should not be used to set "Use" attributes in its current state, because it would force a single "Use" value for all searches.

I disagree that the "@attr 4" "Structure" attributes won't change depending on the search terms. As I mention in the above paragraph, you might want to use @attr 4=1 (phrase) for a phrase search, but you might want to use @attr 4=4 (4 digit year) for a year search. Or maybe @attr 4=6 (word list) where you you're performing a more "Google-like" search. That said, Bib-1 attributes are quite esoteric. Most staff users probably wouldn't know that they could even provide them. Of course, it's probably more likely to have Koha admin than a staff members setting up Z39.50 servers, so it might not matter.

Normally, Z39.50 servers should have a default "Structure" value that is automatically used if the "Structure" attribute is not provided in the search query. In our case, the BANQ server does not provide a default value and requires us to always provide this attribute. The new preference added by this patch is there to force a default value for this server so it actually returns results. It could be used on other servers with special attribute requirements.

After looking at the patch, it looks like it would work for your use case, but it seems like a bit of a hack to me. I also wonder how many people will use it incorrectly.

Maybe it does make sense to make all queries for a server use one structure attribute like 6 (word list), but I think it makes more sense to use a drop-down menu than a free text field. That said, the free text field is more flexible (albeit more error prone).

Since the preference is kind of a "power user" feature with rare use cases, I didn't think it was worth the effort to create a special field structure for this one feature. And, like you said, the free text field allows more flexibility.

I don't mean to sound harsh or overly critical. I'm just trying to add another perspective.

I'm happy to receive constructive criticism like yours. In the end, I agree that you idea of using some sort of "mapping table" would be the ideal solution for many Z39.50 search problems including ours. In our case, with that kind of feature, we could simply add "@attr 4=1" to all search terms in the mapping table for the BANQ server and it would fix our problem. For now, this patch fixes a problem we are experiencing with one of the Z39.50 servers we use. It also improves the PQF query generation code of Z39.50 searches so query attributes can be overwritten on a per-server basis. This patch could be improved to permit more advanced per-server query manipulations. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 Blou <philippe.blouin@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eric.begin@inLibro.com, | |philippe.blouin@inlibro.com | |, | |simith.doliveira@inlibro.co | |m -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 Maxime Beaulieu <maxime.beaulieu@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |maxime.beaulieu@inlibro.com Assignee|fcapovilla@live.ca |maxime.beaulieu@inlibro.com -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #9 from Marc Véron <veron@veron.ch> --- Created attachment 40889 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=40889&action=edit [Signed-off] Bug 11297 - Add support for custom PQF attributes for Z39.50 server searches. Adds the "Attributes" field to z3950 servers. The feature here is not quite de same. In the old patches, the attributes were applied to individual query parts if the part already contains "@attr" and the additionnal attribute is not already in the query part. Here, the content of the new field is prepended to all PQF queries sent to the server. This new way of doing is simpler and works for the sponsor. Test plan: I) Apply the patch II) Run updatedatabase.pl 1) Add a new z3950 server with the following parameters: Hostname : catalogue.banq.qc.ca Port : 210 Database : IRIS Syntax : Marc21 2) Perform a z3950 search on that server. Keyword (Any) : egypt 2.1) Nothing Found. 3) Add attributes on the server administration page @attr 4=1 4) Perform the same z3950 search 4.1) A lot of results modified: C4/Breeding.pm modified: Koha/Schema/Result/Z3950server.pm modified: admin/z3950servers.pl new file: installer/data/mysql/atomicupdate/Bug11297_z3950servers_attributes.sql modified: installer/data/mysql/kohastructure.sql modified: koha-tmpl/intranet-tmpl/prog/en/modules/admin/z3950servers.tt Followed test plan, works as expected. Signed-off-by: Marc Véron <veron@veron.ch> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 Marc Véron <veron@veron.ch> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #11 from Blou <philippe.blouin@inlibro.com> --- fyi, There is nothing in the current patch that would prevent "4=109 and 4=1". -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #13 from Blou <philippe.blouin@inlibro.com> --- Actually, I said that because the old patch (which was waaay impossible to apply anymore) would have replaced one 4= by another. In this new patch, Maxime prepends the 4=1 to whatever is there by default (4=109, let say), so I figure both are sent. small characters: But I might be wrong. :) -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #15 from David Cook <dcook@prosentient.com.au> --- (In reply to Blou from comment #13)

Actually, I said that because the old patch (which was waaay impossible to apply anymore) would have replaced one 4= by another. In this new patch, Maxime prepends the 4=1 to whatever is there by default (4=109, let say), so I figure both are sent.

small characters: But I might be wrong. :)

I just took a look at the patch, and that sounds...interesting. I'd have to do more research, but http://www.loc.gov/z3950/agency/defns/bib1.html recommends not repeating attribute types in an attribute list. I imagine server behaviour might vary here as well... I imagine some servers might use the first value (so 4=1 even though 4=109 comes later), or might return an error, or do some other behaviour. I think what Jonathan was saying is that there's no way to provide 4=1 and 4=109 in a single query. For instance: @and @attr 1=4 @attr 4=1 "The cat in the hat" @attr 1=12 @attr 4=109 "123456789" The patch would only allow the following: @attr 4=1 @and @attr 1=4 "The cat in the hat" @attr 1=12 "123456789" Which effectively is the same as @and @attr 1=4 @attr 4=1 "The cat in the hat" @attr 1=12 @attr 4=1 "123456789" -- This could cause a problem if "123456789" isn't stored in a phrase index but is instead stored in a numeric index. That being said, Zebra's default structure attribute is phrase (4=1)(http://www.indexdata.com/zebra/doc/querymodel-rpn.html). But the default structure attribute can range from server to server... so I don't think it would be wise to add default structure attributes for all servers... -- After some musing, I think that I'm OK with this patch. Overall, I don't like it. I rather given control of the structure to the user doing the Z39.50 search. However, since the user is only inputting the terms and we currently control everything else about the PQF query formation, perhaps it doesn't hurt to allow us to provide values that apply to the whole query. That being said, I think this attribute field needs a validator. Only valid PQF should be allowed in this field, and of that PQF, USE attributes should never ever be allowed. I'm skeptical about other attribute types being allowed as well, but I suppose they may be necessary in the event that a Z39.50 target doesn't have default values... -- I also think tests should be done to see what happens when you provide a structure attribute at the query level and what happens when you provide a different one at the term level... in case we want to change the structure attribute for specific fields in the future... Anyway, that's my 2 cents ;) -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #17 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to David Cook from comment #16)

Ultimately... I've changed my mind and I'm now in favour of this feature... but I think is needs to be validate the attributes that the user is providing.

So could we get your signoff, 2 is better than 1! :) -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #19 from David Cook <dcook@prosentient.com.au> --- Probably won't have time to test this one actually... -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #20)

- Last but not least: I'd really like to see a second use case where this feature is helpful - so not only the one server causing problems, but maybe another good/clever idea on how to use this feature.

It's not just about the one server. It theoretically could be any server. http://www.loc.gov/z3950/agency/bib1.html: "Within an attribute list, each attribute type is optional. However, if a particular attribute type is not supplied, this document does not address target behavior -- a given target might supply a default attribute, dynamically select an appropriate attribute based on the other attributes supplied, or fail the search because it requires that the attribute type be supplied." At the moment, we don't have a way for users to specify Bib-1 attributes, so some servers will fail for all Z39.50 searches. Without this patch (or a similar one), I think the way we do Z39.50 searches is short-sighted. While Zebra will provide default attributes, it's clear that not all Z39.50 targets do. I don't think there are any other good/clever ideas for other ways to use this feature. It basically lets you provide attributes that are applied to the rest of the query. I suppose in other use cases... it lets you specify new defaults. For instance, Zebra will default to a Structure attribute of "phrase". Maybe you would prefer to default to a Structure attribute of "word list" instead, so you can specify that using this patch, and there you go. Or maybe you want to use a "relevance relation" for all your queries to a target, so you specify @attr 2=102. Or maybe you don't want to truncate anything in your queries so you supply @attr 5=100. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #23 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #22)

Hi David, thx for the quick reply and the examples - I'll say that reasolved that point in my list. Your examples would be great for the help page.

I keep meaning to update the manual and help pages in regards to Zebra and Z39.50... I'll keep this on my to do list! -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11297 --- Comment #24 from Blou <philippe.blouin@inlibro.com> --- If I summarize, - Need for the help file to be modified - Require dropdown at the server level. We do not understand that last one : "As David said, I don't think it's a good idea to add this field at the server level, we could have the need to use 4=109 and 4=1 in the same query. A dropdown list in front of each field sounds more useful and flexible." The 4=109 4=1 are currently possible with this patch. As for the dropdown, where would that be? I can't picture it, I'm sorry. But wherever, that would be a more complex patch for a specialized need that "if you need it, you'll know what to do here". IMO. I'm sorry I can't offer any other use case. For us it's major since EVERYBODY requires this since this is our national library's z3950 that needs it. -- You are receiving this mail because: You are watching all bug changes.