[Bug 25532] New: Add a wikipedia "Did you mean" style suggestion to the OPAC search
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Bug ID: 25532 Summary: Add a wikipedia "Did you mean" style suggestion to the OPAC search Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: OPAC Assignee: oleonard@myacpl.org Reporter: david.roberts@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org Some of our (PTFS-E) customer libraries like to have a "Did you mean" feature in the OPAC to help users who mis-spell their search terms. We have developed this using jQuery and Wikipedia, but it would be nice to have this in the core code, behind a system preference for those who don't want it. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|oleonard@myacpl.org |david.roberts@ptfs-europe.c | |om -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Sally <sally.healey@cheshiresharedservices.gov.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sally.healey@cheshireshared | |services.gov.uk -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Add a wikipedia "Did you |Add a wikipedia-style |mean" style suggestion to |search suggestion feature |the OPAC search |to the OPAC -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #1 from David Cook <dcook@prosentient.com.au> --- We created a local package for this called Koha::SuggestionEngine::Plugin::CommonMisspellings which our clients use, but I don't think it's designed well enough for master. I'd be interested in testing out something else though that meets this need. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 --- Comment #2 from David Roberts <david.roberts@ptfs-europe.com> --- Created attachment 105138 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105138&action=edit Bug 25532: Add a wikipedia-style search suggestion feature to the OPAC This patch adds an auto-complete function to the OPAC search and a new system preference 'OPACSearchAutoComplete' To test: 1) Install the patch 2) Go to the OPAC and start typing a search term, e.g. Harry Potter 3) You should see a drop down will appear suggesting likely search terms which can be selected 4) Repeat the search, but mis-spell your search term, e.g. Hrry Potter. 5) You should still see the correctly spelled search term being offered for selection 6) Check that the search is unaffected, and still works correctly, both when logged in and not logged in. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff --- Comment #3 from David Roberts <david.roberts@ptfs-europe.com> --- (In reply to David Cook from comment #1)
We created a local package for this called Koha::SuggestionEngine::Plugin::CommonMisspellings which our clients use, but I don't think it's designed well enough for master.
I'd be interested in testing out something else though that meets this need.
Thanks David - our customers really like this feature, but obviously it does depend on there being a relevant Wikipedia page in order to be able to correct the spelling, or make a suggestion! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 --- Comment #4 from Owen Leonard <oleonard@myacpl.org> --- This is really interesting, and works well. However, I'm concerned about the possibility of confusion on the part of the user. My assumption is that if a user is familiar with how autocomplete works, they will assume that autocomplete is pulling from a list of existing values. I think it will be confusing for a patron to select from the suggested Wikipedia terms only to find that nothing in the catalog matches. It's nonsensical to see in the suggestions an entry like "Harry Potter and the Deathly Hallows – Part 1 (disambiguation)." A partial solution might be to add more explicit labeling to the autocomplete. Something like, "Search terms by Wikipedia" or something like that. I would suggest that the preference be renamed as well to something like "OPACWikipediaSearchSuggestions." -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |ASSIGNED --- Comment #5 from David Roberts <david.roberts@ptfs-europe.com> --- (In reply to Owen Leonard from comment #4)
This is really interesting, and works well. However, I'm concerned about the possibility of confusion on the part of the user.
My assumption is that if a user is familiar with how autocomplete works, they will assume that autocomplete is pulling from a list of existing values. I think it will be confusing for a patron to select from the suggested Wikipedia terms only to find that nothing in the catalog matches.
It's nonsensical to see in the suggestions an entry like "Harry Potter and the Deathly Hallows – Part 1 (disambiguation)."
A partial solution might be to add more explicit labeling to the autocomplete. Something like, "Search terms by Wikipedia" or something like that. I would suggest that the preference be renamed as well to something like "OPACWikipediaSearchSuggestions."
I'd go along with that - I had trouble thinking what to call this feature when I was writing it! I'll change the naming. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 --- Comment #6 from David Cook <dcook@prosentient.com.au> --- Ahhh I thought it was a "Did you mean" feature for after you've done a search, but it's a search suggestion before the search is done. Neato. I agree with what Owen has said. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105138|0 |1 is obsolete| | --- Comment #7 from David Roberts <david.roberts@ptfs-europe.com> --- Created attachment 105220 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105220&action=edit Bug 25532: Add a wikipedia-style search suggestion feature to the OPAC This patch adds an auto-complete function to the OPAC search and a new system preference 'OPACSearchAutoComplete' To test: 1) Install the patch 2) Go to the OPAC and start typing a search term, e.g. Harry Potter 3) You should see a drop down will appear suggesting likely search terms which can be selected 4) Repeat the search, but mis-spell your search term, e.g. Hrry Potter. 5) You should still see the correctly spelled search term being offered for selection 6) Check that the search is unaffected, and still works correctly, both when logged in and not logged in. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Roberts <david.roberts@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff --- Comment #8 from David Roberts <david.roberts@ptfs-europe.com> --- I've implemented the suggestions made so far - you will now see the sys pref has been renamed to OPACWikipediaSearchSuggestions, and you will now see a "Powered by Wikipedia" strap line on the drop down of search suggestions. I've also just realised that I neglected to suggest in my test plan that you actually switch the feature on! So the test plan is: 1) Install the patch and enable the new OPACWikipediaSearchSuggestions system preference. 2) Go to the OPAC and start typing a search term, e.g. Harry Potter 3) You should see a drop down will appear suggesting likely search terms which can be selected 4) Repeat the search, but mis-spell your search term, e.g. Hrry Potter. 5) You should still see the correctly spelled search term being offered for selection 6) Check that the search is unaffected, and still works correctly, both when logged in and not logged in. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 ByWater Sandboxes <bws.sandboxes@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #105220|0 |1 is obsolete| | --- Comment #9 from ByWater Sandboxes <bws.sandboxes@gmail.com> --- Created attachment 105488 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105488&action=edit Bug 25532: Add a wikipedia-style search suggestion feature to the OPAC This patch adds an auto-complete function to the OPAC search and a new system preference 'OPACSearchAutoComplete' To test: 1) Install the patch 2) Go to the OPAC and start typing a search term, e.g. Harry Potter 3) You should see a drop down will appear suggesting likely search terms which can be selected 4) Repeat the search, but mis-spell your search term, e.g. Hrry Potter. 5) You should still see the correctly spelled search term being offered for selection 6) Check that the search is unaffected, and still works correctly, both when logged in and not logged in. Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Sally <sally.healey@cheshiresharedservices.gov.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Joonas Kylmälä <joonas.kylmala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joonas.kylmala@helsinki.fi --- Comment #10 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Instead of depending on Wikipedia being online and transferring users' queries there I think we should use instead one of the search engines Koha supports to do this, Elasticsearch seems to have support for auto completion and correction: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-sugge... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Joonas Kylmälä <joonas.kylmala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |In Discussion --- Comment #11 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Just a gentle reminder I would like to hear others' (especially people using this feature) thoughts on the Elasticsearch vs Wikipedia approach to this. Also have you thought about having this JavaScript code in OPACUserJS? I tried to put it there and it worked flawlessly. Would it be OK solution if we just documented the steps in wiki for example on how to add this feature to the syspref? Some potential pros & cons: Elasticsearch: + user queries stay within Koha instance, no privacy leak + tailored suggestions for the Koha database - more difficult to implement Wikipedia: + simple to implement - suggestion feature availability depends on wikipedia - search queries go to third party server +/- suggestions might provide no results / not tailored to the Koha database -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |oleonard@myacpl.org --- Comment #12 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I am torn here, this works actually really nicely just by putting it into OpacUserJs. And we could document it on https://wiki.koha-community.org/wiki/JQuery_Library but I believe without further "noise" not a lot of libraries will find it there. Adding it to Koha would ensure that and make it really easy to use.
Some potential pros & cons:
Elasticsearch: + user queries stay within Koha instance, no privacy leak + tailored suggestions for the Koha database - more difficult to implement
I think it's not an either or - this can work for libraries until we implement something using Elasticsearch. So yes, having Elasticsearch suggestions woudl be great, but I think that's a separate discussion to have.
Wikipedia: + simple to implement - suggestion feature availability depends on wikipedia - search queries go to third party server +/- suggestions might provide no results / not tailored to the Koha database
The third party is a valid argument (thinking of data privavy, GDPR), but could be remedied by a clear hint and link to terms of use or similar in the pref description. We offer other third party services too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Henry Bolshaw <bolshawh@parliament.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bolshawh@parliament.uk --- Comment #13 from Henry Bolshaw <bolshawh@parliament.uk> --- I think it would be good to have this functionality in Koha itself rather than OpacUserJS because this would: - make the functionality more visible to users and therefore increase the number of people using and benefitting from the functionality - improve the feature documentation e.g. we can add info to the manual - increase the potential number of users and developers working with the feature making it more likely that it will iterate and improve (if we add the code to the wiki then it will probably remain there unchanged indefinitely) We probably wouldn't use this feature in our library currently - the wikipedia search suggestions don't match the titles in our collection very well although I'm sure they do work for some other libraries - so an elasticsearch version based on the koha database would be preferable. However, I agree with Katrin that we don't have to choose one: we can have both this version and an elasticsearch version at a later date. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 --- Comment #14 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Ok, so we all agree having auto-completion would be a great and much wanted feature to have. I still think we should not rush implementing this using the Wikipedia API because it seems like it would not be the ideal solution for many users, if not for all them. Instead the database customized search results using for example Elasticsearch's features seems to be the ideal solution here. Why I think not to rush this, and not to go with Wikipedia suggestions: - Double the maintenance work when Wikipedia and Elasticsearch based implementations are both implemented - The deprecation of Wikipedia implementation will take probably few releases to make sure everybody has stopped using it and switches to Elasticsearch implementation. Meaning the feature still has to be maintained all this time. On a side note: I couldn't find any guidelines for deprecating features in Koha. There are already some cases I think doing just that would help the Koha users because for example the local hold groups feature with correct circulation rules could probably replace the system preference OPACHoldsIfAvailableAtPickup. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Niamh Walker-Headon <Niamh.WalkerHeadon@hse.ie> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |Niamh.WalkerHeadon@hse.ie -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 --- Comment #15 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- In OpacUserJs: $(document).ready(function(){ $("#translControl1").autocomplete({ source: function(request, response) { $.ajax({ url: "https://en.wikipedia.org/w/api.php", dataType: "jsonp", data: { 'action': "opensearch", 'format': "json", 'search': request.term }, success: function(data) { response(data[1]); } }); }, open: function() { $('.ui-autocomplete').append('<li style="text-align:right;font-size:80%;padding: 5px 10px;background:rgba(0,0,0,0.1)">Powered by Wikipedia</li>'); } }); }); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=27113 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #16 from Lucas Gass <lucas@bywatersolutions.com> --- As of 22.05 the jQuery Katrin provided will no longer work in Koha since we've removed jQueryUI and 'autocomplete'. Maybe it is time to revisit this as an ES implementation? Some libraries will miss the autocompete feature provided via jQuery. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |MOVED Status|In Discussion |RESOLVED --- Comment #17 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Lucas Gass from comment #16)
As of 22.05 the jQuery Katrin provided will no longer work in Koha since we've removed jQueryUI and 'autocomplete'.
Maybe it is time to revisit this as an ES implementation? Some libraries will miss the autocompete feature provided via jQuery.
There is already some work on that - I'd love to see comments and get it moving: bug 27113. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25532 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #18 from Fridolin Somers <fridolin.somers@biblibre.com> --- Seeing this JS code inspired me to create a plugin : https://github.com/biblibre/koha-plugin-autocomplete-with-wikipedia Enjoy ;) -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org