[Bug 6411] New: add robots.txt file to allow good indexing, yet prevent searchbot thrashing
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Bug #: 6411 Summary: add robots.txt file to allow good indexing, yet prevent searchbot thrashing Classification: Unclassified Change sponsored?: --- Product: Koha Version: rel_3_4 Platform: All OS/Version: All Status: NEW Severity: enhancement Priority: P5 Component: Searching AssignedTo: gmcharlt@gmail.com ReportedBy: mason@kohaaloha.com QAContact: koha-bugs@lists.koha-community.org adding this proposed robots.txt file allows good searchbot indexing for Koha but also stops searchbots from thrashing Koha systems by refusing to traverse opac-search.pl urls. here's the proposed robots.txt file --------------- User-agent: * Disallow: /cgi-bin/koha/opac-search.pl --------------- -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 --- Comment #1 from Mason J. <mason@kohaaloha.com> 2011-05-26 07:51:08 UTC --- Created attachment 4271 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=4271 patch -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Mason J. <mason@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 |PATCH-Sent -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chris@bigballofwax.co.nz Patch Status|--- |Needs Signoff --- Comment #2 from Chris Cormack <chris@bigballofwax.co.nz> 2011-05-28 07:13:00 UTC --- Please remember to set the patch status to needs sign off so the patch isnt missed. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 --- Comment #3 from James, Mason <mtj@kohaaloha.com> 2011-05-30 08:36:37 UTC --- (In reply to comment #0)
adding this proposed robots.txt file allows good searchbot indexing for Koha
aah, i see there is already an README-robots.txt file in the repo i'll add this info to the 'README-robots.txt' file, instead -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 James, Mason <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|add robots.txt file to |add another example to |allow good indexing, yet |README.robots file |prevent searchbot thrashing | -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 James, Mason <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #4271|0 |1 is obsolete| | --- Comment #4 from James, Mason <mtj@kohaaloha.com> 2011-05-30 09:00:21 UTC --- Created attachment 4300 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=4300 new patch -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@enger.priv.no Patch Status|Needs Signoff |Signed Off --- Comment #5 from Magnus Enger <magnus@enger.priv.no> 2011-06-15 06:47:48 UTC --- Just adds another example to README.robots, not much to test beyond the fact that the patch applies. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #4300|0 |1 is obsolete| | --- Comment #6 from Magnus Enger <magnus@enger.priv.no> 2011-06-15 06:48:22 UTC --- Created attachment 4468 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=4468 Signed-off patch -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch Status|Signed Off |Patch Pushed --- Comment #7 from Chris Cormack <chris@bigballofwax.co.nz> 2011-07-05 02:50:39 UTC --- Pushed please test -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411 James, Mason <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED --- Comment #8 from James, Mason <mtj@kohaaloha.com> 2011-07-20 14:51:22 UTC --- tested, and closing... -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the QA Contact for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org