[Koha-bugs] [Bug 6411] New: add robots.txt file to allow good indexing, yet prevent searchbot thrashing

[bugzilla-daemon at bugs.koha-community.org]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6411

             Bug #: 6411
           Summary: add robots.txt file to allow good indexing, yet
                    prevent searchbot thrashing
    Classification: Unclassified
 Change sponsored?: ---
           Product: Koha
           Version: rel_3_4
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: Searching
        AssignedTo: gmcharlt at gmail.com
        ReportedBy: mason at kohaaloha.com
         QAContact: koha-bugs at lists.koha-community.org


adding this proposed robots.txt file allows good searchbot indexing for Koha

but also stops searchbots from thrashing Koha systems by refusing to traverse
opac-search.pl urls.

here's the proposed robots.txt file

---------------
User-agent: *
Disallow: /cgi-bin/koha/opac-search.pl
---------------

-- 
Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA Contact for the bug.