[Koha-bugs] [Bug 4042] Public OPAC search can fall prey to web crawlers

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4042

Bob Birchall <bob at calyx.net.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |In Discussion

--- Comment #3 from Bob Birchall <bob at calyx.net.au> ---
(In reply to Pablo AB from comment #2)
> As told here
> http://koha.1045719.n5.nabble.com/Help-100-CPU-utilization-running-Koha-
> tp5809357.html
> we could just put a robots.txt like this on /usr/share/koha/opac/htdocs:
> 
>   User-agent: *
>   Disallow:/cgi-bin/koha/opac-search.pl
>   Disallow:/cgi-bin/koha/opac-export.pl
>   Disallow:/cgi-bin/koha/opac-showmarc.pl
>   Disallow:/cgi-bin/koha/opac-ISBDdetail.pl
>   Disallow:/cgi-bin/koha/opac-MARCdetail.pl

This is the file we use:
----------------------------
Crawl-delay: 60

User-agent: *
Disallow: /

User-agent: Googlebot
Disallow: /cgi-bin/koha/opac-search.pl
Disallow: /cgi-bin/koha/opac-showmarc.pl
Disallow: /cgi-bin/koha/opac-detailprint.pl
Disallow: /cgi-bin/koha/opac-ISBDdetail.pl
Disallow: /cgi-bin/koha/opac-MARCdetail.pl
Disallow: /cgi-bin/koha/opac-reserve.pl
Disallow: /cgi-bin/koha/opac-export.pl
Disallow: /cgi-bin/koha/opac-detail.pl
Disallow: /cgi-bin/koha/opac-authoritiesdetail.pl
----------------------------

Can we mark this bug as resolved now?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are watching all bug changes.