[Koha-patches] [PATCH] Add instructions to INSTALL and README.robots about robots.txt

[Michael Hafen]

Instructions are given in INSTALL and README.robots about adding a robots.txt
file to the opac to prevent search engines from indexing Koha.
---
 INSTALL       |    1 +
 README.robots |   18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 0 deletions(-)
 create mode 100644 README.robots

diff --git a/INSTALL b/INSTALL
index df0990c..4e8c20b 100644
--- a/INSTALL
+++ b/INSTALL
@@ -56,6 +56,7 @@ Default installation instructions:
   (note that you will want to run Zebra in daemon mode for a production
   system)
 9. Browse to http://servername:8080/ and answer the questions
+10. Optionally add a robots.txt file.  See README.robots for details
 
 OR if you want to install all dependencies from CPAN and are root, you can
 replace steps 1-3 with "perl install-CPAN.pl" but this is non-standard and
diff --git a/README.robots b/README.robots
new file mode 100644
index 0000000..a08371d
--- /dev/null
+++ b/README.robots
@@ -0,0 +1,18 @@
+The robots.txt file.
+
+Search engines, when looking for sites to show in search results, will first
+look for the file /robots.txt.  If this file is found and has lines that apply
+to them they will do as instructed.  A very basic robots.txt follow as an
+example:
+
+# go away
+User-agent: *
+Disallow: /
+
+This tells every search engine that cares (User-agent: *) to not index the site
+(Disallow everything past /).
+
+If you have installed Koha to /usr/local/koha3 then this file would be placed
+in the directory /usr/local/koha3/opac/htdocs/.  This should prevent search
+engines from browsing every biblio record, and every view of each record, on
+your Koha install periodically.
-- 
1.6.3.3