[Koha-bugs] [Bug 35812] Should specify canonical URLs to help search indexers

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35812

--- Comment #4 from M <schodkowy.omegi-0r at icloud.com> ---
So there's a couple of issues with sitemap and robots.txt approach:

1. It's been a week and Google apparently can decide whether it even ever feels
like even downloading the sitemap and pages on it, which in our case for a
small site it decided not to do at all (apparently it's a common issue upon
some research)

2. Thus blocking opac-search.pl with robots.txt limits the indexability of the
page by Googlebot, because it's the only way effective to discover pages

The solution would be to:

1. Have canonical links for bibs as described above

2. Allow bots to access opac-search.pl, but have "noindex" <meta> attribute
(but no "nofollow"!!). Contrary to robots.txt, this will allow the bot to
download the page and extract links from it to index them, but will not index
the search page itself. Ideal.

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.