Skip to content

Commit

Permalink
Docs: Add robots.txt to only allow indexing of latest and stable (
Browse files Browse the repository at this point in the history
aiidateam#6517)

Currently, all versions of the documentation are indexed with the result
that google searches come up with very outdated versions and the latest
version is almost impossible to find. The `robots.txt` now disallows
any path from being indexed except for the `latest` and `stable`
versions of the documentation.
  • Loading branch information
khsrali authored and mikibonacci committed Sep 3, 2024
1 parent a4c63bc commit a707218
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,9 @@
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
html_show_copyright = False

# This is to tell search engines to index only stable and latest version
html_extra_path = ['robots.txt']

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

Expand Down
4 changes: 4 additions & 0 deletions docs/source/robots.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
User-agent: *
Allow: /projects/aiida-core/en/latest/
Allow: /projects/aiida-core/en/stable/
Disallow: /

0 comments on commit a707218

Please sign in to comment.