Skip crawling for new API documentation #26155
Conversation
k8s-ci-robot
added
cncf-cla: yes
k8s-ci-robot
added
the
sig/docs
|
✔️ Deploy preview for kubernetes-io-master-staging ready! 🔨 Explore the source changes: ec8deb5 🔍 Inspect the deploy logs: https://app.netlify.com/sites/kubernetes-io-master-staging/deploys/6007723cee14a00007fceb35 😎 Browse the preview: https://deploy-preview-26155--kubernetes-io-master-staging.netlify.app |
layouts/robots.txt
Outdated
| @@ -6,4 +6,10 @@ Disallow: /v1.1/ | |||
| Disallow: /404/ | |||
| Disallow: /404.html | |||
|
|
|||
| Allow: /docs/reference/generated/kubernetes-api/ | |||
There was a problem hiding this comment.
is this line the same (save for the disallow) as line 13? (allowing and then disallowing the same path)
There was a problem hiding this comment.
The disallow was originally Disallow: /docs/reference/generated/kubernetes-api/* and I amended it.
This prepares for the launch of new API documentation, initially without indexing by search engines.
sftim
force-pushed
the
20210119_robots_txt_prepare_for_new_api_docs
branch
from
862a182
to
ec8deb5
Compare
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: e7a5e8b54572c1031f3dd74846530b40b5988451
|
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: zacharysarah The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
Small change to lay the path for merging PR #23294. Approach as per #23294 (comment) but I tweaked it based on what I think provides broadest compatibility: I want to cover many search engines both current and not.
This prepares for the launch of new API documentation, initially without indexing by search engines.