Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

self-hosted-runners: add admonitions, clarifications #423

Merged
merged 1 commit into from
Dec 12, 2022
Merged

self-hosted-runners: add admonitions, clarifications #423

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 46 additions & 23 deletions content/docs/self-hosted-runners.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ However, there are many great reasons to use your own runners: to take advantage
of GPUs, orchestrate your team's shared computing resources, or train in the
cloud.

<admon type="tip">
<admon type="info">

Check out the official documentation from
For background information on self-hosted runners, see the official
documentation from
[GitHub](https://help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners),
[GitLab](https://docs.gitlab.com/runner) and
[Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/runners) for more
information on self-hosted runners.
[Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/runners).

</admon>

Expand Down Expand Up @@ -147,15 +147,11 @@ pipelines:
</tab>
</toggle>

In the workflow above, the `launch-runner` step launches an EC2 `p2.xlarge`
In the workflow above, the `launch-runner` job launches an EC2 `p2.xlarge`
instance in the `us-west` region. The `train-and-report` job then runs on the
newly-launched instance. See [Environment Variables](#environment-variables)
below for details on the `secrets` required.

🎉 **Note that jobs can use any Docker container!** To use commands such as
`cml comment create` from a job, the only requirement is to
[have CML installed](/doc/install).

## Docker Images

The CML Docker images (`docker://iterativeai/cml` or
Expand All @@ -172,6 +168,14 @@ convention is `{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`:
For example, `docker://iterativeai/cml:0-dvc2-base1-gpu`, or
`docker://ghcr.io/iterative/cml:0-dvc2-base1`.

<admon type="tip">

**Using your own custom Docker images**: To use commands such as
`cml comment create`, make sure to
[install CML in your Docker image](/doc/install#docker).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically non-existent #id; will be addressed in #404


</admon>

## Options

The `cml runner` command supports many options (see the
Expand Down Expand Up @@ -200,8 +204,11 @@ The `cml runner` command supports many options (see the
or
[GCP instance service account](https://cloud.google.com/compute/docs/access/service-accounts).

☝️ **Tip!** Check out the full
[`cml runner` command reference](/doc/ref/runner).
<admon type="tip">

Check out the full [`cml runner` command reference](/doc/ref/runner).

</admon>

## Environment Variables

Expand All @@ -222,10 +229,14 @@ workflow above, this token is stored as `PERSONAL_ACCESS_TOKEN`.

</admon>

🛈 If using the `--cloud` option, you will also need to provide access
credentials for your cloud compute resources as secrets. In the above example,
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (with privileges to create &
destroy EC2 instances) are required.
<admon type="info">

If using the `--cloud` option, you will also need to provide
[access credentials for your cloud compute resources](#cloud-compute-resource-credentials)
as secrets (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` in the above
example).

</admon>

### Personal Access Token

Expand Down Expand Up @@ -400,9 +411,6 @@ See the
[AWS credentials docs](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-creds)
for obtaining these keys.

☝️ **Note** The same credentials can also be used for
[configuring cloud storage](/doc/cml-with-dvc#cloud-storage-provider-credentials).

</tab>
<tab title="Azure">

Expand Down Expand Up @@ -433,6 +441,13 @@ provisioned through environment variables instead of files.
</tab>
</toggle>

<admon type="tip">

The same credentials can also be used for
[configuring cloud storage](/doc/cml-with-dvc#cloud-storage-provider-credentials).

</admon>

### On-premise (Local) Runners

The `cml runner` command can also be used to manually set up a local machine,
Expand All @@ -449,13 +464,17 @@ $ cml runner launch \

The machine will listen for jobs from your repository and execute them locally.

⚠️ **Warning:** anyone with access to your repository (everybody for public
ones) may be able to execute arbitrary code on your machine. Refer to the
corresponding
<admon type="warn">

Anyone with push/PR access to your repository (potentially everybody for public
repositories) may be able to execute arbitrary code on your machine. Refer to
the corresponding
[GitHub](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#hardening-for-self-hosted-runners)
and [GitLab](https://docs.gitlab.com/runner/security) documentation for
additional guidance.

</admon>

## Debugging

If `cml runner` fails with a Terraform error message, setting the environment
Expand Down Expand Up @@ -491,8 +510,12 @@ $ sudo journalctl -n all -u cml.service --no-pager > cml.log
$ sudo dmesg --ctime > system.log
```

☝️ **Note** Please give your `cml.log` a visual scan, entries like IP addresses
and Git repository names may be present and considered sensitive in some cases.
<admon type="warn">

Please check your `cml.log` and censor potentially sensitive entries (e.g. IP
addresses and Git repository names).

</admon>

You can then copy those logs to your local machine with:

Expand Down