Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New User Management documentation #2541

Merged
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c639e42
Wip User Management
frascuchon Mar 14, 2023
bff752d
Apply suggestions from code review
frascuchon Mar 15, 2023
d67314f
Apply suggestions from code review
frascuchon Mar 15, 2023
7690bb5
Apply suggestions from code review
frascuchon Mar 16, 2023
f091331
Apply suggestions from code review
frascuchon Mar 16, 2023
d88ae3e
Adding command outputs and python code snippets
frascuchon Mar 16, 2023
8a70ba1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 16, 2023
987bc5c
Merge branch 'feat/add-basic-database-support' into docs-about-new-us…
frascuchon Mar 16, 2023
767236d
User Management actions review
frascuchon Mar 16, 2023
b83fe63
Pulish bash commands
frascuchon Mar 16, 2023
3bb459e
Apply suggestions from code review
frascuchon Mar 16, 2023
38b68d7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 16, 2023
4c2357b
Update docs/_source/getting_started/installation/configurations/user_…
frascuchon Mar 16, 2023
70756c6
Update docs/_source/getting_started/installation/configurations/user_…
frascuchon Mar 16, 2023
a85ff4f
Improve section about migrating users with Docker Compose
jfcalvo Mar 16, 2023
5648037
Merge branch 'feat/add-basic-database-support' into docs-about-new-us…
jfcalvo Mar 17, 2023
b561fb6
Use docker-compose instead of docker compose for command lines and fi…
jfcalvo Mar 17, 2023
20f5ee2
Update some details related with Quickstart Dockerfile
jfcalvo Mar 17, 2023
5f4be25
Update docs/_source/getting_started/installation/deployments/docker-q…
davidberenstein1957 Mar 21, 2023
fcfca10
Update docs/_source/getting_started/installation/deployments/huggingf…
davidberenstein1957 Mar 21, 2023
0b5ae43
Update docs/_source/getting_started/installation/deployments/huggingf…
davidberenstein1957 Mar 21, 2023
e537b00
Update docs/_source/getting_started/installation/deployments/docker-q…
jfcalvo Mar 22, 2023
3326fa6
Update docs/_source/getting_started/installation/deployments/docker-q…
jfcalvo Mar 22, 2023
9e5421e
Apply suggestions from code review
jfcalvo Mar 22, 2023
87c7514
Update quickstart.README.md
jfcalvo Mar 22, 2023
9a94ce1
Apply suggestions from code review
jfcalvo Mar 22, 2023
0aafa61
Fix some code syntax problems
jfcalvo Mar 22, 2023
e01ca54
Fix wrong username
jfcalvo Mar 22, 2023
6d8b473
Fix some small errors
jfcalvo Mar 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ Within Elastic, it is possible to create snapshots of a running cluster. We high

### Mount back-up volume

When deploying Elastic, we need to define a `path.repo` via setting this as an environment variable in your `docker-compose.yml` or by setting this in your `elasticsearch.yml`, and passing this as config. Additionally, we need to pass the same `path.repo` to a mounted volume. By default, we set this `elasticdata:/usr/share/elasticsearch/backups` because the `elasticsearch` user needs to have full permisions to act on the repo. Hence, setting the volume to something different might require some additional permission configurations. Note that the `minimum_master_nodes` need to be explicitly set when bound on a public IP.
When deploying Elastic, we need to define a `path.repo` via setting this as an environment variable in your `docker-compose.yaml` or by setting this in your `elasticsearch.yml`, and passing this as config. Additionally, we need to pass the same `path.repo` to a mounted volume. By default, we set this `elasticdata:/usr/share/elasticsearch/backups` because the `elasticsearch` user needs to have full permisions to act on the repo. Hence, setting the volume to something different might require some additional permission configurations. Note that the `minimum_master_nodes` need to be explicitly set when bound on a public IP.

#### `docker-compose.yml`
#### `docker-compose.yaml`

```yaml
elasticsearch:
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,10 @@ docker context use default

### 4. Deploy the Argilla Server

To deploy the Argilla Server, you can use the Docker Compose file provided in the [Argilla repository](https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml) and the `docker compose up` command:
To deploy the Argilla Server, you can use the Docker Compose file provided in the [Argilla repository](https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml) and the `docker-compose up` command:

```bash
wget -O docker-compose.yml https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml && docker-compose up -d
wget -O docker-compose.yaml https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml && docker-compose up -d
```

This guide is adapted from this [blog post by Ben Burtenshaw](https://medium.com/@ben.burtenshaw/zero-to-demo-on-azure-with-docker-compose-and-container-instances-4e83b78003b). There's also an official [tutorial](https://learn.microsoft.com/en-us/azure/container-instances/tutorial-docker-compose) on Microsoft Learn.
Expand All @@ -122,4 +122,3 @@ This guide is adapted from this [blog post by Ben Burtenshaw](https://medium.com
Coming soon!

> 🚒 **If you'd like support with this and/or want to contribute this gude, join the [Slack Community](https://join.slack.com/t/rubrixworkspace/shared_invite/zt-whigkyjn-a3IUJLD7gDbTZ0rKlvcJ5g)**

Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Docker Quickstart

Argilla is composed of a Python Server with Elasticsearch as the database layer, and a Python Client to create and
manage datasets.

Expand All @@ -16,23 +17,24 @@ Apple Silicon M1/M2 users might have issues with this deployment. To resolve thi

</div>

This will run the latest quickstart docker image with 2 users `argilla` and `team`. The password for these users is
`1234` . You can also configure these [environment variables](#environment-variables) as per you needs.
This will run the latest quickstart docker image with 2 users `admin` and `argilla`. The password for these users is `12345678`. You can also configure these [environment variables](#environment-variables) as per your needs.

## Environment Variables

- `ARGILLA_API_KEY`: Argilla provides a Python library to interact with the app (read, write, and update data, log model
predictions, etc.). If you don't set this variable, the library and your app will use the default API key. If you want
to secure your app for reading and writing data, we recommend you to set up this variable. The API key you choose
can be any string of your choice and you can check an online generator if you like.
- `ARGILLA_PASSWORD`: This sets a custom password for login into the app with the `argilla` username. The default
password is `1234`. By setting up a custom password you can use your own password to login into the app.
- `TEAM_API_KEY`: This sets the root user's API key. The API key you choose can be any string of your choice and you can
check an online generator if you like.
- `TEAM_PASSWORD`: This sets a custom password for login into the app with the `argilla` username. The default password
is `1234`. By setting up a custom password you can use your own password to login into the app.
- `ADMIN_USERNAME`: The admin username to log in Argilla. The default admin username is `admin`. By setting up
a custom username you can use your own username to log in to the app.
- `ADMIN_API_KEY`: Argilla provides a Python library to interact with the app (read, write, and update data, log model
predictions, etc.). If you don't set this variable, the library and your app will use the default API key
i.e. `admin.apikey`. If you want to secure your app for reading and writing data, we recommend you to set up this
variable. The API key you choose can be any string of your choice and you can check an online generator if you like.
- `ADMIN_PASSWORD`: This sets a custom password to log in to the app with the `argilla` username. The default
password is `12345678`. By setting up a custom password you can use your own password to login into the app.
- `ANNOTATOR_USERNAME`: The annotator username to login in Argilla. The default annotator username is `argilla`. By setting up
a custom username you can use your own username to login into the app.
- `ANNOTATOR_PASSWORD`: This sets a custom password for login into the app with the `argilla` username. The default password
is `12345678`. By setting up a custom password you can use your own password to login into the app.
- `LOAD_DATASETS`: This variables will allow you to load sample datasets. The default value will be `full`. The
supported values for this variable is as follows:
1. `single`: Load single datasets for TextClassification task.
2. `full`: Load all the sample datasets for NLP tasks (TokenClassification, TextClassification, Text2Text)
3. `none`: No datasets being loaded.
3. `none`: No datasets being loaded.
jfcalvo marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ mkdir argilla && cd argilla
and launch the docker-contained web app with the following command:

```bash
wget -O docker-compose.yml https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml && docker-compose up -d
wget -O docker-compose.yaml https://raw.githubusercontent.com/argilla-io/argilla/main/docker-compose.yaml && docker-compose up -d
```
:::{warning}
Latest versions of docker should be executed without the dash '-', e.g:

```bash
docker compose up -d
docker-compose up -d
```
:::

Expand All @@ -40,7 +40,7 @@ you will lose all your datasets in Argilla!
:::{tip}
Instead, execute
```bash
docker compose stop
docker-compose stop
```
:::

Expand All @@ -53,7 +53,7 @@ By default, telemetry is enabled. This helps us to improve our product. For more
To avoid losing all the data when the docker-compose/server goes down, you can add some persistence by mounting a
volume in the docker compose.

To this end, **modify the elasticsearch service and create a new volume** in the docker-compse.yml file:
To this end, **modify the elasticsearch service and create a new volume** in the `docker-compose.yaml` file:

```yaml
services:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ If you want to run the web app of the master branch **without** docker, we refer
```bash
# get the docker-compose yaml file
mkdir argilla && cd argilla
wget -O docker-compose.yml https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.yaml
wget -O docker-compose.yaml https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.yaml
# use the master image of the argilla container instead of the latest
sed -i 's/argilla:latest/argilla:master/' docker-compose.yml
sed -i 's/argilla:latest/argilla:master/' docker-compose.yaml
# start all services
docker-compose up
```
Expand All @@ -31,4 +31,4 @@ If you want to use vanilla docker (and have your own Elasticsearch instance runn

```bash
docker run -p 6900:6900 -e "ELASTICSEARCH=<your-elasticsearch-endpoint>" --network argilla-net --name argilla argilla/argilla-server:develop
```
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dvsrepo @davidberenstein1957 please can you take a look and review this file?

Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ You'll see the `Building` status and once it becomes `Running` your space is rea


:::{tip}
You'll see the login screen where you need to use either argilla or team with the default passwords (1234). Remember you can change the passwords using secret variables. If you get a 500 error when introducing the credentials, make sure you have correctly introduce the password.
You'll see the login screen where you need to use either admin or argilla with the default passwords (12345678). Remember you can change the passwords using secret environment variables. If you get a 500 error when introducing the credentials, make sure you have introduced the correct password.
:::

<div class="flex justify-center">
Expand Down Expand Up @@ -133,23 +133,28 @@ The Space template provides a way to set up different **optional settings** focu

To set up these secrets, you can go to the Settings tab on your created Space. Make sure to save these values somewhere for later use.

The template space has two users: `team` and `argilla`. The username `team` corresponds to the root user, who can upload datasets and access any workspace within your Argilla Space. The username `argilla` is a normal user with access to the `team` workspace and its own workspace called `argilla`.
The template space has two users: `admin` and `argilla`. The username `admin` corresponds to the root user, who can upload datasets and access any workspace within your Argilla Space. The username `argilla` is a normal user with access to the `argilla` workspace.

Currently, these user names can't be configured. The passwords and API keys to upload, read, update, and delete datasets can be configured using the following secrets:

- `ARGILLA_API_KEY`: Argilla provides a Python library to interact with the app (read, write, and update data, log model predictions, etc.). If you don't set this variable, the library and your app will use the default API key. If you want to secure your Space for reading and writing data, we recommend you to set up this variable. The API key you choose can be any string of your choice and you can check an online generator if you like.

- `ARGILLA_PASSWORD`: This sets a custom password for login into the app with the `argilla` username. The default password is `1234`. By setting up a custom password you can use your own password to login into the app.

- `TEAM_API_KEY`: This sets the root user's API key. The API key you choose can be any string of your choice and you can check an online generator if you like.

- `TEAM_PASSWORD`: This sets a custom password for login into the app with the `team` username. The default password is `1234`. By setting up a custom password you can use your own password to login into the app.
- `ADMIN_USERNAME`: The admin username to log in Argilla. The default admin username is `admin`. By setting up
a custom username you can use your own username to log in to the app.
- `ADMIN_API_KEY`: Argilla provides a Python library to interact with the app (read, write, and update data, log model
predictions, etc.). If you don't set this variable, the library and your app will use the default API key
i.e. `admin.apikey`. If you want to secure your app for reading and writing data, we recommend you to set up this
variable. The API key can be any string of your choice. You can check an online generator if you like.
- `ADMIN_PASSWORD`: This sets a custom password to log in to the app with the `argilla` username. The default
password is `12345678`. By setting up a custom password you can use your own password to log in to the app.
- `ANNOTATOR_USERNAME`: The annotator username to log in to Argilla. The default annotator username is `argilla`. By setting up
a custom username you can use your own username to log in to the app.
- `ANNOTATOR_PASSWORD`: This sets a custom password to log in to the app with the `argilla` username. The default password
is `12345678`. By setting up a custom password you can use your own password to log in to the app.

The combination of these secret variables gives you the following setup options:

1. *I want to avoid that anyone without the API keys can add, delete, or update datasets using the Python client*: You need to setup `ARGILLA_API_KEY` and `TEAM_API_KEY`.
2. *Additionally, I want to avoid that the `argilla` username can delete datasets from the UI*: You need to setup `TEAM_PASSWORD` and use `TEAM_API_KEY` with the Python Client. This option might be interesting if you want to control dataset management but want anyone to browse your datasets using the `argilla` user.
3. *Additionally, I want to avoid that anyone without password can browse my datasets with the `argilla` user*: You need to setup `ARGILLA_PASSWORD`. In this case, you can use `ARGILLA_API_KEY` and/or `TEAM_API_KEY` with the Python Client depending on your needs for dataset deletion rights.
1. *I want to avoid that anyone without the API keys adds, deletes, or updates datasets using the Python client*: You need to setup `ADMIN_PASSWORD` and `ADMIN_API_KEY`.
2. *Additionally, I want to avoid that the `argilla` username deletes datasets from the UI*: You need to setup `ANNOTATOR_PASSWORD` and use the `argilla` generated API key with the Python Client (check your Space logs). This option might be interesting if you want to control dataset management but want anyone to browse your datasets using the `argilla` user.
3. *Additionally, I want to avoid that anyone without password browses my datasets with the `argilla` user*: You need to setup `ANNOTATOR_PASSWORD`. In this case, you can use the `argilla` generated API key and/or `ADMIN_API_KEY` values with the Python Client depending on your needs for dataset deletion rights.

Additionally, the `LOAD_DATASETS` will let you configure the sample datasets that will be pre-loaded. The default value is `single` and the supported values for this variable are:
1. `single`: Load single datasets for TextClassification task.
Expand Down
4 changes: 2 additions & 2 deletions docs/_source/getting_started/installation/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Note
Check the [setup and installation section](./installation/installation.md) for further options and configurations regarding Elasticsearch and Opensearch.
</div>

Now you can launch Elasticsearch or Opensearch using `docker compose` as follows:
Now you can launch Elasticsearch or Opensearch using `docker-compose` as follows:

### Opensearch
You can launch Opensearch with Docker as follows:
Expand Down Expand Up @@ -108,4 +108,4 @@ If you now go to your Argilla app at [http://localhost:6900/](http://localhost:6

deployments/deployments.md
configurations/configurations.md
```
```
4 changes: 2 additions & 2 deletions docs/_source/getting_started/quickstart.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"\n",
"[![deploy on spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/deploy-to-spaces-lg.svg)](https://huggingface.co/new-space?template=argilla/argilla-template-space)\n",
"\n",
"If everything goes well, you'll see your online Argilla UI login page. You can login with username `argilla` and password `1234`. You can find the direct URL by clicking on the Embed space button. You'll use this URL for sending data to your Argilla instance. \n",
"If everything goes well, you'll see your online Argilla UI login page. You can log in with username `admin` and password `12345678`. You can find the direct URL by clicking on the Embed space button. You'll use this URL for sending data to your Argilla instance. \n",
"\n",
"<div class=\"flex justify-center\">\n",
"<img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/spaces-argilla-duplicate-space.png\"/>\n",
Expand Down Expand Up @@ -73,7 +73,7 @@
"\n",
"</div>\n",
"\n",
"If everything goes well, you can go to [http://localhost:6900](http://localhost:6900) and login with username `argilla` and password `1234`. You'll see several example datasets already pre-loaded. That means you are ready to start playing with Argilla! \n",
"If everything goes well, you can go to [http://localhost:6900](http://localhost:6900) and log in with username `admin` and password `12345678`. You'll see several example datasets already pre-loaded. That means you are ready to start playing with Argilla! \n",
"\n",
"As the next step, we recommend [finding a tutorial](../tutorials/tutorials.md), downloading the notebook using the link `View source` at top-right corner of each tutorial page, and run it using VS Code, Jupyter Lab, or any other Jupyter IDE.\n",
"\n",
Expand Down
4 changes: 2 additions & 2 deletions docs/_source/guides/label_records_with_semanticsearch.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,15 @@
"</div>\n",
"\n",
"### Opensearch backend\n",
"If you don't have a another instance of Elasticsearch or Opensearch running, or don't want to keep previous Argilla datasets, you can launch a clean instance of Opensearch by downloading the [docker-compose.yml file](https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.opensearch.yaml) and running:\n",
"If you don't have another instance of Elasticsearch or Opensearch running, or don't want to keep previous Argilla datasets, you can launch a clean instance of Opensearch by downloading the [docker-compose.yaml file](https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.opensearch.yaml) and running:\n",
"\n",
"```bash\n",
"docker-compose -f docker-compose.opensearch.yaml up\n",
"```\n",
"Once the service is up you can launch the Argilla Server with `python -m argilla`.\n",
"\n",
"### Elasticsearch backend\n",
"If you don't have a another instance of Elasticsearch or Opensearch running, or don't want to keep previous Argilla datasets, you can launch a clean instance of Elasticsearch by downloading the [docker-compose.yml](https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.elasticsearch.yaml) and running:\n",
"If you don't have another instance of Elasticsearch or Opensearch running, or don't want to keep previous Argilla datasets, you can launch a clean instance of Elasticsearch by downloading the [docker-compose.yaml](https://raw.githubusercontent.com/argilla-io/argilla/develop/docker-compose.elasticsearch.yaml) and running:\n",
"\n",
"```bash\n",
"docker-compose -f docker-compose.elasticsearch.yaml up\n",
Expand Down
3 changes: 1 addition & 2 deletions quickstart.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM docker.elastic.co/elasticsearch/elasticsearch:8.5.3

# Environment variable
# Environment variables
ENV ADMIN_USERNAME=admin
ENV ADMIN_PASSWORD=12345678
ENV ADMIN_API_KEY=admin.apikey
Expand All @@ -23,7 +23,6 @@ COPY scripts/start_quickstart_argilla.sh /
COPY scripts/load_data.py /
COPY dist/*.whl /packages/


# Install packages
RUN apt update && \
apt -y install curl python3.9 python3.9-dev python3.9-distutils gcc gnupg apache2-utils sudo openssl systemctl && \
Expand Down
2 changes: 1 addition & 1 deletion quickstart.README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ To get started you just need to run the docker image with following command:
```

This will run the latest quickstart docker image with 2 users `argilla` and `team`. The password for these users is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember correctly, these users were admin and argilla in a different file. Are these different here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're totally right. These should be admin and argilla. Thanks!

jfcalvo marked this conversation as resolved.
Show resolved Hide resolved
`1234` . You can also configure these [environment variables](#environment-variables) as per you needs.
`12345678`. You can also configure these [environment variables](#environment-variables) as per you needs.

### Environment Variables

Expand Down