Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Remove prefect-agent chart #378

Merged
merged 2 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .github/linters/agent-ct.yaml

This file was deleted.

7 changes: 0 additions & 7 deletions .github/updatecli/manifest-major.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@ sources:
sourceid: postgresql
conditions: {}
targets:
agent_common:
name: bump chart dependencies
kind: yaml
spec:
file: charts/prefect-agent/Chart.yaml
key: dependencies[0].version
sourceid: common
prometheus_prefect_exporter_common:
name: bump chart dependencies
kind: yaml
Expand Down
7 changes: 0 additions & 7 deletions .github/updatecli/manifest-minor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@ sources:
sourceid: postgresql
conditions: {}
targets:
agent_common:
name: bump chart dependencies
kind: yaml
spec:
file: charts/prefect-agent/Chart.yaml
key: dependencies[0].version
sourceid: common
prometheus_prefect_exporter_common:
name: bump chart dependencies
kind: yaml
Expand Down
52 changes: 0 additions & 52 deletions .github/workflows/agent-lint-and-test.yaml

This file was deleted.

23 changes: 1 addition & 22 deletions .github/workflows/helm-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,26 +59,6 @@ jobs:
run: |
helm repo add bitnami https://charts.bitnami.com/bitnami

- name: Package Agent helm chart
run: |
mkdir -p /tmp/chart
cd charts
# Update the prefect version tag in values.yaml
sed -i "s/prefectTag:.*$/prefectTag: $PREFECT_VERSION-python3.10/g" prefect-agent/values.yaml
helm package prefect-agent \
--destination /tmp/chart \
--dependency-update \
--version $RELEASE_VERSION \
--app-version $PREFECT_VERSION \
--sign --key '[email protected]' \
--keyring $SIGN_KEYRING \
--passphrase-file $SIGN_PASSPHRASE_FILE
env:
RELEASE_VERSION: ${{ steps.get_version.outputs.RELEASE_VERSION }}
PREFECT_VERSION: ${{ steps.get_version.outputs.PREFECT_VERSION }}
SIGN_KEYRING: ${{ env.SIGN_KEYRING }}
SIGN_PASSPHRASE_FILE: ${{ env.SIGN_PASSPHRASE_FILE }}

- name: Package Worker helm chart
run: |
mkdir -p /tmp/chart
Expand Down Expand Up @@ -129,10 +109,9 @@ jobs:
run: |
cp /tmp/chart/artifacthub-repo.yml .
cp /tmp/chart/index.yaml .
cp /tmp/chart/prefect-agent-$RELEASE_VERSION.* ./charts
cp /tmp/chart/prefect-server-$RELEASE_VERSION.* ./charts
cp /tmp/chart/prefect-worker-$RELEASE_VERSION.* ./charts
git add ./artifacthub-repo.yml ./index.yaml ./charts/prefect-agent-$RELEASE_VERSION.* ./charts/prefect-server-$RELEASE_VERSION.* ./charts/prefect-worker-$RELEASE_VERSION.*
git add ./artifacthub-repo.yml ./index.yaml ./charts/prefect-server-$RELEASE_VERSION.* ./charts/prefect-worker-$RELEASE_VERSION.*
git commit -m "Release $RELEASE_VERSION"
git push origin gh-pages
env:
Expand Down
37 changes: 5 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,8 @@ This repository contains the official Prefect Helm charts for installing and con

[Workers](https://docs.prefect.io/latest/concepts/work-pools/#worker-overview) are lightweight polling services that retrieve scheduled runs from a work pool and execute them.

Workers are similar to agents, but offer greater control over infrastructure configuration and the ability to route work to specific types of execution environments.

Workers each have a type corresponding to the execution environment to which they will submit flow runs. Workers are only able to join work pools that match their type. As a result, when deployments are assigned to a work pool, you know in which execution environment scheduled flow runs for that deployment will run.

## [Prefect agent](charts/prefect-agent/)

### Note: Workers are recommended

Agents are part of the block-based deployment model. [Work Pools and Workers](https://docs.prefect.io/latest/concepts/work-pools/) simplify the specification of a flow's infrastructure and runtime environment. If you have existing agents, you can [upgrade from agents to workers](https://docs.prefect.io/latest/guides/upgrade-guide-agents-to-workers/) to significantly enhance the experience of deploying flows.

[Agent](https://docs.prefect.io/latest/concepts/agents/) processes are lightweight polling services that get scheduled work from a work pool and deploy the corresponding flow runs.

Agents poll for work every 15 seconds by default. This interval is configurable in your profile settings with the `PREFECT_AGENT_QUERY_INTERVAL` setting.

It is possible for multiple agent processes to be started for a single work pool. Each agent process sends a unique ID to the server to help disambiguate themselves and let users know how many agents are active.

## [Prefect server](charts/prefect-server/)

[Prefect server](https://docs.prefect.io/latest/guides/host/) is a self-hosted open source backend that makes it easy to observe and orchestrate your Prefect flows. It is an alternative to using the hosted [Prefect Cloud](https://docs.prefect.io/latest/cloud/) platform. Prefect Cloud provides additional features including automations and user management.
Expand All @@ -30,6 +16,10 @@ It is possible for multiple agent processes to be started for a single work pool

The Prometheus Prefect Exporter is a tool to pull relevant Prefect metrics from a hosted Prefect Server instance

## Prefect agent

Prefect Agents have been deprecated. `prefect-helm` version `2024.8.30163822` and earlier will contain the `prefect-agent` chart.

## Usage

[Helm](https://helm.sh) must be installed to use the charts.
Expand Down Expand Up @@ -129,7 +119,7 @@ See comments in `values.yaml`.

### Security context

By default, the worker (or agent), and server run as an unprivileged user with a read-only root filesystem. You can customize the security context settings for both the worker and server in the `values.yaml` file for your use case.
By default, the worker, and server run as an unprivileged user with a read-only root filesystem. You can customize the security context settings for both the worker and server in the `values.yaml` file for your use case.

If you need to install system packages or configure other settings at runtime, you can configure a writable filesystem and run as root by configuring the pod and container security context accordingly:

Expand Down Expand Up @@ -162,23 +152,6 @@ containerSecurityContext:

The other default settings, such as a read-only root filesystem, are suitable for an OpenShift environment.

## Additional permissions for Prefect agents

### Dask

If you are running flows on your worker’s pod (i.e. with Process infrastructure), and using the Dask task runner to create Dask Kubernetes clusters, you will need to grant the following permissions within `values.yaml`.

```yaml
role:
extraPermissions:
- apiGroups: [""]
resources: ["pods", "services"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["policy"]
resources: ["poddisruptionbudgets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
```

## Version support policy

Prefect follows the [upstream Kubernetes support policy](https://kubernetes.io/releases/version-skew-policy/), meaning that we test against the three most recent minor version releases of Kubernetes. The charts may be compatible with older releases of Kubernetes, however, we do not test against those versions and may choose to reject issues or patches to add support.
Expand Down
23 changes: 0 additions & 23 deletions charts/prefect-agent/.helmignore

This file was deleted.

26 changes: 0 additions & 26 deletions charts/prefect-agent/Chart.yaml

This file was deleted.

84 changes: 0 additions & 84 deletions charts/prefect-agent/README.md

This file was deleted.

1 change: 0 additions & 1 deletion charts/prefect-agent/templates/NOTES.txt

This file was deleted.

35 changes: 0 additions & 35 deletions charts/prefect-agent/templates/_helpers.tpl

This file was deleted.

Loading
Loading