Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add demo upgrade issue template #120

Merged
merged 6 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/ISSUE_TEMPLATE/pre-release-chart-updates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
name: Pre-Release Demo Chart Updates
about: This template can be used to track the Helm chart updates for all demos.
title: "chore(tracking): Update Helm charts for XX.(X)X"
labels: []
assignees: ''
---

## Pre-Release Demo Chart Updates

There are a bunch of demos which directly use Helm charts to install some required parts of the
demo / stack. These charts need updating **before** we start testing the stable to nightly upgrade.

Most of the charts are located in the `stacks/_templated` folder. Additional folders are:
`stacks/observability` and `stacks/signal-processing`. It is recommended to search for `releaseName`
to find all referenced charts.

### Update Instructions

These instructions help to update the charts:

```shell
# List all available repos
helm repo list
```

Make sure, you have the following repos added:

```plain
NAME URL
bitnami https://charts.bitnami.com/bitnami
stackable-dev https://repo.stackable.tech/repository/helm-dev/
stackable-stable https://repo.stackable.tech/repository/helm-stable/
timescale https://charts.timescale.com/
prometheus-community https://prometheus-community.github.io/helm-charts
vector https://helm.vector.dev
grafana https://grafana.github.io/helm-charts
jaeger https://jaegertracing.github.io/helm-charts
opentelemetry https://open-telemetry.github.io/opentelemetry-helm-charts
jupyterhub https://jupyterhub.github.io/helm-chart/
opensearch https://opensearch-project.github.io/helm-charts
```

Techassi marked this conversation as resolved.
Show resolved Hide resolved
Next, search for the latest version of the desired chart, eg `grafana/grafana`:

```shell
# Display the latest versions of a chart
$ helm search repo grafana/grafana --versions | head -5
NAME CHART VERSION APP VERSION
grafana/grafana 8.6.0 11.3.0
grafana/grafana 8.5.12 11.3.0
grafana/grafana 8.5.11 11.3.0
grafana/grafana 8.5.10 11.3.0
```

Use the latest version displayed to replace the current one in the YAML files. Take a look at
previous PRs for additional guidance, eg. <https://github.com/stackabletech/demos/pull/119>.
Techassi marked this conversation as resolved.
Show resolved Hide resolved
123 changes: 123 additions & 0 deletions .github/ISSUE_TEMPLATE/pre-release-nightly-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
name: Pre-Release Demo Upgrade Testing from Stable to Nightly
about: |
This template can be used to track the upgrade testing of demos from stable to nightly leading
up to the next Stackable release
title: "chore(tracking): Test demos on nightly versions for XX.(X)X"
labels: ['epic']
assignees: ''
---

<!--
Make sure to update the link in '.github/ISSUE_TEMPLATE/release.md' when
you change the filename.
-->

<!--
DO NOT REMOVE THIS COMMENT. It is intended for people who might copy/paste from the previous release issue.
This was created by an issue template: https://github.com/stackabletech/issues/issues/new/choose.
-->

## Pre-Release Demo Testing on Nightly

Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>

This is testing:

1. The demos documented in nightly (with the updated product versions) still work.
2. That the operators can be upgraded from the current release to the nightly release and do not
negatively impact the products.

> [!NOTE]
> Record any issues or anomalies during the process in a comment on this issue.
> Eg:
>
> ```plain
> :green_circle: **airflow-scheduled-job**
>
> The CRD had been updated and I needed to change the following in the manifest:
> ...
> ```

Replace the items in the task lists below with the applicable Pull Requests (if any).

<!--
The following list was generated by:

# go to the demos repository, then run:
yq '.demos | keys' demos/demos-v2.yaml \
| sed -e 's/- //g' \
| sort \
| xargs -I {} echo "- [ ] [{}](https://docs.stackable.tech/home/nightly/demos/{})"
-->

```[tasklist]
### Testing Demos on Nightly
- [ ] [airflow-scheduled-job](https://docs.stackable.tech/home/nightly/demos/airflow-scheduled-job)
- [ ] [data-lakehouse-iceberg-trino-spark](https://docs.stackable.tech/home/nightly/demos/data-lakehouse-iceberg-trino-spark)
- [ ] [end-to-end-security](https://docs.stackable.tech/home/nightly/demos/end-to-end-security)
- [ ] [hbase-hdfs-load-cycling-data](https://docs.stackable.tech/home/nightly/demos/hbase-hdfs-load-cycling-data)
- [ ] [jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data](https://docs.stackable.tech/home/nightly/demos/jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data)
- [ ] [logging](https://docs.stackable.tech/home/nightly/demos/logging)
- [ ] [nifi-kafka-druid-earthquake-data](https://docs.stackable.tech/home/nightly/demos/nifi-kafka-druid-earthquake-data)
- [ ] [nifi-kafka-druid-water-level-data](https://docs.stackable.tech/home/nightly/demos/nifi-kafka-druid-water-level-data)
- [ ] [signal-processing](https://docs.stackable.tech/home/nightly/demos/signal-processing)
- [ ] [spark-k8s-anomaly-detection-taxi-data](https://docs.stackable.tech/home/nightly/demos/spark-k8s-anomaly-detection-taxi-data)
- [ ] [trino-iceberg](https://docs.stackable.tech/home/nightly/demos/trino-iceberg)
- [ ] [trino-taxi-data](https://docs.stackable.tech/home/nightly/demos/trino-taxi-data)
```

### Testing Instructions

These instructions are for deploying the nightly demo, as well as upgrading the operators and CRDS.

<!--
Make sure to update the version mentioned below when creating the issue.
-->

```shell
# Install demo (stable operators) for the previous release (24.7)
# For now, we have to deploy from the release branch, otherwise we get new changes.
# Stackablectl doesn't yet support deploying a demo from a branch
git checkout release-24.7
git pull
stackablectl --stack-file=stacks/stacks-v2.yaml --demo-file=demos/demos-v2.yaml demo install <DEMO_NAME>

# --- IMPORTANT ---
# Run through the nightly demo instructions (refer to the tasklist below).

# Get a list of installed operators
stackablectl operator installed --output=plain

# --- OPTIONAL ---
# Sometimes it is necessary to upgrade Helm charts. Look for other Helm Charts
# which might need updating.

# First, see which charts are installed. You can ignore the stackable-operator
# charts, or anything that might have been installed outside of this demo.
helm list

# Next, add the applicable Helm Chart repositories. For example:
helm repo add minio https://charts.min.io/
helm repo add bitnami https://charts.bitnami.com/bitnami

# Finally, upgrade the Charts to what is defined in `main`.
# For example:
helm upgrade minio minio/minio --version x.x.x
helm upgrade postgresql-hive bitnami/postgresql --version x.x.x
# --- OPTIONAL END ---

# Uninstall operators
stackablectl release uninstall 24.7

# Update CRDs to nightly version (on main)
# Repeat this for every operator used by the demo (use the list from the earlier step before deleting the operators)
kubectl replace -f https://raw.githubusercontent.com/stackabletech/commons-operator/main/deploy/helm/commons-operator/crds/crds.yaml
kubectl replace -f https://raw.githubusercontent.com/stackabletech/...-operator/main/deploy/helm/...-operator/crds/crds.yaml

# Install nightly version of operators (use the list from the earlier step before deleting the operators)
stackablectl operator install commons ...

# Optionally update the product versions in the CRDs (to the latest non-experimental version for the new release), e.g.:
kubectl patch hbaseclusters/hbase --type='json' -p='[{"op": "replace", "path": "/spec/image/productVersion", "value":"x.x.x"}]' # changed
```