diff --git a/docs/testing.md b/docs/testing.md index 47d67dea5..02373617f 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,200 +1,171 @@ --- title: Testing -types: docs +type: docs menu: docs --- -# Testing +# KUDO Test Harness -KUDO uses a declarative integration testing harness for testing itself and Operators built on it. Test cases are written as plain Kubernetes resources and can be run against any Kubernetes cluster - allowing testing of Operators, KUDO, and any other Kubernetes resources or controllers. +KUDO includes a declarative integration testing harness for testing Operators, KUDO, and any other Kubernetes applications or controllers. Test cases are written as plain Kubernetes resources and can be run against a mocked control plane, locally in KIND, or any other Kubernetes cluster. -## Table of Contents +Whether you are developing an application, controller, operator, or deploying Kubernetes clusters the KUDO test harness helps you easily write portable end-to-end, integration, and conformance tests for Kubernetes without needing to write any code. -* [Test harness usage](#test-harness-usage) - * [Run the Operator tests](#run-the-operator-tests) -* [Writing test cases](#writing-test-cases) - * [Test case directory structure](#test-case-directory-structure) - * [Test steps](#test-steps) - * [Deleting resources](#deleting-resources) - * [Test assertions](#test-assertions) - * [Listing objects](#listing-objects) - * [Advanced test assertions](#advanced-test-assertions) -* [Further Reading](#further-reading) +* [Installation](#installation) +* [Writing your first test](#writing-your-first-test) + * [Create a test case](#create-a-test-case) + * [Run the tests](#run-the-tests) + * [Write a second test step](#write-a-second-test-step) + * [Test suite configuration](#test-suite-configuration) -## Test harness usage +# Installation -The Operator test suite is written and run by Operator developers and CI to test that Operators work correctly. - -First, clone the [Operators repository](https://github.com/kudobuilder/operators): +The test harness CLI is included in the KUDO CLI, to install we can install the CLI using [krew](https://github.com/kubernetes-sigs/krew): ``` -git clone https://github.com/kudobuilder/operators.git -cd operators +krew install kudo ``` -Make sure that you have the latest version of kudoctl installed. - -#### Run the Operator tests - -To run the Operator test suite, run the following in the [Operators repository](https://github.com/kudobuilder/operators): +You can now invoke the kudo test CLI: ``` -kubectl kudo test +kubectl kudo test --help ``` -To run against a production Kubernetes cluster, run: +See the [KUDO installation guide](/docs/cli#install) for alternative installation methods. -``` -kubectl kudo test --start-kind=false -``` +# Writing your first test -Operator test suites are stored in the `tests` subdirectory of each [Operator](https://github.com/kudobuilder/operators/tree/master/repository), e.g.: +Now that the kudo CLI is installed, we can write a test. The KUDO test CLI organizes tests into suites: -``` -./repository/zookeeper/tests/ -./repository/mysql/tests/ -``` - -Every `Operator` and `OperatorVersion` is installed into the cluster by `make test` prior to running the test suite. - -## Writing test cases +* A "test step" defines a set of Kubernetes manifests to apply and a state to assert on (wait for or expect). +* A "test case" is a collection of test steps that are run serially - if any test step fails then the entire test case is considered failed. +* A "test suite" is comprised of many test cases that are run in parallel. +* The "test harness" is the tool that runs test suites (the KUDO CLI). -To write a test case: +## Create a test case -1. Create a directory for the test case. The directory should be created inside of the suite that the test case is a part of. For example, a Zookeeper Operator test case called `upgrade-test` would be created as `./repository/zookeeper/tests/upgrade-test/`. -2. Define the Kubernetes resources to apply and states to assert on in sequential test steps. - -### Test case directory structure - -Given the above Zookeeper test case called `upgrade-test`, an example test case directory structure with two test steps might look like: +First, let's create a directory for our test suite, let's call it `tests/e2e`: ``` -./repository/zookeeper/tests/upgrade-test/00-instance.yaml -./repository/zookeeper/tests/upgrade-test/00-configmap.yaml -./repository/zookeeper/tests/upgrade-test/00-assert.yaml -./repository/zookeeper/tests/upgrade-test/01-upgrade.yaml -./repository/zookeeper/tests/upgrade-test/01-assert.yaml +mkdir -p tests/e2e ``` -Each file in the test case directory should start with a number indicating the index of the step. All files with the same index are a part of the same test step and applied simultaneously. Steps are applied serially in numerical order. +Next, we'll create a directory for our test case, the test case will be called `example-test`: -A file called `$index-assert.yaml` (where `$index` is the index of the test case) must also exist indicating the state that the test harness should wait for before continuing to the next step. +``` +mkdir tests/e2e/example-test +``` -By default, the test harness will wait for up to 10 seconds for the assertion defined in the assertion file to be true before considering the test step failed. +Inside of `tests/e2e/example-test/` create our first test step, `00-install.yaml`, which will create a deployment called `example-deployment`: -In order for a test case to be successful, all steps must also complete successfully. +``` +apiVersion: apps/v1 +kind: Deployment +metadata: + name: example-deployment + labels: + app: nginx +spec: + replicas: 3 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:latest + ports: + - containerPort: 80 +``` -### Test steps +Note that in this example, the Deployment does not have a `namespace` set. The test harness will create a namespace for each test case and run all of the test steps inside of it. However, if a resource already has a namespace set (or is not a namespaced resource), then the harness will respect the namespace that is set. -The test step files can contain any number of Kubernetes resources that should be applied as a part of the test step. Typically, this would be a KUDO `Instance` (see: [KUDO concepts](https://kudo.dev/docs/concepts/) or a `Deployment` or other resources required for the test, such as `Secrets` or test `Pods`. +Each filename in the test case directory should start with an index (in this example `00`) that indicates which test step the file is a part of. Files that do not start with a step index are ignored and can be used for documentation or other test data. Test steps are run in order and each must be successful for the test case to be considered successful. -Continuing with the upgrade-test example, create `00-instance.yaml`: +Now that we have a test step, we need to create a test assert. The assert's filename should be the test step index followed by `-assert.yaml`. Create `tests/e2e/example-test/00-assert.yaml`: ``` -apiVersion: kudo.dev/v1alpha1 -kind: Instance +apiVersion: apps/v1 +kind: Deployment metadata: - name: zk -spec: - operatorVersion: - name: zookeeper-0.1.0 - namespace: default - type: OperatorVersions - name: "zk" - parameters: - cpus: "0.3" + name: example-deployment +status: + readyReplicas: 3 ``` -This test step will create a Zookeeper `Instance`. The namespace should not be specified in the resources as a namespace is created for the test case to run in. +This test step will be considered completed once the pod matches the state that we have defined. If the state is not reached by the time the assert's timeout has expired (30 seconds, by default), then the test step and case will be considered failed. -#### Deleting resources +## Run the tests -It is possible to delete existing resources at the beginning of a test step. Create a `TestStep` object in your step to configure it: +Let's run this test suite: ``` -apiVersion: kudo.dev/v1alpha1 -kind: TestStep -delete: -- name: my-pod - kind: Pod - apiVersion: v1 -- kind: Pod - apiVersion: v1 - labels: - app: nginx -- kind: Pod - apiVersion: v1 +kubectl kudo test --start-kind=true ./tests/e2e/ ``` -The test harness will delete for each resource referenced in the delete list and wait for them to disappear from the API. If the object fails to delete, the test step will fail. +Running this command will: -In the first delete example, the `Pod` called `my-pod` will be deleted. In the second, all `Pods` matching the `app=nginx` label will be deleted. In the third example, all pods in the namespace would be deleted. +* Start a [kind (kubernetes-in-docker) cluster](https://github.com/kubernetes-sigs/kind), if there is not already one running. +* Create a new namespace for the test case. +* Create the resources defined in `tests/e2e/example-test/00-install.yaml`. +* Wait for the state defined in `tests/e2e/example-test/00-assert.yaml` to be reached. +* Collect the kind cluster's logs. +* Tear down the kind cluster (or you can run kudo test with `--skip-cluster-delete` to keep the cluster around after the tests run). -#### Test assertions +## Write a second test step -Test assert files contain any number of Kubernetes resources that are expected to be created. Each resource must specify the `apiVersion`, `kind`, and `metadata`. The test harness watches each defined resource and waits for the state defined to match the state in Kubernetes. Once all resources have the correct state simultaneously, the test is considered successful. +Now that we have successfully written a test case, let's add another step to it. In this step, let's increase the number of replicas on the deployment we created in the first step from 3 to 4. -Continuing with the `upgrade-test` example, create `00-instance.yaml`: +Create `tests/e2e/example-test/01-scale.yaml`: ``` -apiVersion: kudo.dev/v1alpha1 -kind: Instance -metadata: - name: zk -status: - status: COMPLETE ---- apiVersion: apps/v1 -kind: StatefulSet +kind: Deployment metadata: - name: zk-zk + name: example-deployment spec: - template: - spec: - containers: - - name: kubernetes-zookeeper - resources: - requests: - memory: "1Gi" - cpu: "300m" -status: - readyReplicas: 3 + replicas: 4 ``` -This watches an `Instance` called `zk` to have its status set to `COMPLETE` and it expects a `StatefulSet` to also be created called `zk-zk` and it waits for all `Pods` in the `StatefulSet` to be ready. - -##### Listing objects - -If the object `name` is omitted from the object metadata, it is possible to list objects and verify that one of them matches the desired state. This can be useful, for example, to check the `Pods` created by a `Deployment`. +Now create an assert for it in `tests/e2e/example-test/01-assert.yaml`: ``` -apiVersion: v1 -kind: Pod +apiVersion: apps/v1 +kind: Deployment metadata: - labels: - app: nginx + name: example-deployment status: - phase: Running + readyReplicas: 4 ``` -This would verify that a pod with the `app=nginx` label is running. +Run the test suite again and the test will pass: -##### Advanced test assertions +``` +kubectl kudo test --start-kind=true ./tests/e2e/ +``` + +## Test suite configuration -The test harness recognizes special `TestAssert` objects defined in the assert file. If present, they override default settings of the test assert. +To add this test suite to your project, create a `kudo-test.yaml` file: ``` apiVersion: kudo.dev/v1alpha1 -kind: TestAssert -timeout: 120 +kind: TestSuite +testDirs: +- ./tests/e2e/ +startKIND: true ``` -Options: +Now we can run the tests just by running `kubectl kudo test` with no arguments. -* `timeout`: the number of seconds to wait for the assertion to be true (default: `10`, type: `int`). +Any arguments provided on the command line will override the settings in the `kudo-test.yaml` file, e.g. to skip using kind and run the tests against a live Kubernetes cluster, run: -## Further Reading +``` +kubectl kudo test --start-kind=false +``` -* [Zookeeper Operator tests](https://github.com/kudobuilder/operators/tree/master/repository/zookeeper/tests) -* Design documentation for test harness: [KEP-0008 - Operator Testing](https://github.com/kudobuilder/kudo/blob/master/keps/0008-operator-testing.md) -* KUDO testing infrastructure and policies document: [KEP-0004 - Add Testing Infrastructure](https://github.com/kudobuilder/kudo/blob/master/keps/0004-add-testing-infrastructure.md) +Now that your first test suite is configured, see [test environments](/docs/testing/test-environments) for documentation on customizing your test environment or the [test step documentation](/docs/testing/steps) to write more advanced tests. diff --git a/docs/testing/asserts.md b/docs/testing/asserts.md new file mode 100644 index 000000000..5246dd68c --- /dev/null +++ b/docs/testing/asserts.md @@ -0,0 +1,52 @@ +--- +title: Asserts +type: docs +menu: + docs: + parent: 'Testing' +weight: 4 +--- + +# Asserts + +Test asserts are the part of a [test step](/docs/testing/steps) that define the state to wait for Kubernetes to reach. It is possible to match specific objects by name as well as match any object that matches a defined state. + +* [Format](#format) +* [Getting a resource from the cluster](#getting-a-resource-from-the-cluster) +* [Listing resources in the cluster](#listing-resources-in-the-cluster) + +# Format + +The test assert file for a test step is found at `$index-assert.yaml`. So, if the test step index is `00`, the assert should be called `00-assert.yaml`. This file can contain any number of objects to match on. If the objects have a namespace set, it will be respected, but if a namespace is not set, then the test harness will look for the objects in the test case's namespace. + +By default, a test step will wait for up to 30 seconds for the defined state to reached, see the [configuration reference](/docs/testing/reference#TestAssert) for documentation on configuring test asserts. + +Note that an assertion file is optional, if it is not present, the test step will be considered successful immediately, once the objects in the test step have been created. It is also valid to create a test step that does not create any objects, but only has an assertion file. + +# Getting a resource from the cluster + +If an object has a name set, then the harness will look specifically for that object to exist and then verify that its state matches what is defined in the assert file. For example, if the assert file has: + +``` +apiVersion: v1 +kind: Pod +metadata: + name: my-pod +status: + phase: Successful +``` + +Then the test harness will wait for the `my-pod` pod in the test namespace to have `status.phase=Successful`. Note that any fields *not* specified in the assert file will be ignored, making it possible to specify only the important fields for the test step. + +# Listing resources in the cluster + +If an object in the assert file has no name set, then the harness will list objects of that kind and expect there to be one that matches. For example, an assert: + +``` +apiVersion: v1 +kind: Pod +status: + phase: Successful +``` + +This example would wait for *any* pod to exist in the test namespace with the `status.phase=Successful`. diff --git a/docs/testing/reference.md b/docs/testing/reference.md new file mode 100644 index 000000000..95f6aa83d --- /dev/null +++ b/docs/testing/reference.md @@ -0,0 +1,93 @@ +--- +title: Configuration Reference +type: docs +menu: + docs: + parent: 'Testing' +weight: 6 +--- + +# Configuration Reference + +* [TestSuite](#testsuite) +* [TestStep](#teststep) +* [TestAssert](#testassert) + +# TestSuite + +The `TestSuite` object specifies the settings for the entire test suite and should live in the test suite configuration file (`kudo-test.yaml` by default, or `--config`): + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestSuite +startKIND: true +startKUDO: true +testDirs: +- tests/e2e/ +timeout: 120 +``` + +Supported settings: + +Field | Type | Description | Default +------------------|-----------------|------------------------------------------------------------------------------------------|-------- +crdDir | string | Path to CRDs to install before running tests. | +manifestDirs | list of strings | Paths to manifests to install before running tests. | +testDirs | list of strings | Directories containing test cases to run. | +startControlPlane | bool | Whether or not to start a local etcd and kubernetes API server for the tests. | false +startKIND | bool | Whether or not to start a local kind cluster for the tests. | false +kindConfig | string | Path to the KIND configuration file to use. | +kindContext | string | KIND context to use. | "kind" +startKUDO | bool | Whether or not to start the KUDO controller for the tests. | false +skipDelete | bool | If set, do not delete the resources after running the tests (implies SkipClusterDelete). | false +skipClusterDelete | bool | If set, do not delete the mocked control plane or kind cluster. | false +timeout | int | Override the default timeout of 30 seconds (in seconds). | 30 +parallel | int | The maximum number of tests to run at once. | 8 +artifactsDir | string | The directory to output artifacts to (current working directory if not specified). | . + +# TestStep + +The `TestStep` object can be used to specify settings for a test step and can be specified in any test step YAML. + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestStep +delete: +- apiVersion: v1 + kind: Pod + name: my-pod +``` + +Supported settings: + +Field | Type | Description +--------|---------------------------|--------------------------------------------------------------------- +delete | list of object references | A list of objects to delete, if they do not already exist, at the beginning of the test step. The test harness will wait for the objects to be successfully deleted before applying the objects in the step. +index | int | Override the test step's index. + +Object Reference: + +Field | Type | Description +-----------|--------|--------------------------------------------------------------------- +apiVersion | string | The Kubernetes API version of the objects to delete. +kind | string | The Kubernetes kind of the objects to delete. +name | string | If specified, the name of the object to delete. If not specified, all objects that match the specified labels will be deleted. +namespace | string | The namespace of the objects to delete. +labels | map | If specified, a label selector to use when looking up objects to delete. If both labels and name are unspecified, then all resources of the specified kind in the namespace will be deleted. + + +# TestAssert + +The `TestAssert` object can be used to specify settings for a test step's assert and must be specified in the test step's assert YAML. + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestAssert +timeout: 30 +``` + +Supported settings: + +Field | Type | Description | Default +--------|------|-------------------------------------------------------|------------- +timeout | int | Number of seconds that the test is allowed to run for | 30 diff --git a/docs/testing/steps.md b/docs/testing/steps.md new file mode 100644 index 000000000..7e882118a --- /dev/null +++ b/docs/testing/steps.md @@ -0,0 +1,112 @@ +--- +title: Steps +type: docs +menu: + docs: + parent: 'Testing' +weight: 3 +--- + +# Steps + +* [Format](#format) +* [Creating objects](#creating-objects) +* [Updating objects](#updating-objects) +* [Deleting objects](#deleting-objects) +* [Kubectl commands](#kubectl-commands) + +Each test case is broken down into test steps. Test steps within a test case are run sequentially: if any of the test steps fail, the entire test case is considered failed. + +A test step can create, update, and delete objects as well as run any kubectl command. + +# Format + +A test step can include many YAML files and each YAML file can contain many Kubernetes objects. In a test case's directory, each file that begins with the same index is considered a part of the same test step. All objects inside of a test step are operated on by the test harness simultaneously, so use separate test steps to order operations. + +E.g., in a test case directory: + +``` +tests/e2e/example/00-pod.yaml +tests/e2e/example/00-example.yaml +tests/e2e/example/01-staging.yaml +``` + +There are two test steps: + +* `00`, which includes `00-pod.yaml` and `00-example.yaml`. +* `01`, which includes `01-staging.yaml`. + +The test harness would run test step `00` and once completed, run test step `01`. + +A namespace is created by the test harness for each test case, so if an object in the step does not have a namespace set, then it will be created in the test case's namespace. If a namespace is set, then that namespace will be respected throughout the tests (making it possible to test resources that reside in standardized namespaces). + +See the [configuration reference](/docs/testing/reference#TestStep) for documentation on configuring test steps. + +# Creating objects + +Any objects specified in a test step will be created if they do not already exist. + +# Updating objects + +If an object does already exist in Kubernetes, then the object in Kubernetes will be updated with the changes specified. + +The test harness uses merge patching for updating objects, so it is possible to specify minimal updates. For example, to change the replicas on a Deployment but leave all other settings untouched, a step could be written: + +``` +apiVersion: apps/v1 +kind: Deployment +metadata: + name: my-deployment +spec: + replicas: 4 +``` + +# Deleting objects + +To delete objects at the beginning of a test step, you can specify object references to delete in your `TestStep` configuration. In a test step file, add a `TestStep` object: + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestStep +delete: +# Delete a Pod +- apiVersion: v1 + kind: Pod + name: my-pod +# Delete all Pods with app=nginx +- apiVersion: v1 + kind: Pod + labels: + app: nginx +# Delete all Pods in the test namespace +- apiVersion: v1 + kind: Pod +``` + +The `delete` object references can delete: + +* A single object by specifying its `name`. +* If `labels` is set and `name` is omitted, then objects matching the labels and kind will be deleted. +* If both `name` and `labels` omitted, all objects of the specified kind in the test namespace will be deleted. + +The test harness will wait for the objects to be successfully deleted, if they exist, before continuing with the test step - if the objects do not get deleted before the timeout has expired the test step is considered failed. + +# Kubectl commands + +A `TestStep` configuration can also specify kubectl commands to run before running the step: + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestStep +kubectl: +- apply -f https://raw.githubusercontent.com/kudobuilder/kudo/master/docs/deployment/10-crds.yaml +``` + +It is also possible to use any installed kubectl plugin when calling kubectl commands: + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestStep +kubectl: +- kudo install zookeeper --skip-instance +``` diff --git a/docs/testing/test-environments.md b/docs/testing/test-environments.md new file mode 100644 index 000000000..4a13e5189 --- /dev/null +++ b/docs/testing/test-environments.md @@ -0,0 +1,119 @@ +--- +title: Test Environments +type: docs +menu: + docs: + parent: 'Testing' +weight: 2 +--- + +# Test Environments + +The kudo test harness can run tests against several different test environments, allowing your test suites to be used in many different environments. + +A default environment for the tests can be defined in `kudo-test.yaml` allowing each test suite or project to easily use the correct environment. + +* [Live cluster](#live-cluster) +* [Kubernetes-in-docker](#kubernetes-in-docker) +* [Mocked control plane](#mocked-control-plane) +* [Environment setup](#environment-setup) + * [Starting KUDO](#starting-kudo) + +# Live cluster + +If no configuration is provided, the tests will run against your default cluster context using whatever Kubernetes cluster is configured in your kubeconfig. + +You can also provide an alternative kubeconfig file by either setting `$KUBECONFIG` or the `--kubeconfig` flag: + +``` +kubectl kudo test --kubeconfig=mycluster.yaml +``` + +# Kubernetes-in-docker + +The kudo test harness has a built in integration with [kind](https://github.com/kubernetes-sigs/kind) to start and interact with kubernetes-in-docker clusters. + +To start a kind cluster in your tests either specify it on the command line: + +``` +kubectl kudo test --start-kind=true +``` + +Or specify it in your `kudo-test.yaml`: + +``` +apiVersion: kudo.k8s.io/v1alpha1 +kind: TestSuite +startKIND: true +``` + +By default kudo will use the default kind cluster name of "kind". If a kind cluster is already running with that name, it will use the existing cluster. + +The kind cluster name can be overriden by setting either `kindContext` in your configuration or `--kind-context` on the command line. + +It is also possible to provide a custom kind configuration file. For example, to override the Kubernetes cluster version, create a kind configuration file called `kind.yaml`: + +``` +kind: Cluster +apiVersion: kind.sigs.k8s.io/v1alpha3 +nodes: +- role: control-plane + image: kindest/node:v1.14.3 +``` + +See the [kind documentation](https://kind.sigs.k8s.io/docs/user/quick-start/#configuring-your-kind-cluster) for all options supported by kind. + +Now specify either `--kind-config` or `kindConfig` in your configuration file: + +``` +kubectl kudo test --kind-config=kind.yaml +``` + +*Note*: Once the tests have been completed, the test harness will collect the kind cluster's logs and then delete it, unless `--skip-cluster-delete` has been set. + +# Mocked control plane + +The above environments are great for end to end testing, however, for integration test use-cases it may be unnecessary to create actual pods or other resources. This can make the tests a lot more flaky or slow than they need to be. + +To write integration tests using the kudo test harness, it is possible to start a mocked control plane that starts only the Kubernetes API server and etcd. In this environment, objects can be created and operated on by custom controllers, however, there is no scheduler, nodes, or built-in controllers. This means that Pods will never run and built-in types, such as, Deployments cannot create Pods. + +Currently, the only supported controller in this environment is the KUDO controller, however, there are plans to expand this to other controllers as well. + +To start the mocked control plane, specify either `--start-control-plane` on the CLI or `startControlPlane` in the configuration file: + +``` +kubectl kudo test --start-control-plane +``` + +# Environment setup + +Before running a test suite, it may be necessary to setup the Kubernetes cluster - typically, either installing required services or custom resource definitions. + +Your `kudo-test.yaml` can specify the settings needed to setup the cluster: + +``` +apiVersion: kudo.dev/v1alpha1 +kind: TestSuite +startKIND: true +testDirs: +- tests/e2e/ +manifestDirs: +- tests/manifests/ +crdDir: tests/crds/ +kubectl: +- apply -f https://raw.githubusercontent.com/kudobuilder/kudo/master/docs/deployment/10-crds.yaml +``` + +The above configuration would start kind, install all of the CRDs in `tests/crds/`, and run all of the commands defined in `kubectl` before running the tests in `testDirs`. + +See the [configuration reference](/docs/testing/reference#TestSuite) for documentation on configuring test suites. + +## Starting KUDO + +In some test suites, it may be useful to have the KUDO controller running. To start the KUDO controller, specify either `--start-kudo` on the command line or `startKUDO` in the configuration file: + +``` +kubectl kudo test --start-kudo=true +``` + +The KUDO controller is built in to the KUDO CLI, so specifying `--start-kudo` will use the version of KUDO the CLI was built with. This makes it easy to test KUDO in a cluster that does not have KUDO installed and also prevents version drift issues when testing. diff --git a/docs/testing/tips.md b/docs/testing/tips.md new file mode 100644 index 000000000..5d9b742d5 --- /dev/null +++ b/docs/testing/tips.md @@ -0,0 +1,83 @@ +--- +title: Tips and Tricks +type: docs +menu: + docs: + parent: 'Testing' +weight: 5 +--- + +# Tips and Tricks + +This document contains some tips and gotchas that can be helpful when writing tests. + +* [Kubernetes Events](#kubernetes-events) +* [Custom Resource Definitions](#custom-resource-definitions) + +# Kubernetes Events + +Kubernetes events are regular Kubernetes objects and can be asserted on just like any other object: + +``` +apiVersion: v1 +kind: Event +reason: Started +source: + component: kubelet +involvedObject: + apiVersion: v1 + kind: Pod + name: my-pod +``` + +# Custom Resource Definitions + +New Custom Resource Definitions are not immediately available for use in the Kubernetes API until the Kubernetes API has acknowledged them. + +If a Custom Resource Definition is being defined inside of a test step, be sure to to wait for the `CustomResourceDefinition` object to appear. + +For example, given this Custom Resource Definition in `tests/e2e/crd-test/00-crd.yaml`: + +``` +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: mycrds.mycrd.k8s.io +spec: + group: mycrd.k8s.io + version: v1alpha1 + names: + kind: MyCRD + listKind: MyCRDList + plural: mycrds + singular: mycrd + scope: Namespaced +``` + +Create the following assert `tests/e2e/crd-test/00-assert.yaml`: + +``` +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: mycrds.mycrd.k8s.io +status: + acceptedNames: + kind: MyCRD + listKind: MyCRDList + plural: mycrds + singular: mycrd + storedVersions: + - v1alpha1 +``` + +And then the CRD can be used in subsequent steps, `tests/e2e/crd-test/01-use.yaml`: + +``` +apiVersion: mycrd.k8s.io/v1alpha1 +kind: MyCRD +spec: + test: test +``` + +Note that CRDs created via the `crdDir` test suite configuration are available for use immediately and do not require an assert like this. diff --git a/site/config.yml b/site/config.yml index caf4ed663..072d9d396 100644 --- a/site/config.yml +++ b/site/config.yml @@ -3,4 +3,5 @@ baseURL: https://kudo.dev/ title: "The Kubernetes Universal Declarative Operator" disableKinds: ["taxonomy", "taxonomyTerm"] + # themesDir: ../.. diff --git a/site/layouts/docs/baseof.html b/site/layouts/docs/baseof.html index 515c13d39..f30f80eb0 100644 --- a/site/layouts/docs/baseof.html +++ b/site/layouts/docs/baseof.html @@ -14,6 +14,17 @@
  • {{ .Pre }}{{ .Name }} + + {{ if .HasChildren }} + + {{ end }}
  • {{ end }}