The intent of this Operator is to deploy common workloads to establish a performance baseline of Kubernetes cluster on your provider.
Installing the benchmark-operator is easiest by using the helm chart and can be done with the following commands. This requires your machine to have Helm installed. Install Helm
Note: If running on openshift you'll need to run this command before installing the chart.
oc adm policy -n my-ripsaw add-scc-to-user privileged -z benchmark-operator
git clone https://github.com/cloud-bulldozer/benchmark-operator
cd benchmark-operator/charts/benchmark-operator
helm install benchmark-operator . -n my-ripsaw --create-namespaceTo delete this release, you can do so with the following command:
helm delete benchmark-operator -n my-ripsaw --purge| Workload | Use | ElasticSearch indexing | Reconciliation usage | VM support (kubevirt) | Kata Containers | CI Tested |
|---|---|---|---|---|---|---|
| UPerf | Network Performance | Yes | Used, default : 3second | Working | Working | Yes |
| Iperf3 | Network Performance | No | Used, default : 3second | Not Supported | Preview | Yes |
| fio | Storage IO | Yes | Used, default : 3second | Working | Working | Yes |
| Sysbench | System Performance | No | Used, default : 3second | Not Supported | Preview | Yes |
| YCSB | Database Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| Byowl | User defined workload | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| Pgbench | Postgres Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| Smallfile | Storage IO Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| fs-drift | Storage IO Longevity | Yes | Not used | Not Supported | Preview | Yes |
| hammerdb | Database Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| Service Mesh | Microservices | No | Used, default : 3second | Not Supported | Preview | No |
| Vegeta | HTTP Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| Scale Openshift | Scale Openshift Cluster | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| stressng | Stress system resources | Yes | Used, default: 3second | Working | Preview | Yes |
| kube-burner | k8s Performance | Yes | Used, default : 3second | Not Supported | Preview | Yes |
| cyclictest | Real-Time Performance | Yes | Used, default : 3second | Not Supported | Preview | No |
| oslat | Real-Time Latency | Yes | Used, default : 3second | Not Supported | Preview | No |
| testpmd | TestPMD DPDK App | No | Used | Not Supported | Preview | No |
| Flent | Network Performance | Yes | Used, default : 3second | Not Supported | Not Supported | Yes |
Previously the Benchmark Operator didn't properly take advantage of the reconciliation period. Going forward we will make every attempt to utilize the reconciliation period.
Why did we decide to switch to this? Our operator would implement long running tasks, due to the nature of benchmarks.
However, long running tasks blocks the Operator, causing us to delete the Operator and re-create the operator to
un-block it. The benchmarks mentioned above that state Used for Reconciliation, no longer have this issue.
Optional locations for workload images can now be added easily without the need to rebuild the operator. To do so in the workload args section of the CR add image: [location]
NOTE: This is not a required arguement. If omitted it will default to the currently verified workload image. Additionally, this is NOT enabled for YCSB
For Example:
apiVersion: ripsaw.cloudbulldozer.io/v1alpha1
kind: Benchmark
metadata:
name: example-benchmark
namespace: my-ripsaw
spec:
elasticsearch:
url: "http://my-es.foo.bar:80"
metadata_collection: true
cleanup: false
workload:
name: "foo"
args:
image: my.location/foo:latest
All benchmarks in the benchmark-operator utilize a UUID for tracking and indexing purposes. This UUID is, by default, generated when the workload is first started. However, if desired, a user provided UUID can be added to the workload cr.
NOTE: The provided UUID must be in format XXXXX-XXXXX-XXXXX
For Example:
apiVersion: ripsaw.cloudbulldozer.io/v1alpha1
kind: Benchmark
metadata:
name: example-benchmark
namespace: my-ripsaw
spec:
uuid: 6060004a-7515-424e-93bb-c49844600dde
elasticsearch:
url: "http://my-es.foo.bar:80"
metadata_collection: true
cleanup: false
workload:
name: "foo"
args:
image: my.location/foo:latest
Key Members(slack_usernames): ravi, mohit, dry923, rsevilla or rook