From e032745f670f1923722571f1be041de7e8b63e5e Mon Sep 17 00:00:00 2001 From: Natan Yellin Date: Sun, 10 Dec 2023 21:51:00 +0200 Subject: [PATCH 1/5] Update README.md --- README.md | 202 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 121 insertions(+), 81 deletions(-) diff --git a/README.md b/README.md index 7a181977..bb7335d8 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ . Usage ยท - How it works + How KRR works . Slack Integration
@@ -62,17 +62,18 @@ Robusta KRR (Kubernetes Resource Recommender) is a CLI tool for optimizing resource allocation in Kubernetes clusters. It gathers pod usage data from Prometheus and recommends requests and limits for CPU and memory. This reduces costs and improves performance. -_Supports: [Prometheus](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Thanos](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Victoria Metrics](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Amazon Managed Prometheus](#amazon-managed-prometheus), [Azure](#azure-managed-prometheus), [Coralogix](#coralogix-managed-prometheus) and [Grafana Cloud](#grafana-cloud-managed-prometheus)_ +_Supports: [Prometheus](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Thanos](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Victoria Metrics](#prometheus-victoria-metrics-and-thanos-auto-discovery), [Google Managed Prometheus](./docs/google-cloud-managed-service-for-prometheus.md), [Amazon Managed Prometheus](#amazon-managed-prometheus), [Azure Managed Prometheus](#azure-managed-prometheus), [Coralogix](#coralogix-managed-prometheus) and [Grafana Cloud](#grafana-cloud-managed-prometheus)_ ### Features - **No Agent Required**: Run a CLI tool on your local machine for immediate results. (Or run in-cluster for weekly [Slack reports](#slack-integration).) - **Prometheus Integration**: Get recommendations based on the data you already have +- **Explainability**: Understand how recommendations were calculated - **Extensible Strategies**: Easily create and use your own strategies for calculating resource recommendations. - **Free SaaS Platform**: See why KRR recommends what it does, by using the [free Robusta SaaS platform](https://home.robusta.dev/). - **Future Support**: Upcoming versions will support custom resources (e.g. GPUs) and custom metrics. -### Resource Allocation Statistics +### Why Use KRR? According to a recent [Sysdig study](https://sysdig.com/blog/millions-wasted-kubernetes/), on average, Kubernetes clusters have: @@ -101,7 +102,10 @@ Additionally to that, [kube-state-metrics](https://github.com/kubernetes/kube-st _Note: If one of last three metrics is absent KRR will still work, but it will only consider currently-running pods when calculating recommendations. Historic pods that no longer exist in the cluster will not be taken into consideration._ -### With brew (MacOS/Linux): +### Installation Methods + +
+ Brew (Mac/Linux) 1. Add our tap: @@ -120,12 +124,16 @@ brew install krr ```sh krr --help ``` +
-### On Windows: +
+ Windows -You can install using brew (see above) on [WSL2](https://docs.brew.sh/Homebrew-on-Linux), or install manually. +You can install using brew (see above) on [WSL2](https://docs.brew.sh/Homebrew-on-Linux), or install from source (see below). +
-### Manual Installation +
+ From Source 1. Make sure you have [Python 3.9](https://www.python.org/downloads/) (or greater) installed 2. Clone the repo: @@ -150,37 +158,73 @@ python krr.py --help Notice that using source code requires you to run as a python script, when installing with brew allows to run `krr`. All above examples show running command as `krr ...`, replace it with `python krr.py ...` if you are using a manual installation. -

(back to top)

+
-### Other Configuration Methods +### Additional Options - [View KRR Reports in a Web UI](#optional-free-saas-platform) -- [Get a Weekly Message in Slack with KRR Recommendations](#slack-integration) -- Setup KRR on [Google Cloud Managed Prometheus - ](./docs/google-cloud-managed-service-for-prometheus.md) -- Setup KRR for [Azure managed Prometheus](#azure-managed-prometheus) +- [Receive KRR Reports Weekly in Slack](#slack-integration) + +### Environment-Specific Instructions +Setup KRR for... +- [Google Cloud Managed Prometheus](./docs/google-cloud-managed-service-for-prometheus.md) +- [Azure Managed Prometheus](#azure-managed-prometheus) +- [Amazon Managed Prometheus](#amazon-managed-prometheus) +- [Coralogix Managed Prometheus](#coralogix-managed-prometheus) +- [Grafana Cloud Managed Prometheus](#grafana-cloud-managed-prometheus) + +

(back to top)

## Usage -Straightforward usage, to run the simple strategy: - +
+ Basic usage + ```sh krr simple ``` +
+ +
+ Tweak the recommendation algorithm + +Most helpful flags: -If you want only specific namespaces (default and ingress-nginx): +- `--cpu-min` Sets the minimum recommended cpu value in millicores +- `--mem-min` Sets the minimum recommended memory value in MB +- `--history_duration` The duration of the prometheus history data to use (in hours) + +More specific information on Strategy Settings can be found using + +```sh +krr simple --help +``` +
+ +
+ Run on specific namespaces + +List as many namespaces as you want with `-n` (in this case, `default` and `ingress-nginx`) ```sh krr simple -n default -n ingress-nginx ``` +
-Filtering by labels (more info [here](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#api)): +
+ Run on workloads filtered by label + +Use a label selector ```sh python krr.py simple --selector 'app.kubernetes.io/instance in (robusta, ingress-nginx)' ``` +
+ +
+ Override the kubectl context By default krr will run in the current context. If you want to run it in a different context: @@ -188,36 +232,80 @@ By default krr will run in the current context. If you want to run it in a diffe krr simple -c my-cluster-1 -c my-cluster-2 ``` -If you want to get the output in JSON format (--logtostderr is required so no logs go to the result file): +
+ +
+ Customize output (JSON, YAML, and more + +Currently KRR ships with a few formatters to represent the scan data: + +- `table` - a pretty CLI table used by default, powered by [Rich](https://github.com/Textualize/rich) library +- `json` +- `yaml` +- `pprint` - data representation from python's pprint library + +To run a strategy with a selected formatter, add a `-f` flag: + +```sh +krr simple -f json +``` + +For JSON output, add --logtostderr so no logs go to the result file: ```sh krr simple --logtostderr -f json > result.json ``` -If you want to get the output in YAML format: +For YAML output, do the same: ```sh krr simple --logtostderr -f yaml > result.yaml ``` +
-If you want to see additional debug logs: +
+ Centralized Prometheus (multi-cluster) + +If your Prometheus monitors multiple clusters we require the label you defined for your cluster in Prometheus. + +For example, if your cluster has the Prometheus label `cluster: "my-cluster-name"`, then run this command: ```sh -krr simple -v +krr.py simple --prometheus-label cluster -l my-cluster-name ``` -Other helpful flags: +You may also need the `-p` flag to explicitly give Prometheus' URL. -- `--cpu-min` Sets the minimum recommended cpu value in millicores -- `--mem-min` Sets the minimum recommended memory value in MB -- `--history_duration` The duration of the prometheus history data to use (in hours) -More specific information on Strategy Settings can be found using +
+ Giving an Explicit Prometheus URL + +If your prometheus is not auto-connecting, you can use `kubectl port-forward` for manually forwarding Prometheus. + +For example, if you have a Prometheus Pod called `kube-prometheus-st-prometheus-0`, then run this command to port-forward it: ```sh -krr simple --help +kubectl port-forward pod/kube-prometheus-st-prometheus-0 9090 +``` + +Then, open another terminal and run krr in it, giving an explicit prometheus url: + +```sh +krr simple -p http://127.0.0.1:9090 +``` +
+ + +
+ Debug mode +If you want to see additional debug logs: + +```sh +krr simple -v ``` +
+

(back to top)

## Optional: Free SaaS Platform @@ -232,7 +320,7 @@ With the [free Robusta SaaS platform](https://home.robusta.dev/) you can:

(back to top)

-## How it works +## How KRR works ### Metrics Gathering @@ -278,9 +366,10 @@ Find about how KRR tries to find the default prometheus to connect (back to top)

-## Example of using port-forward for Prometheus - -If your prometheus is not auto-connecting, you can use `kubectl port-forward` for manually forwarding Prometheus. - -For example, if you have a Prometheus Pod called `kube-prometheus-st-prometheus-0`, then run this command to port-forward it: - -```sh -kubectl port-forward pod/kube-prometheus-st-prometheus-0 9090 -``` - -Then, open another terminal and run krr in it, giving an explicit prometheus url: - -```sh -krr simple -p http://127.0.0.1:9090 -``` - -

(back to top)

- -## Scanning with a centralized Prometheus - -If your Prometheus monitors multiple clusters we require the label you defined for your cluster in Prometheus. - -For example, if your cluster has the Prometheus label `cluster: "my-cluster-name"` and your prometheus is at url `http://my-centralized-prometheus:9090`, then run this command: - -```sh -krr.py simple -p http://my-centralized-prometheus:9090 --prometheus-label cluster -l my-cluster-name -``` - -

(back to top)

- -## Azure managed Prometheus +## Azure Managed Prometheus For Azure managed Prometheus you need to generate an access token, which can be done by running the following command: @@ -431,7 +490,7 @@ Additional optional parameters are:

(back to top)

-## Coralogix managed Prometheus +## Coralogix Managed Prometheus For Coralogix managed Prometheus you need to specify your prometheus link and add the flag coralogix_token with your Logs Query Key @@ -443,7 +502,7 @@ python krr.py simple -p "https://prom-api.coralogix..." --coralogix_token

(back to top)

-## Grafana Cloud managed Prometheus +## Grafana Cloud Managed Prometheus For Grafana Cloud managed Prometheus you need to specify prometheus link, prometheus user, and an access token of your Grafana Cloud stack. The Prometheus link and user for the stack can be found on the Grafana Cloud Portal. An access token with a `metrics:read` scope can also be created using Access Policies on the same portal. @@ -457,25 +516,6 @@ python krr.py simple -p $PROM_URL --prometheus-auth-header "Bearer ${PROM_USER}:

(back to top)

- - -## Available formatters - -Currently KRR ships with a few formatters to represent the scan data: - -- `table` - a pretty CLI table used by default, powered by [Rich](https://github.com/Textualize/rich) library -- `json` -- `yaml` -- `pprint` - data representation from python's pprint library - -To run a strategy with a selected formatter, add a `-f` flag: - -```sh -krr simple -f json -``` - -

(back to top)

- ## Creating a Custom Strategy/Formatter From 6f3e2e14bfc8161369baeb5323bd92295bbf4eca Mon Sep 17 00:00:00 2001 From: Natan Yellin Date: Sun, 10 Dec 2023 21:52:12 +0200 Subject: [PATCH 2/5] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index bb7335d8..9a98ae5f 100644 --- a/README.md +++ b/README.md @@ -275,7 +275,7 @@ krr.py simple --prometheus-label cluster -l my-cluster-name ``` You may also need the `-p` flag to explicitly give Prometheus' URL. - +
Giving an Explicit Prometheus URL From 0435a3e43b453437572ac00703ced816b3dce6eb Mon Sep 17 00:00:00 2001 From: Natan Yellin Date: Sun, 10 Dec 2023 21:53:09 +0200 Subject: [PATCH 3/5] Update README.md --- README.md | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 9a98ae5f..02fecaf7 100644 --- a/README.md +++ b/README.md @@ -203,6 +203,24 @@ krr simple --help ```
+
+ Giving an Explicit Prometheus URL + +If your prometheus is not auto-connecting, you can use `kubectl port-forward` for manually forwarding Prometheus. + +For example, if you have a Prometheus Pod called `kube-prometheus-st-prometheus-0`, then run this command to port-forward it: + +```sh +kubectl port-forward pod/kube-prometheus-st-prometheus-0 9090 +``` + +Then, open another terminal and run krr in it, giving an explicit prometheus url: + +```sh +krr simple -p http://127.0.0.1:9090 +``` +
+
Run on specific namespaces @@ -277,25 +295,6 @@ krr.py simple --prometheus-label cluster -l my-cluster-name You may also need the `-p` flag to explicitly give Prometheus' URL.
-
- Giving an Explicit Prometheus URL - -If your prometheus is not auto-connecting, you can use `kubectl port-forward` for manually forwarding Prometheus. - -For example, if you have a Prometheus Pod called `kube-prometheus-st-prometheus-0`, then run this command to port-forward it: - -```sh -kubectl port-forward pod/kube-prometheus-st-prometheus-0 9090 -``` - -Then, open another terminal and run krr in it, giving an explicit prometheus url: - -```sh -krr simple -p http://127.0.0.1:9090 -``` -
- -
Debug mode If you want to see additional debug logs: From 38cbd4b8112322f674d02f5344a8067d551a3ed7 Mon Sep 17 00:00:00 2001 From: Natan Yellin Date: Sun, 10 Dec 2023 21:54:32 +0200 Subject: [PATCH 4/5] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 02fecaf7..1508494e 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ According to a recent [Sysdig study](https://sysdig.com/blog/millions-wasted-kub By right-sizing your containers with KRR, you can save an average of 69% on cloud costs. -Read more about [how KRR works](#how-it-works) and [KRR vs Kubernetes VPA](#difference-with-kubernetes-vpa) +Read more about [how KRR works](#how-krr-works) and [KRR vs Kubernetes VPA](#difference-with-kubernetes-vpa) From 618effeb3e3147a4e63dd6eeecbaf146a6ae5ad9 Mon Sep 17 00:00:00 2001 From: Natan Yellin Date: Sun, 10 Dec 2023 21:57:25 +0200 Subject: [PATCH 5/5] Update README.md --- README.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 1508494e..f625aacd 100644 --- a/README.md +++ b/README.md @@ -283,16 +283,8 @@ krr simple --logtostderr -f yaml > result.yaml
Centralized Prometheus (multi-cluster) +

See below on filtering output from a centralized prometheus, so it matches only one cluster

-If your Prometheus monitors multiple clusters we require the label you defined for your cluster in Prometheus. - -For example, if your cluster has the Prometheus label `cluster: "my-cluster-name"`, then run this command: - -```sh -krr.py simple --prometheus-label cluster -l my-cluster-name -``` - -You may also need the `-p` flag to explicitly give Prometheus' URL.
@@ -447,6 +439,17 @@ If none of those labels result in finding Prometheus, Victoria Metrics or Thanos

(back to top)

+## Scanning with a Centralized Prometheus +If your Prometheus monitors multiple clusters we require the label you defined for your cluster in Prometheus. + +For example, if your cluster has the Prometheus label `cluster: "my-cluster-name"`, then run this command: + +```sh +krr.py simple --prometheus-label cluster -l my-cluster-name +``` + +You may also need the `-p` flag to explicitly give Prometheus' URL. + ## Azure Managed Prometheus For Azure managed Prometheus you need to generate an access token, which can be done by running the following command: