feat: add otel-demo tutorial

Co-authored-by: Suhyen Im <[email protected]>
Co-authored-by: Jaeyeon Park <[email protected]>
Signed-off-by: Suhyen Im <[email protected]>
Signed-off-by: Jaeyeon Park <[email protected]>

monitoring/README.md

@@ -12,6 +12,10 @@ This directory contains chaos interleaved grafana dashboards along with the util
 
   > Contains utilities required to setup monitoring infrastructure on a kubernetes cluster.
 
+- [Tutorials](./tutorials)
+
+  > Contains tutorials on monitoring target applications under chaos using various tools.
+
 ## Setup the LitmusChaos Infrastructure
 
 - Install the litmus chaos operator and CRDs

monitoring/tutorials/README.md

@@ -0,0 +1,8 @@
+# Tutorials
+
+This directory contains tutorials on monitoring target applications under chaos using various tools.
+
+- [Otel-demo](./otel-demo)
+
+  > Contains a tutorial on setting up and using OpenTelemetry for monitoring target applications under chaos.
+

monitoring/tutorials/otel-demo/README.md

@@ -0,0 +1,101 @@
+# Otel-demo
+
+This tutorial provides instructions on setting up and using OpenTelemetry for monitoring target applications under chaos.
+
+## Setups
+
+### 0. Prerequisites
+- Kubernetes 1.24+
+- 6 GB of free RAM for the application
+- Helm 3.9+
+
+### 1. Install Litmus
+1. Create the Litmus namespace:
+   ```bash
+   kubectl create ns litmus
+   ```
+2. Add the Litmus Helm repository:
+   ```bash
+   helm repo add litmuschaos https://litmuschaos.github.io/litmus-helm/
+   ```
+3. Install Litmus using Helm:
+   ```bash
+   helm install chaos litmuschaos/litmus \
+   --namespace=litmus \
+   --set portal.frontend.service.type=NodePort \
+   --set mongodb.image.registry=ghcr.io/zcube \
+   --set mongodb.image.repository=bitnami-compat/mongodb \
+   --set mongodb.image.tag=6.0.5
+   ```
+4. Verify the installation:
+   ```bash
+   kubectl get all -n litmus
+   ```
+5. Forward the Litmus frontend service port:
+   ```bash
+   kubectl port-forward svc/chaos-litmus-frontend-service 9091:9091 -n litmus
+   ```
+   Access the Litmus frontend at [http://localhost:9091](http://localhost:9091) and log in with `admin` / `litmus`.
+
+### 2. Set Up Litmus Environments
+1. Create a new environment:
+   - Environment Name: `local`
+   - Environment Type: `Production`
+2. Enable Chaos:
+   - Name: `local`
+   - Chaos Components Installation: `Cluster-wide access`
+   - Installation Location (Namespace): `litmus`
+   - Service Account Name: `litmus`
+   - Kubernetes Setup Instructions: Download
+3. Deploy your infrastructure:
+   ```bash
+   cd ~/Downloads
+   kubectl apply -f local-litmus-chaos-enable.yml
+   ```
+   Wait until the status shows `CONNECTED`.
+
+### 3. Install Target Applications & OpenTelemetry
+1. Create the `otel-demo` namespace:
+   ```bash
+   kubectl create ns otel-demo
+   ```
+2. Add the OpenTelemetry Helm repository:
+   ```bash
+   helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
+   ```
+3. Install the OpenTelemetry demo chart:
+   ```bash
+   cd litmus/monitoring/tutorials/otel-demo
+   helm install my-otel-demo open-telemetry/opentelemetry-demo --namespace otel-demo --values custom_otel_demo_values.yml
+   ```
+4. Verify the installation:
+   ```bash
+   kubectl get all -n otel-demo
+   ```
+5. Forward the OpenTelemetry frontend proxy port:
+   ```bash
+   kubectl port-forward svc/my-otel-demo-frontendproxy 8080:8080 -n otel-demo
+   ```
+6. Access the following services:
+   - Web store: [http://localhost:8080/](http://localhost:8080/)
+   - Grafana: [http://localhost:8080/grafana/](http://localhost:8080/grafana/)
+   - Load Generator UI: [http://localhost:8080/loadgen/](http://localhost:8080/loadgen/)
+   - Jaeger UI: [http://localhost:8080/jaeger/ui/](http://localhost:8080/jaeger/ui/)
+
+### 4. Add Grafana Panel
+Import the `chaos-experiments-dashboard.json` file into Grafana to visualize the results of chaos experiments.
+![chaos-experiments-dashboard-1.png](./screenshots/chaos-experiments-dashboard-1.png)
+![chaos-experiments-dashboard-2.png](./screenshots/chaos-experiments-dashboard-2.png)
+
+### 5. Experiments on Target Applications
+Explore the following experiments to test and observe chaos in target applications:
+
+- [Pod Delete](https://github.com/litmuschaos/litmus/blob/master/monitoring/tutorials/otel-demo/recommendation-service)
+  > Performs a Pod delete experiment on the recommendation service.
+
+- [Pod Network Latency](https://github.com/litmuschaos/litmus/blob/master/monitoring/tutorials/otel-demo/cart-service)
+  > Conducts a network latency experiment on the cart service.
+
+---
+
+Feel free to adjust any details as needed!

monitoring/tutorials/otel-demo/cart-service/README.md

@@ -0,0 +1,25 @@
+# cart service pod network latency
+## Description
+- This experiment injects network latency to the cart service pod.
+- The Probe checks Prometheus metrics Latency of cart service requests.
+## Steps
+### 1. Probe Settings
+- probe type: `Prometheus Probe`
+- name: `cart-service-pod-network-latency-probe`
+- timeout: 3s
+- interval: 3s
+- prometheus endpoint: `http://my-otel-demo-prometheus-server.otel-demo:9090`
+- prometheus query: `histogram_quantile(0.99, sum(rate(duration_milliseconds_bucket{service_name=\"cartservice\"}[5m])) by (le))/1000`
+- Data Comparison:
+  - Type: Float
+  - Criteria: `<`
+  - Value: `3.0`
+### 2. Make Experiment
+1. New Experimnet
+2. Complete Overview
+3. Start off by Upload YML(cart-service-pod-network-latency.yml)
+### 3. Run Experiment
+1. Click on the `Run` button
+2. Check Experiment Status and Logs
+3. Check the Resilience Score
+4. Check cart service Spanmetrics Metrics using Grafana ![cartservice_spanmetrics.png](../screenshots/cartservice_spanmetrics.png)