fix(helm): Various fixes and enhancements for bloom components #14128

chaudum · 2024-09-12T11:44:57Z

What this PR does / why we need it:

Fix: Do not create bloom Deployment/Statefulset if replicas=0 (fixes helm: bloom-builder deployment created always #14090)
Fix: Configure PV claims for bloom gateway and bloom builder correctly (fixes bloom-builder: new component cannot start up because helm chart is not creating /var/loki path #14082)
Fix: Configure emphemeral storage for bloom builder working directory (fixes bloom-builder: panic: mkdir /tmp/bloom-block-1802389595: read-only file system #14084)
Add required configuration for bloom builder and bloom gateway

Checklist

Reviewed the CONTRIBUTING.md guide (required)
Documentation added
Tests updated
Title matches the required conventional commits format, see here
- Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

diranged · 2024-09-12T15:40:38Z

production/helm/loki/templates/bloom-builder/service-bloom-builder-headless.yaml

@@ -1,5 +1,5 @@
 {{- $isDistributed := eq (include "loki.deployment.isDistributed" .) "true" -}}
-{{- if $isDistributed -}}
+{{- if (and $isDistributed (gt (int .Values.bloomBuilder.replicas) 0)) -}}


I think you need an or .Values.bloomBuilder.autoscaling.enabled as well ... because we explicitly leave replicas: 0 and set autosacling.enabled in our environment

Good catch! Haven't thought about that.

diranged

Mostly good, one comment.. thanks for addressing these issues!

trevorwhitney

LGTM, can we get a volunteer from the group that submitted bugs from the first PR to run this branch? We all run things a little different, and I'd like to make sure we're actually fixing the problem for those who reported it (since it wasn't a problem for us).

chaudum · 2024-09-17T14:59:53Z

LGTM, can we get a volunteer from the group that submitted bugs from the first PR to run this branch? We all run things a little different, and I'd like to make sure we're actually fixing the problem for those who reported it (since it wasn't a problem for us).

@diranged @z0rc We would very much appreciate if you could test this branch before we merge it. We want to ensure that we fix the mentioned issues.

mzupan · 2024-09-23T02:30:40Z

for what its worth I tested this out. I'm not running distrubuted but basically the read/write/gateway and it appears to be working

loki-backend-2 loki level=info ts=2024-09-23T02:27:15.848562225Z caller=planner.go:181 component=bloom-planner msg="starting initial bloom build iteration"
loki-backend-2 loki level=info ts=2024-09-23T02:27:15.848643576Z caller=planner.go:243 component=bloom-planner msg="running bloom build iteration"
loki-backend-2 loki level=info ts=2024-09-23T02:27:15.955126399Z caller=planner.go:361 component=bloom-planner msg="bloom build iteration completed" tasks=0 tasksSucceed=0 duration=0.106483693

Signed-off-by: Christian Haudum <[email protected]>

chaudum · 2024-09-23T06:27:27Z

for what its worth I tested this out. I'm not running distrubuted but basically the read/write/gateway and it appears to be working

loki-backend-2 loki level=info ts=2024-09-23T02:27:15.848562225Z caller=planner.go:181 component=bloom-planner msg="starting initial bloom build iteration"
loki-backend-2 loki level=info ts=2024-09-23T02:27:15.848643576Z caller=planner.go:243 component=bloom-planner msg="running bloom build iteration"
loki-backend-2 loki level=info ts=2024-09-23T02:27:15.955126399Z caller=planner.go:361 component=bloom-planner msg="bloom build iteration completed" tasks=0 tasksSucceed=0 duration=0.106483693

Thank you!

github-actions · 2024-09-23T06:27:53Z

Kubernetes Manifest Diff Summary

Scenario: default-single-binary-values (Added: 0, Modified: 11, Removed: 3)

Summary:

Added: 0
Modified: 11
Removed: 3

Added Files

No added files

Modified Files

loki/templates/query-scheduler/deployment-query-scheduler.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/query-scheduler/deployment-query-scheduler.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/query-scheduler/deployment-query-scheduler.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 27,33 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 27,33 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/querier/deployment-querier.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/querier/deployment-querier.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/querier/deployment-querier.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/index-gateway/statefulset-index-gateway.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/index-gateway/statefulset-index-gateway.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/index-gateway/statefulset-index-gateway.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 26,32 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 26,32 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/ingester/statefulset-ingester-zone-c.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-c.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-c.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 34,40 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 34,40 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/ingester/statefulset-ingester-zone-b.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-b.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-b.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 34,40 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 34,40 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/ingester/statefulset-ingester-zone-a.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-a.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/ingester/statefulset-ingester-zone-a.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 34,40 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 34,40 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/config.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/config.yaml	2024-09-23 06:27:46.645579818 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/config.yaml	2024-09-23 06:27:48.716604461 +0000
***************
*** 15,20 ****
--- 15,28 ----
 config.yaml: |
 
 auth_enabled: true
+ bloom_build:
+ builder:
+ planner_address: loki-test-chart-name-bloom-planner-headless.default.svc.cluster.local:9095
+ enabled: false
+ bloom_gateway:
+ client:
+ addresses: dnssrvnoa+_grpc._tcp.loki-test-chart-name-bloom-gateway-headless.default.svc.cluster.local
+ enabled: false
 chunk_store_config:
 chunk_cache_config:
 background:
***************
*** 102,107 ****
--- 110,117 ----
 http_server_read_timeout: 600s
 http_server_write_timeout: 600s
 storage_config:
+ bloom_shipper:
+ working_directory: /var/loki/data/bloomshipper
 boltdb_shipper:
 index_gateway_client:
 server_address: dns+loki-test-chart-name-index-gateway-headless.default.svc.cluster.local:9095

loki/templates/distributor/deployment-distributor.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/distributor/deployment-distributor.yaml	2024-09-23 06:27:46.646579831 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/distributor/deployment-distributor.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/query-frontend/deployment-query-frontend.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/query-frontend/deployment-query-frontend.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/query-frontend/deployment-query-frontend.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 27,33 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 27,33 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/ruler/statefulset-ruler.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/ruler/statefulset-ruler.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/ruler/statefulset-ruler.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 24,30 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 24,30 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/compactor/statefulset-compactor.yaml

*** /home/runner/work/loki/loki/output/base/default-single-binary-values/loki/templates/compactor/statefulset-compactor.yaml	2024-09-23 06:27:46.647579842 +0000
--- /home/runner/work/loki/loki/output/pr/default-single-binary-values/loki/templates/compactor/statefulset-compactor.yaml	2024-09-23 06:27:48.718604485 +0000
***************
*** 29,35 ****
 template:
 metadata:
 annotations:
! checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 29,35 ----
 template:
 metadata:
 annotations:
! checksum/config: 3a704f0745d9b20313bee4136da3f795ba47ce32c539a51990d1b95e8bd51eb0
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

Removed Files

loki/templates/bloom-builder/service-bloom-builder.yaml

---
# Source: loki/templates/bloom-builder/service-bloom-builder.yaml
apiVersion: v1
kind: Service
metadata:
 name: loki-test-chart-name-bloom-builder
 namespace: default
 labels:
 helm.sh/chart: loki-6.13.0
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/version: "3.1.1"
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: bloom-builder
spec:
 type: ClusterIP
 publishNotReadyAddresses: true
 ports:
 - name: http-metrics
 port: 3100
 targetPort: http-metrics
 protocol: TCP
 - name: grpc
 port: 9095
 targetPort: grpc
 protocol: TCP
 - name: grpclb
 port: 9096
 targetPort: grpc
 protocol: TCP
 selector:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: bloom-builder

loki/templates/bloom-builder/deployment-bloom-builder.yaml

---
# Source: loki/templates/bloom-builder/deployment-bloom-builder.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
 name: loki-test-chart-name-bloom-builder
 namespace: default
 labels:
 helm.sh/chart: loki-6.13.0
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/version: "3.1.1"
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: bloom-builder
spec:
 replicas: 0
 strategy:
 rollingUpdate:
 maxSurge: 0
 maxUnavailable: 1
 revisionHistoryLimit: 10
 selector:
 matchLabels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: bloom-builder
 template:
 metadata:
 annotations:
 checksum/config: a7e3454f7154399eb2c053dc5285c70dded26d29c3b5f80fc62da92650d20762
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: bloom-builder
 app.kubernetes.io/part-of: memberlist
 spec:
 serviceAccountName: loki
 
 securityContext:
 fsGroup: 10001
 runAsGroup: 10001
 runAsNonRoot: true
 runAsUser: 10001
 terminationGracePeriodSeconds: 30
 containers:
 - name: bloom-builder
 image: docker.io/grafana/loki:3.1.1
 imagePullPolicy: IfNotPresent
 args:
 - -config.file=/etc/loki/config/config.yaml
 - -target=bloom-builder
 ports:
 - name: http-metrics
 containerPort: 3100
 protocol: TCP
 - name: grpc
 containerPort: 9095
 protocol: TCP
 - name: http-memberlist
 containerPort: 7946
 protocol: TCP
 securityContext:
 allowPrivilegeEscalation: false
 capabilities:
 drop:
 - ALL
 readOnlyRootFilesystem: true
 readinessProbe:
 httpGet:
 path: /ready
 port: http-metrics
 initialDelaySeconds: 30
 timeoutSeconds: 1
 volumeMounts:
 - name: config
 mountPath: /etc/loki/config
 - name: runtime-config
 mountPath: /etc/loki/runtime-config
 resources:
 {}
 affinity:
 podAntiAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 - labelSelector:
 matchLabels:
 app.kubernetes.io/component: bloom-builder
 topologyKey: kubernetes.io/hostname
 volumes:
 - name: config
 configMap:
 name: loki
 items:
 - key: "config.yaml"
 path: "config.yaml"
 - name: runtime-config
 configMap:
 name: loki-runtime

loki/templates/bloom-builder/service-bloom-builder-headless.yaml

---
# Source: loki/templates/bloom-builder/service-bloom-builder-headless.yaml
apiVersion: v1
kind: Service
metadata:
 name: loki-test-chart-name-bloom-builder-headless
 namespace: default
 labels:
 helm.sh/chart: loki-6.13.0
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/version: "3.1.1"
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: bloom-builder
 prometheus.io/service-monitor: "false"
spec:
 clusterIP: None
 type: ClusterIP
 publishNotReadyAddresses: true
 ports:
 - name: http-metrics
 port: 3100
 targetPort: http-metrics
 protocol: TCP
 - name: grpc
 port: 9095
 targetPort: grpc
 protocol: TCP
 - name: grpclb
 port: 9096
 targetPort: grpc
 protocol: TCP
 selector:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: bloom-builder

Scenario: default-values (Added: 0, Modified: 4, Removed: 0)

Summary:

Added: 0
Modified: 4
Removed: 0

Added Files

No added files

Modified Files

loki/templates/backend/statefulset-backend.yaml

*** /home/runner/work/loki/loki/output/base/default-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:46.739580937 +0000
--- /home/runner/work/loki/loki/output/pr/default-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:48.810605580 +0000
***************
*** 33,39 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 33,39 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/config.yaml

*** /home/runner/work/loki/loki/output/base/default-values/loki/templates/config.yaml	2024-09-23 06:27:46.737580913 +0000
--- /home/runner/work/loki/loki/output/pr/default-values/loki/templates/config.yaml	2024-09-23 06:27:48.809605568 +0000
***************
*** 15,20 ****
--- 15,28 ----
 config.yaml: |
 
 auth_enabled: true
+ bloom_build:
+ builder:
+ planner_address: loki-backend-headless.default.svc.cluster.local:9095
+ enabled: false
+ bloom_gateway:
+ client:
+ addresses: dnssrvnoa+_grpc._tcp.loki-backend-headless.default.svc.cluster.local
+ enabled: false
 chunk_store_config:
 chunk_cache_config:
 background:
***************
*** 97,102 ****
--- 105,112 ----
 http_server_read_timeout: 600s
 http_server_write_timeout: 600s
 storage_config:
+ bloom_shipper:
+ working_directory: /var/loki/data/bloomshipper
 boltdb_shipper:
 index_gateway_client:
 server_address: dns+loki-backend-headless.default.svc.cluster.local:9095

loki/templates/write/statefulset-write.yaml

*** /home/runner/work/loki/loki/output/base/default-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:46.739580937 +0000
--- /home/runner/work/loki/loki/output/pr/default-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:48.811605592 +0000
***************
*** 29,35 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 29,35 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/read/deployment-read.yaml

*** /home/runner/work/loki/loki/output/base/default-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:46.739580937 +0000
--- /home/runner/work/loki/loki/output/pr/default-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:48.810605580 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki

Removed Files

No removed files

Scenario: ingress-values (Added: 0, Modified: 4, Removed: 0)

Summary:

Added: 0
Modified: 4
Removed: 0

Added Files

No added files

Modified Files

loki/templates/backend/statefulset-backend.yaml

*** /home/runner/work/loki/loki/output/base/ingress-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:46.830582020 +0000
--- /home/runner/work/loki/loki/output/pr/ingress-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:48.904606698 +0000
***************
*** 33,39 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 33,39 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/config.yaml

*** /home/runner/work/loki/loki/output/base/ingress-values/loki/templates/config.yaml	2024-09-23 06:27:46.828581996 +0000
--- /home/runner/work/loki/loki/output/pr/ingress-values/loki/templates/config.yaml	2024-09-23 06:27:48.903606687 +0000
***************
*** 15,20 ****
--- 15,28 ----
 config.yaml: |
 
 auth_enabled: true
+ bloom_build:
+ builder:
+ planner_address: loki-backend-headless.default.svc.cluster.local:9095
+ enabled: false
+ bloom_gateway:
+ client:
+ addresses: dnssrvnoa+_grpc._tcp.loki-backend-headless.default.svc.cluster.local
+ enabled: false
 chunk_store_config:
 chunk_cache_config:
 background:
***************
*** 97,102 ****
--- 105,112 ----
 http_server_read_timeout: 600s
 http_server_write_timeout: 600s
 storage_config:
+ bloom_shipper:
+ working_directory: /var/loki/data/bloomshipper
 boltdb_shipper:
 index_gateway_client:
 server_address: dns+loki-backend-headless.default.svc.cluster.local:9095

loki/templates/write/statefulset-write.yaml

*** /home/runner/work/loki/loki/output/base/ingress-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:46.830582020 +0000
--- /home/runner/work/loki/loki/output/pr/ingress-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:48.904606698 +0000
***************
*** 29,35 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 29,35 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/read/deployment-read.yaml

*** /home/runner/work/loki/loki/output/base/ingress-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:46.830582020 +0000
--- /home/runner/work/loki/loki/output/pr/ingress-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:48.904606698 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki

Removed Files

No removed files

Scenario: legacy-monitoring-values (Added: 0, Modified: 4, Removed: 0)

Summary:

Added: 0
Modified: 4
Removed: 0

Added Files

No added files

Modified Files

loki/templates/backend/statefulset-backend.yaml

*** /home/runner/work/loki/loki/output/base/legacy-monitoring-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:46.928583186 +0000
--- /home/runner/work/loki/loki/output/pr/legacy-monitoring-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:49.004607888 +0000
***************
*** 33,39 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 33,39 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/config.yaml

*** /home/runner/work/loki/loki/output/base/legacy-monitoring-values/loki/templates/config.yaml	2024-09-23 06:27:46.927583174 +0000
--- /home/runner/work/loki/loki/output/pr/legacy-monitoring-values/loki/templates/config.yaml	2024-09-23 06:27:49.003607876 +0000
***************
*** 15,20 ****
--- 15,28 ----
 config.yaml: |
 
 auth_enabled: true
+ bloom_build:
+ builder:
+ planner_address: loki-backend-headless.default.svc.cluster.local:9095
+ enabled: false
+ bloom_gateway:
+ client:
+ addresses: dnssrvnoa+_grpc._tcp.loki-backend-headless.default.svc.cluster.local
+ enabled: false
 chunk_store_config:
 chunk_cache_config:
 background:
***************
*** 97,102 ****
--- 105,112 ----
 http_server_read_timeout: 600s
 http_server_write_timeout: 600s
 storage_config:
+ bloom_shipper:
+ working_directory: /var/loki/data/bloomshipper
 boltdb_shipper:
 index_gateway_client:
 server_address: dns+loki-backend-headless.default.svc.cluster.local:9095

loki/templates/write/statefulset-write.yaml

*** /home/runner/work/loki/loki/output/base/legacy-monitoring-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:46.928583186 +0000
--- /home/runner/work/loki/loki/output/pr/legacy-monitoring-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:49.004607888 +0000
***************
*** 29,35 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name
--- 29,35 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/name: loki
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/read/deployment-read.yaml

*** /home/runner/work/loki/loki/output/base/legacy-monitoring-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:46.928583186 +0000
--- /home/runner/work/loki/loki/output/pr/legacy-monitoring-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:49.004607888 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: 50875f962cafe0724798b63920b3403f07dddc2e97630fba443e8de4d3bfd76d
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: 43e7f774023ec5af4d02af739ab8867a2342f4173635103c7d4c71999c84f632
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: loki

Removed Files

No removed files

Scenario: simple-scalable-aws-kube-irsa-values (Added: 0, Modified: 6, Removed: 0)

Summary:

Added: 0
Modified: 6
Removed: 0

Added Files

No added files

Modified Files

loki/templates/backend/statefulset-backend.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:47.032584424 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/backend/statefulset-backend.yaml	2024-09-23 06:27:49.114609197 +0000
***************
*** 33,39 ****
 template:
 metadata:
 annotations:
! checksum/config: 3c395f3b5ad04718fba5377dd110239fd453c73781221cdd1f40b324320fe38b
 labels:
 app.kubernetes.io/name: enterprise-logs
 app.kubernetes.io/instance: loki-test-chart-name
--- 33,39 ----
 template:
 metadata:
 annotations:
! checksum/config: bbb678049f2a53f448530a4a7b842310fa561a334b43cb0659cbdfd8cf9d223c
 labels:
 app.kubernetes.io/name: enterprise-logs
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/admin-api/deployment-admin-api.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/admin-api/deployment-admin-api.yaml	2024-09-23 06:27:47.032584424 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/admin-api/deployment-admin-api.yaml	2024-09-23 06:27:49.114609197 +0000
***************
*** 33,39 ****
 target: admin-api
 app.kubernetes.io/part-of: memberlist
 annotations:
! checksum/config: 3c395f3b5ad04718fba5377dd110239fd453c73781221cdd1f40b324320fe38b
 spec:
 serviceAccountName: enterprise-logs
 securityContext:
--- 33,39 ----
 target: admin-api
 app.kubernetes.io/part-of: memberlist
 annotations:
! checksum/config: bbb678049f2a53f448530a4a7b842310fa561a334b43cb0659cbdfd8cf9d223c
 spec:
 serviceAccountName: enterprise-logs
 securityContext:

loki/templates/gateway/deployment-gateway-enterprise.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/gateway/deployment-gateway-enterprise.yaml	2024-09-23 06:27:47.032584424 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/gateway/deployment-gateway-enterprise.yaml	2024-09-23 06:27:49.114609197 +0000
***************
*** 28,34 ****
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: gateway
 annotations:
! checksum/config: 3c395f3b5ad04718fba5377dd110239fd453c73781221cdd1f40b324320fe38b
 spec:
 serviceAccountName: enterprise-logs
 securityContext:
--- 28,34 ----
 app.kubernetes.io/instance: loki-test-chart-name
 app.kubernetes.io/component: gateway
 annotations:
! checksum/config: bbb678049f2a53f448530a4a7b842310fa561a334b43cb0659cbdfd8cf9d223c
 spec:
 serviceAccountName: enterprise-logs
 securityContext:

loki/templates/config.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/config.yaml	2024-09-23 06:27:47.031584411 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/config.yaml	2024-09-23 06:27:49.113609185 +0000
***************
*** 22,27 ****
--- 22,35 ----
 auth:
 type: enterprise
 auth_enabled: true
+ bloom_build:
+ builder:
+ planner_address: enterprise-logs-backend-headless.default.svc.cluster.local:9095
+ enabled: false
+ bloom_gateway:
+ client:
+ addresses: dnssrvnoa+_grpc._tcp.enterprise-logs-backend-headless.default.svc.cluster.local
+ enabled: false
 chunk_store_config:
 chunk_cache_config:
 background:
***************
*** 109,114 ****
--- 117,124 ----
 http_server_read_timeout: 600s
 http_server_write_timeout: 600s
 storage_config:
+ bloom_shipper:
+ working_directory: /var/loki/data/bloomshipper
 boltdb_shipper:
 index_gateway_client:
 server_address: dns+enterprise-logs-backend-headless.default.svc.cluster.local:9095

loki/templates/write/statefulset-write.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:47.033584435 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/write/statefulset-write.yaml	2024-09-23 06:27:49.114609197 +0000
***************
*** 29,35 ****
 template:
 metadata:
 annotations:
! checksum/config: 3c395f3b5ad04718fba5377dd110239fd453c73781221cdd1f40b324320fe38b
 labels:
 app.kubernetes.io/name: enterprise-logs
 app.kubernetes.io/instance: loki-test-chart-name
--- 29,35 ----
 template:
 metadata:
 annotations:
! checksum/config: bbb678049f2a53f448530a4a7b842310fa561a334b43cb0659cbdfd8cf9d223c
 labels:
 app.kubernetes.io/name: enterprise-logs
 app.kubernetes.io/instance: loki-test-chart-name

loki/templates/read/deployment-read.yaml

*** /home/runner/work/loki/loki/output/base/simple-scalable-aws-kube-irsa-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:47.032584424 +0000
--- /home/runner/work/loki/loki/output/pr/simple-scalable-aws-kube-irsa-values/loki/templates/read/deployment-read.yaml	2024-09-23 06:27:49.114609197 +0000
***************
*** 28,34 ****
 template:
 metadata:
 annotations:
! checksum/config: 3c395f3b5ad04718fba5377dd110239fd453c73781221cdd1f40b324320fe38b
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: enterprise-logs
--- 28,34 ----
 template:
 metadata:
 annotations:
! checksum/config: bbb678049f2a53f448530a4a7b842310fa561a334b43cb0659cbdfd8cf9d223c
 labels:
 app.kubernetes.io/part-of: memberlist
 app.kubernetes.io/name: enterprise-logs

Removed Files

No removed files

…na#14128) * Fix: Do not create bloom Deployment/Statefulset if replicas=0 (fixes grafana#14090) * Fix: Configure PV claims for bloom gateway and bloom builder correctly (fixes grafana#14082) * Fix: Configure emphemeral storage for bloom builder working directory (fixes grafana#14084) * Add required configuration for bloom builder and bloom gateway --- Signed-off-by: Christian Haudum <[email protected]>

chaudum requested a review from a team as a code owner September 12, 2024 11:44

pull-request-size bot added the size/L label Sep 12, 2024

github-actions bot added area/helm type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories labels Sep 12, 2024

chaudum mentioned this pull request Sep 12, 2024

feat(helm): Replace bloom compactor with bloom planner and builder #14003

Merged

7 tasks

chaudum force-pushed the chaudum/helm-fix-bloom-planner-builder-storage branch 2 times, most recently from e287c7c to 6d45dbf Compare September 12, 2024 14:58

diranged reviewed Sep 12, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

trevorwhitney approved these changes Sep 17, 2024

View reviewed changes

chaudum added 5 commits September 23, 2024 08:26

Add required configuration for bloom builder and bloom gateway

ea44045

Signed-off-by: Christian Haudum <[email protected]>

Fix: Configure emphemeral storage for bloom builder working directory

9386938

Signed-off-by: Christian Haudum <[email protected]>

Fix: Configure PV claims for bloom gateway and bloom builder correctly

9d0040a

Signed-off-by: Christian Haudum <[email protected]>

Fix: Do not create bloom Deployment/Statefulset if replicas=0

404a875

Signed-off-by: Christian Haudum <[email protected]>

Fix type in Helm changelog

3ea028b

Signed-off-by: Christian Haudum <[email protected]>

chaudum force-pushed the chaudum/helm-fix-bloom-planner-builder-storage branch from 00cb557 to 3ea028b Compare September 23, 2024 06:27

chaudum merged commit dc0cbd4 into main Sep 23, 2024
66 checks passed

chaudum deleted the chaudum/helm-fix-bloom-planner-builder-storage branch September 23, 2024 06:45

loki-gh-app bot mentioned this pull request Sep 30, 2024

chore(k222): release 3.2.0 #14305

Open

loki-gh-app bot mentioned this pull request Oct 7, 2024

chore(k223): release 3.2.0 #14402

Open

This was referenced Oct 14, 2024

chore(k224): release 3.2.0 #14486

Open

chore(k225): release 3.2.0 #14543

Closed

This was referenced Oct 21, 2024

chore(k225): release 3.2.0 #14548

Open

chore(k226): release 3.2.0 #14625

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(helm): Various fixes and enhancements for bloom components #14128

fix(helm): Various fixes and enhancements for bloom components #14128

chaudum commented Sep 12, 2024 •

edited

Loading

diranged Sep 12, 2024

chaudum Sep 12, 2024

diranged left a comment

This comment has been minimized.

trevorwhitney left a comment

chaudum commented Sep 17, 2024

mzupan commented Sep 23, 2024

chaudum commented Sep 23, 2024

github-actions bot commented Sep 23, 2024

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

fix(helm): Various fixes and enhancements for bloom components #14128

fix(helm): Various fixes and enhancements for bloom components #14128

Conversation

chaudum commented Sep 12, 2024 • edited Loading

diranged Sep 12, 2024

Choose a reason for hiding this comment

chaudum Sep 12, 2024

Choose a reason for hiding this comment

diranged left a comment

Choose a reason for hiding this comment

This comment has been minimized.

trevorwhitney left a comment

Choose a reason for hiding this comment

chaudum commented Sep 17, 2024

mzupan commented Sep 23, 2024

chaudum commented Sep 23, 2024

github-actions bot commented Sep 23, 2024

Kubernetes Manifest Diff Summary

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

Added Files

Modified Files

Removed Files

chaudum commented Sep 12, 2024 •

edited

Loading