You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pod logs with the same timestamp are ignored by alloy, only the first one gets into Loki
Below is a screenshot of the logs from the node, which shows the same timestamp and container logs
Steps to reproduce
run any application that writes a log with the same timestamp
System information
kubernetes 1.29.5
Software version
Grafana Alloy 1.4.3
Configuration
logging {
level = "error"
format = "json"
}
discovery.kubernetes "kubernetes_pods" {
role = "pod"
}
What's wrong?
Pod logs with the same timestamp are ignored by alloy, only the first one gets into Loki
Below is a screenshot of the logs from the node, which shows the same timestamp and container logs
Steps to reproduce
run any application that writes a log with the same timestamp
System information
kubernetes 1.29.5
Software version
Grafana Alloy 1.4.3
Configuration
logging {
level = "error"
format = "json"
}
discovery.kubernetes "kubernetes_pods" {
role = "pod"
}
discovery.relabel "kubernetes_pods" {
targets = discovery.kubernetes.kubernetes_pods.targets
rule {
source_labels = ["__meta_kubernetes_pod_controller_name"]
regex = "([0-9a-z-.]+?)(-[0-9a-f]{8,10})?"
target_label = "__tmp_controller_name"
}
rule {
source_labels = ["__meta_kubernetes_pod_label_app_io_name", "__meta_kubernetes_pod_label_app", "__tmp_controller_name", "__meta_kubernetes_pod_name"]
regex = "^;([^;]+)(;.)?$"
target_label = "app"
}
rule {
source_labels = ["__meta_kubernetes_pod_label_app_io_component", "__meta_kubernetes_pod_label_component"]
regex = "^;([^;]+)(;.)?$"
target_label = "component"
}
rule {
source_labels = ["__meta_kubernetes_pod_node_name"]
target_label = "node_name"
}
rule {
source_labels = ["__meta_kubernetes_namespace"]
target_label = "namespace"
}
rule {
source_labels = ["namespace", "app"]
separator = "/"
target_label = "job"
}
rule {
source_labels = ["__meta_kubernetes_pod_name"]
target_label = "pod"
}
rule {
source_labels = ["__meta_kubernetes_pod_container_name"]
target_label = "container"
}
rule {
source_labels = ["__meta_kubernetes_pod_label_service"]
target_label = "service"
}
}
loki.source.kubernetes "kubernetes_pods" {
targets = discovery.relabel.kubernetes_pods.output
forward_to = [loki.process.pod_logs.receiver]
}
loki.process "pod_logs" {
stage.cri { }
forward_to = [loki.write.default.receiver]
}
loki.write "default" {
endpoint {
url = "http://xxx.xxx.xxx.xxx:3100/loki/api/v1/push"
}
external_labels = { kubernetes = "XXX"}
}
Logs
No response
The text was updated successfully, but these errors were encountered: