Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update getting started demo to Loki 3.0 #12723

Merged
merged 7 commits into from
Apr 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions docs/sources/get-started/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,15 @@ If you want to experiment with Loki, you can run Loki locally using the Docker C
The Docker Compose configuration instantiates the following components, each in its own container:

- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **Promtail** which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL.
- One Loki **read** component.
- One Loki **write** component.
- One Loki **read** component (Query Frontend, Querier).
- One Loki **write** component (Distributor, Ingester).
- One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)).
- **Minio** an S3-compatible object store which Loki uses to store its index and chunks.
- **Grafana** which provides visualization of the log lines captured within Loki.

{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v2.png" caption="Getting started sample application" alt="Getting started sample application">}}
{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application">}}

## Installing Loki and collecting sample logs

Expand All @@ -41,11 +42,11 @@ This quickstart assumes you are running Linux.
cd evaluate-loki
```

1. Download `loki-config.yaml`, `promtail-local-config.yaml`, and `docker-compose.yaml`:
1. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`:

```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/promtail-local-config.yaml -O promtail-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```

Expand All @@ -63,16 +64,20 @@ This quickstart assumes you are running Linux.
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-promtail-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```

1. (Optional) Verify that the Loki cluster is up and running.
- The read component returns `ready` when you point a web browser at [http://localhost:3101/ready](http://localhost:3101/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
- The write component returns `ready` when you point a web browser at [http://localhost:3102/ready](http://localhost:3102/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.

1. (Optional) Verify that Grafana Alloy is running.
- Grafana Alloy's UI can be accessed at [http://localhost:12345](http://localhost:12345).

## Viewing your logs in Grafana

Expand Down
30 changes: 30 additions & 0 deletions examples/getting-started/alloy-local-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
discovery.docker "flog_scrape" {
host = "unix:///var/run/docker.sock"
refresh_interval = "5s"
}

discovery.relabel "flog_scrape" {
targets = []

rule {
source_labels = ["__meta_docker_container_name"]
regex = "/(.*)"
target_label = "container"
}
}

loki.source.docker "flog_scrape" {
host = "unix:///var/run/docker.sock"
targets = discovery.docker.flog_scrape.targets
forward_to = [loki.write.default.receiver]
relabel_rules = discovery.relabel.flog_scrape.rules
refresh_interval = "5s"
}

loki.write "default" {
endpoint {
url = "http://gateway:3100/loki/api/v1/push"
tenant_id = "tenant1"
}
external_labels = {}
}
29 changes: 23 additions & 6 deletions examples/getting-started/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ networks:

services:
read:
image: grafana/loki:2.9.2
image: grafana/loki:3.0.0
command: "-config.file=/etc/loki/config.yaml -target=read"
ports:
- 3101:3100
Expand All @@ -27,7 +27,7 @@ services:
- loki

write:
image: grafana/loki:2.9.2
image: grafana/loki:3.0.0
command: "-config.file=/etc/loki/config.yaml -target=write"
ports:
- 3102:3100
Expand All @@ -45,12 +45,14 @@ services:
networks:
<<: *loki-dns

promtail:
image: grafana/promtail:2.9.2
alloy:
image: grafana/alloy:latest
volumes:
- ./promtail-local-config.yaml:/etc/promtail/config.yaml:ro
- ./alloy-local-config.yaml:/etc/alloy/config.alloy:ro
- /var/run/docker.sock:/var/run/docker.sock
command: -config.file=/etc/promtail/config.yaml
command: run --server.http.listen-addr=0.0.0.0:12345 --storage.path=/var/lib/alloy/data /etc/alloy/config.alloy
ports:
- 12345:12345
depends_on:
- gateway
networks:
Expand Down Expand Up @@ -118,6 +120,20 @@ services:
networks:
- loki

backend:
image: grafana/loki:3.0.0
volumes:
- ./loki-config.yaml:/etc/loki/config.yaml
ports:
- "3100"
- "7946"
command: "-config.file=/etc/loki/config.yaml -target=backend -legacy-read-mode=false"
depends_on:
- gateway
networks:
- loki


gateway:
image: nginx:latest
depends_on:
Expand Down Expand Up @@ -186,6 +202,7 @@ services:
retries: 5
networks:
- loki


flog:
image: mingrammer/flog
Expand Down
16 changes: 14 additions & 2 deletions examples/getting-started/loki-config.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
---
server:
http_listen_address: 0.0.0.0
http_listen_port: 3100

memberlist:
join_members:
- loki:7946
join_members: ["read", "write", "backend"]
dead_node_reclaim_time: 30s
gossip_to_dead_nodes_time: 15s
left_ingesters_timeout: 30s
bind_addr: ['0.0.0.0']
bind_port: 7946
gossip_interval: 2s

schema_config:
configs:
- from: 2021-08-01
Expand All @@ -16,6 +24,7 @@ schema_config:
common:
path_prefix: /loki
replication_factor: 1
compactor_address: http://backend:3100
storage:
s3:
endpoint: minio:9000
Expand All @@ -31,3 +40,6 @@ ruler:
storage:
s3:
bucketnames: loki-ruler

compactor:
working_directory: /tmp/compactor
22 changes: 0 additions & 22 deletions examples/getting-started/promtail-local-config.yaml

This file was deleted.

Loading