Skip to content

Latest commit

 

History

History
140 lines (119 loc) · 4.46 KB

README.md

File metadata and controls

140 lines (119 loc) · 4.46 KB
Raw

Bull Queue Exporter

Prometheus exporter for Bull metrics.


UI

Grafana Dashboard

Setup

Prometheus

An existing prometheus server is required to use this project

To learn more about how to setup promethues and grafana see: https://eksworkshop.com/monitoring/

Grafana

The dashboard pictured above is available to download from grafana. It will work aslong as EXPORTER_STAT_PREFIX is not changed.

Queue Discovery

Queues are discovered at start up by running KEYS bull:*:id this can also be triggered manually from the /discover_queues endpoint curl -XPOST localhost:9538/discover_queues

Metrics

Metric type description
bull_queue_completed counter Total number of completed jobs
bull_queue_complete_duration summary Processing time for completed jobs
bull_queue_active counter Total number of active jobs (currently being processed)
bull_queue_delayed counter Total number of jobs that will run in the future
bull_queue_failed counter Total number of failed jobs
bull_queue_waiting counter Total number of jobs waiting to be processed
bull_queue_prioritized counter Total number of prioritized jobs

Kubernetes Usage

Environment variables for default docker image

variable default description
EXPORTER_REDIS_URL redis://localhost:6379/0 Redis uri to connect
EXPORTER_PREFIX bull prefix for queues
EXPORTER_STAT_PREFIX bull_queue_ prefix for exported metrics
EXPORTER_QUEUES - a space separated list of queues to check
EXPORTER_AUTODISCOVER - set to '0' or 'false' to disable queue discovery

Example deployment

see: k8s-sample.yaml for more options

apiVersion: apps/v1

kind: Deployment
metadata:
  name: bull-exporter
  labels:
    app: bull
    role: exporter

spec:
  selector:
    matchLabels:
      app: bull
      role: exporter
  replicas: 1
  template:
    metadata:
      labels:
        app: bull
        role: exporter
    spec:
      containers:
        - name: bull-exporter
          image: uphabit/bull_exporter:latest
          securityContext:
            runAsGroup: 65534 # nobody
            runAsUser: 65534 # nobody
            runAsNonRoot: true
            privileged: false
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - all
          resources:
            requests:
              cpu: 100m
              memory: 128M
            limits:
              cpu: 200m
              memory: 512M
          env:
              # space delimited list of queues
            - name: EXPORTER_QUEUES
              value: "mail job_one video audio"

              # find the redis service in the cluster
            - name: EXPORTER_REDIS_URL
              value: redis://redis:6379/0
---
apiVersion: v1
kind: Service
metadata:
  name: bull-exporter
  labels:
    app: bull
    role: exporter
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port: '9538'
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 9538
      targetPort: 9538
  selector:
    app: bull
    role: exporter