Skip to content

Latest commit

 

History

History
74 lines (73 loc) · 4.38 KB

README.md

File metadata and controls

74 lines (73 loc) · 4.38 KB
Raw

Lifting bitbucket pipelines runner limitations

The scripts and services in this repository are intended for lifting the resource limitations imposed on self-hosted Bitbucket Pipelines Linux Docker runners.

Details

As of 2024, Bitbucket Cloud infrastructure runners have a configurable CPU count and memory amount via the size option, whereas self-hosted Linux Docker runners are fixed to 4 CPU's:

size Bitbucket Cloud Self-hosted Lifted
1x 4 vCPU, 4 GB 4 CPU, 4 GB ∞ CPU, 1 TB *
2x 4 vCPU, 8 GB 4 CPU, 8 GB ∞ CPU, 1 TB *
4x 8 vCPU, 16 GB 4 CPU, 16 GB ∞ CPU, 1 TB *
8x 16 vCPU, 32 GB 4 CPU, 32 GB ∞ CPU, 1 TB *

* Limited by the capabilities of the host.

Cost example

With 1000 Build minutes on Bitbucket Cloud infrastructure being sold for $10 and multiplied with size, this can lead to huge costs. Consider the following cost example assuming one runner being busy for one month runtime:

Hetzner Cloud self-hosted Bitbucket Cloud infrastructure
~$5 (CX22) $432 (1x)
~$10 (CX32) $864 (2x)
~$20 (CX42) $1728 (4x)
~$40 (CX52) $3456 (8x)

Setup

crictl as container runtime (e. g. using hetzner-k3s and runners-autoscaler)

hetzner-k3s_cluster_config.yaml

...
  post_create_commands:
  - git clone https://github.com/tramseyer/bitbucket-pipelines-runner-unlimited.git
  - cd bitbucket-pipelines-runner-unlimited
  - ./setup.sh crictl
...

docker as container runtime (e. g. using a dedicated Linux Docker runner)

[email protected]:tramseyer/bitbucket-pipelines-runner-unlimited.git
cd bitbucket-pipelines-runner-unlimited
./setup.sh docker

kubernetes (e. g. using Docker-based runner on Kubernetes or Autoscaler for Runners on Kubernetes)

kustomize/base/cm-job-template.yaml / config/runners-autoscaler-cm-job.template.yaml

...
                - name: runner
                  ...
                - name: docker
                  ...
                - name: docker-cpu-quota
                  image: docker:cli
                  command: ["/bin/sh", "-c", "wget -qO- https://raw.githubusercontent.com/tramseyer/bitbucket-pipelines-runner-unlimited/master/docker-cpu-quota.sh | sh"]
                  volumeMounts:
                    - name: var-run
                      mountPath: /var/run
                - name: docker-memory-limit
                  image: docker:cli
                  command: ["/bin/sh", "-c", "wget -qO- https://raw.githubusercontent.com/tramseyer/bitbucket-pipelines-runner-unlimited/master/docker-memory-limit.sh | sh"]
                  volumeMounts:
                    - name: var-run
                      mountPath: /var/run
...

Monitoring of services

on kubernetes node

systemctl status --no-pager crictl-{cpu-quota,memory-limit}

on dedicated Linux Docker runner

systemctl status --no-pager docker-{cpu-quota,memory-limit}

Links