Skip to content

Commit

Permalink
Increasing sidecar default CPU limits and memory. (#21537)
Browse files Browse the repository at this point in the history
Related to airbytehq/airbyte-cloud#3814.

These vars govern the heartbeat and socat Kube sidecars.

Testing has shown that socat requires at least 1.5 CPU to support > 20 MB/s throughput.

Bump to 2.0 CPU just in case. This is fine because CPU limits affect throttling, and not scheduling, so this change will not affect OSS users or baseline Kube deployment requirements.

Also a modest increase in default resource request for the sidecars for better reliability.
  • Loading branch information
davinchia authored Jan 18, 2023
1 parent d287928 commit 6a04fb8
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,11 @@ public class EnvConfigs implements Configs {
private static final String SIDECAR_KUBE_MEMORY_LIMIT = "SIDECAR_KUBE_MEMORY_LIMIT";
private static final String DEFAULT_SIDECAR_KUBE_CPU_REQUEST = "0.1";
private static final String SIDECAR_KUBE_CPU_REQUEST = "SIDECAR_KUBE_CPU_REQUEST";
private static final String DEFAULT_SIDECAR_KUBE_CPU_LIMIT = "0.2";
// Test show at least 1.5 CPU is required to hit >20 Mb/s. Overprovision to ensure sidecar resources
// do not cause bottlenecks.
// This is fine as the limit only affects whether the container is throttled by Kube. It does not
// affect scheduling.
private static final String DEFAULT_SIDECAR_KUBE_CPU_LIMIT = "2.0";
private static final String SIDECAR_KUBE_CPU_LIMIT = "SIDECAR_KUBE_CPU_LIMIT";
public static final String JOB_KUBE_SOCAT_IMAGE = "JOB_KUBE_SOCAT_IMAGE";
private static final String SOCAT_KUBE_CPU_LIMIT = "SOCAT_KUBE_CPU_LIMIT";
Expand Down

2 comments on commit 6a04fb8

@hakuno
Copy link

@hakuno hakuno commented on 6a04fb8 Jan 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davinchia How could I change the DEFAULT_SIDECAR_KUBE_CPU_REQUEST? Is that any environment variable for setup? Do you know?

@davinchia
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! You can configure this via this env var. Looks like docs aren't updated for this. I'll make sure they are when I come back.

Please sign in to comment.