local execute build command fails when sidecar is present and using the latest docker version

[nanophate]

Meta:

CircleCI CLI Version:
0.1.27660+35d39ea (release)

Operating System:
macOS 13.4.1 (c) (22F770820d)

CircleCI CLI Diagnostic:

circleci diagnostic                                                                                                                     INT ✘

---
CircleCI CLI Diagnostics
---
Debugger mode: false
Config found: /Users/vivio/.circleci/cli.yml
API host: https://circleci.com
API endpoint: graphql-unstable
OK, got a token.
Trying an introspection query on API...
Ok.
Hello, Naoya Matayoshi.

---

Current behavior:

No links but it is possible to replicate the issue by creating a following config and running circleci local execute build with latest docker desktop on mac. Server Version: 24.0.2

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/ruby:3.2
      - image: cimg/redis:15.3
    steps:
      - run: echo "heelo"

Error display after command

Starting container cimg/postgres:15.3
Creating Docker containers in parallel ..Error:   error starting container cimg/postgres:15.3: Error response from daemon: invalid UTS mode: containere48ed9db8ce062238a67ce919ed3f438e2b4a6efaeab1e7bd6794193be8b3bf0

Error: 
Unexpected environment preparation error: Error response from daemon: invalid UTS mode: containere48ed9db8ce062238a67ce919ed3f438e2b4a6efaeab1e7bd6794193be8b3bf0

Step failed
Task failed
Error: Unhandled prepare executor error: Error response from daemon: invalid UTS mode: containere48ed9db8ce062238a67ce919ed3f438e2b4a6efaeab1e7bd6794193be8b3bf0

Expected behavior:

On docker version Server Version: 20.10.23 it did not show any error and was working expected.

Starting container cimg/postgres:15.3
3.2: Pulling from cimg/ruby
Digest: sha256:fa15234a551a1777d9e63d036bf1305679f7a2c0d5882810598831b0550d81aa
Status: Image is up to date for cimg/ruby:3.2
Creating Docker containers in parallel ....................................................................................
cimg/postgres:15.3:
  using image cimg/postgres@sha256:788e6d47c0413873309ad2c78a9dc7bf84d33743d26b1f617cfe28816df8a2ba
  pull stats: download 339.9MiB in 56.636s (6.001MiB/s), extract 339.8MiB in 22.896s (14.84MiB/s)
  time to create container: 765ms
Time to upload agent and config: 3.285449024s
Time to start containers: 849.631467ms
====>> Container cimg/postgres:15.3
====>> Preparing environment variables
Using build environment variables:
  BASH_ENV=/tmp/.bash_env-localbuild-1689921565
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=
  CIRCLE_BUILD_NUM=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  CIRCLE_REPOSITORY_URL=
  CIRCLE_SHA1=
  CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1689921565
  CIRCLE_WORKING_DIRECTORY=~/project


The redacted variables listed above will be masked in run step output.====>> echo "heelo"
  #!/bin/bash -eo pipefail
echo "heelo"
heelo
Success!
Step canceled

When did this begin / Was this previously working?:

After upgrading to docker 24 the issue started to happen. docker 20 did not have this issue.

Additional Information:

[cameroncaci]

This is pretty stale, but is an issue I encountered as well when attempting to get CircleCI to use 2 images on the same executor on MacOS with Docker version 26. Similar to what andyatkinson/rideshare#99 found, I was able to get a workaround by using the CircleCI setup_remote_docker attribute and setting it to the now deprecated 20.10.24, as well as creating a Docker in Docker setup for my local pipeline.

For example, if you wanted an executor with 2 images such as cimg/ruby and cimg/postgres, the workaround I found was to have one image be the "host" of a docker-in-docker network, and instead of the host container running both images in parallel, it would instead create the second image as a docker container inside of itself. The following is a sample code output to describe what I'm mentioning

version: 2.1

references:
  ruby-docker: &ruby-docker cimg/ruby:3.2
  # cimg/postgres would've gone here and placed within the executor
  # but instead we'll create it within a job step

executors:
  ruby_exector:
    docker:
      - image: *ruby-docker
jobs:
  test:
    executor: ruby_exector
  steps:
      - setup_remote_docker:
          version: 20.10.24
      - run:
          name: Start new docker network
          command: sudo docker network create my_docker_network
      - run:
          name: Join the host container on the network
          command: sudo docker network connect my_docker_network $(hostname)
      - run:
          name: Start PostgreSQL remote container
          command: sudo docker run --network my_docker_network --network-alias postgres_db_container --rm -d --name setup_tests_postgres_container -e POSTGRES_PASSWORD=mysecretpassword -e POSTGRES_DB=test_db -v /dev/shm/pgdata:/var/lib/postgresql/data cimg/postgres:12.11 -c 'listen_addresses=*'
      - run:
          name: Get PostgreSQL container IP address
          command: |
            export DB_HOST=$(sudo docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' setup_tests_postgres_container)
      - run:
          name: Wait for PostgreSQL to be ready on remote test container
          command: |
            until sudo docker exec setup_tests_postgres_container pg_isready -h ${DB_HOST} -p 5432 -U postgres; do
              echo "Waiting for PostgreSQL on ${DB_HOST}:5432..."
              sleep 2
              # Heads up there's no timeout here in this sample
            done
      - run:
          name: Stop PostgreSQL remote container
          command: sudo docker stop setup_tests_postgres_container
          when: always
      - run:
          name: Disconnect host from docker network
          command: sudo docker network disconnect my_docker_network $(hostname)
          when: always
      - run:
          name: Stop docker network
          command: sudo docker network rm my_docker_network
          when: always

I did encounter some local CircleCI workspace and Docker volume mounting issues with this approach, though that could've been due to user error. The workaround for these issues was to feed files between the docker containers through input streams with docker run -i allowing me to use standard input for file transfer.