Failed to compute cache key in newer version

[DRoet]

This is a docker issue but it seems to be related to BuildKit only. this is something that was still working in docker ~19.03.10 but stopped functioning in 20.10.0+. I managed to bring down my DockerFile to a minimal repro:

This works (A.DockerFile):

FROM php:7.4.13-cli

COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer

This also works (B.DockerFile):

FROM php:7.4.13-cli

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/

This no longer works (C.DockerFile):

FROM php:7.4.13-cli

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer

Output from running A and C after eachother:

C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f A.Dockerfile .
[+] Building 3.6s (7/7) FINISHED
 => [internal] load build definition from A.Dockerfile                                                                                                                                                           0.0s
 => => transferring dockerfile: 132B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                2.9s
 => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
 => => resolve docker.io/library/composer:2.0.8                                                                                                                                                                  0.5s
 => CACHED [stage-0 1/2] FROM docker.io/library/php:7.4.13-cli@sha256:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                           0.0s
 => [stage-0 2/2] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                           0.1s
 => exporting to image                                                                                                                                                                                           0.1s
 => => exporting layers                                                                                                                                                                                          0.0s
 => => writing image sha256:ea6d75bc9ad24e800c8083e9ea6b7774f2bd9610cb0e61b3640058c9c7fe34c6                                                                                                                     0.0s

C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f C.Dockerfile .
[+] Building 1.0s (8/8) FINISHED
 => [internal] load build definition from C.Dockerfile                                                                                                                                                           0.0s
 => => transferring dockerfile: 221B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                0.2s
 => FROM docker.io/mlocati/php-extension-installer:latest                                                                                                                                                        0.0s
 => => resolve docker.io/mlocati/php-extension-installer:latest                                                                                                                                                  0.0s
 => => sha256:ccf3a05d8241580ad9d2a6c884a735bb248e90942ab23e0f8197f851a999ddac 526B / 526B                                                                                                                       0.0s
 => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
 => [stage-0 1/3] FROM docker.io/library/php:7.4.13-cli@sha256:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                                  0.0s
 => CACHED [stage-0 2/3] COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/                                                                                                   0.0s
 => ERROR [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                     0.0s
------
 > [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer:
------
failed to compute cache key: "/usr/bin/composer" not found: not found

This doesn't happen consistently in my build, sometimes everything builds fine and there are no issues.
I'm using windows 10 (20H2) and the latest version of Docker Desktop that includes Docker version 20.10.2, build 2291f61, but I have also seen this happen on Linux with the same version

[yogevyuval]

Expiriencing the same thing, any update?

[adrubesh]

Same issue here. Experiencing the issue using both docker-compose build and docker compose build. Disabling buildkit for docker compose (COMPOSE_DOCKER_CLI_BUILD=0) works for a temporary workaround for now.

[wizu]

Same issue with COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/ and BuildKit

[max-sixty]

Here's another example:

FROM python:3.8-slim-buster as base

COPY --from=hairyhenderson/gomplate:v3.8.0 /gomplate /usr/local/bin/gomplate
COPY --from=bitnami/kubectl:1.20 /opt/bitnami/kubectl/bin/kubectl /usr/local/bin/kubectl
COPY --from=argoproj/argocli:v2.12.8 /bin/argo /usr/local/bin/argo

This seems to work with two but not three of them (!). As above, only on buildkit.

And it works if those images are pulled separately prior to building. If those images are pulled prior to building, the build reports as successful, but some of the files are not copied in. The inconsistent silent failure is not ideal...

[summera]

Also experiencing this issue with buildkit. Without buildkit, builds fine.

[matti]

when will this fix land in docker for mac?

[matti]

docker for mac update check says that 3.1.0 is the latest, but the latest is 3.2.0

[DRoet]

Docker for mac 3.2.0 has docker engine v20.10.3, the fix for this issue is included with docker v20.10.4 so you'll need to wait until the next release.

[newtorob]

I'm on 20.10.5 on mac and still running into this issue.

[carlydf]

I'm on 20.10.5 on mac and still running into this issue.

same here

[DRoet]

Seems to be resolved for me on both Windows/Linux. Maybe create a new issue with an example if you are still having issues.

[alexdruhet]

I'm on 20.10.5 on mac and still running into this issue.

I'm experiencing the exact same issue

[Plasma]

A brand new VS solution today breaks with "not found" in the path.

[emma-campbell]

I'm on 20.10.5 on mac and still running into this issue.

I'm in the same boat -- running into the same issue.

[RedactedProfile]

On Windows, 20.10.5 and encountering this issue

[barkurguru]

Hello

I am also facing the same issue here.. Is there any fix for this please..

---

[2/2] COPY ./target/employee-jdbc-0.0.1-SNAPSHOT.jar .:

---

failed to compute cache key: "/target/employee-jdbc-0.0.1-SNAPSHOT.jar" not found: not found

Thanks
Guru

[yogevyuval]

@thaJeztah I see this issue is closed but many people are still expiriencing the issue. It is because the fix is not rolled out yet, or is it because the problem is still not solved?

[Plasma]

I'm not sure if this fixes it for you: https://stackoverflow.com/a/66611529/554037

Try running:

docker build -f PROJECT_DIR\Dockerfile .

[jonyhy96]

Disable buildkit works for me.
Try

DOCKER_BUILDKIT=0 docker build ...

[thehim]

I'm using version 20.10.5 of the Docker Engine on a Windows machine and I'm seeing this error using a very basic Dockerfile. Any updates?

[MarcoMartins86]

Disable buildkit works for me.
Try

DOCKER_BUILDKIT=0 docker build ...

This flag helped me to get the real error and then I solved it from there when using docker-compose. With the original error, I would not reach this fast to the cause of it.
With flag (improved error):

COPY ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json /tmp/agents_mapping.json
COPY failed: forbidden path outside the build context: ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json ()

Without flag (original error):

COPY ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json /tmp/agents_mapping.json:
failed to compute cache key: "/server/src/test/resources/elasticsearch/agents/agents_mapping.json" not found: not found

[uplinkgit]

Windows 20.10.5 users, here is how I fixed it:

1. Locate daemon.json (should be in c:\Users\CURRENT_USER\.docker folder).
2. Add the following property: "features": {"buildkit": false}. It should look something like:

{
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": true,
  "experimental": false,
  "features": {"buildkit": false}
}

Cheers.

[BrendonW]

It looks like Docker for [Mac, Windows] have some crippling regressions in buildkit. As far as I can tell, they are using a build version greater than this fix and also #1159 which ends up making it very difficult to build anything non-trivial using caching.

[gwynnarth]

Docker Desktop on Mac v3.3.0 (62916), Engine: 20.10.5 still has this issue.
Only thing that helps is disabling Buildkit... This issue should be reopened, IMHO.

[JonasAgger]

Docker Desktop on windows v.3.3.1 (63152), Engine 20.10.5 also still has this issue

[DRoet]

I don't have this issue anymore with my original reproduction above, so I would recommend creating a new issue.

[mratchkauskas]

Any update on this, it is happening on my 20.10.6 (OSX) version as well...

[kbrown]

still a bug

[miquelvalverdep]

In my case it "failed to walk" because I was copying resources from outside the root folder (where we build the Dockerfile).
Solved it by contextualizing the parent folder of the root inside docker-compose.yml:

        build:
            context: ../

[DataScientistSamChan]

still a bug on mac catalina v10.15.7

[simibac]

Make sure that the file that you are trying to copy is not part of the .dockerignore file.

[chris-cardwell]

@simibac thanks, resolved my issue!

[voronin-de]

I'm not sure if this fixes it for you: https://stackoverflow.com/a/66611529/554037

Try running:

docker build -f PROJECT_DIR\Dockerfile .

This actually worked!

I had the following structure:

|
| - persistence
| --> changelogs
| - docker
| --> Dockerfile

Running on Ubuntu 20.04LTS on WSL on Windows 10 (20H2), the following command got the build done docker build -f docker/Dockerfile .. The Dockerfile contained the following line, that was previously breaking the build: COPY persistence/changelogs/ /liquibase/changelogs/.
Executing the command above from the root directory (in my example above) worked.

[ryan-williams]

Echoing #1939 (comment) and #1939 (comment) above:

1. I saw this error because I was trying to COPY a file that was .dockerignore'd
2. it seems like the error message is much clearer about this when DOCKER_BUILDKIT=0; arguably this is a (minor) regression in buildkit.

With buildkit:

…
 => ERROR [2/3] COPY ./test.py /test.py                                                         0.0s
------
 > [2/3] COPY ./test.py /test.py:
------
failed to compute cache key: "/test.py" not found: not found

Without buildkit:

…
Step 2/4 : COPY ./test.py /test.py
COPY failed: file not found in build context or excluded by .dockerignore: stat test.py: file does not exist

[thaJeztah]

it seems like the error message is much clearer about this when DOCKER_BUILDKIT=0; arguably this is a (minor) regression in buildkit.

Agreed, I created #1647 (and somewhat related, #2130) for that.

[ghost]

In my case issue was file was in the .dockerignore. Removing it from there worked

[zmrfzn]

DOCKER_BUILDKIT=0 gave a clear message.

Docker for desktop 3.5.2(66501) on windows 10 still has this issue.

[tiago-marques]

Make sure that the file that you are trying to copy is not part of the .dockerignore file.

You're genius !

[schematical]

In my case issue was file was in the .dockerignore. Removing it from there worked

100% this was the case for me. Thank you!

[hannah23280]

After lot of struggling, able to get it work. Not because of .dockerignore (i don't have this file), but cos of the slash

Not working
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .

Working
docker image build --build-arg JAR_FILE="build/libs/licensing-service-0.0.1-SNAPSHOT.jar" .

Do note that my host machine is windows, 10 pro.

Why would / works, and not \ , until now its still a mystery to me.

[ksuedile]

After lot of struggling, able to get it work. Not because of .dockerignore (i don't have this file), but cos of the slash

Not working
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .

Working
docker image build --build-arg JAR_FILE="build/libs/licensing-service-0.0.1-SNAPSHOT.jar" .

Do note that my host machine is windows, 10 pro.

Why would / works, and not \ , until now its still a mystery to me.

@hanct probably because your base image (FROM ...) of your Dockerfile is unix-based. And on unix systems back-slashes ( \ ) is not the norm. That is a fundamental difference between windows systems and unix systems. So just adapt your dockerfile so it can reach sub folders using regular slashes

[voronin-de]

After lot of struggling, able to get it work. Not because of .dockerignore (i don't have this file), but cos of the slash
Not working
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
Working
docker image build --build-arg JAR_FILE="build/libs/licensing-service-0.0.1-SNAPSHOT.jar" .
Do note that my host machine is windows, 10 pro.
Why would / works, and not \ , until now its still a mystery to me.

@hanct probably because your base image (FROM ...) of your Dockerfile is unix-based. And on unix systems back-slashes ( \ ) is not the norm. That is a fundamental difference between windows systems and unix systems. So just adapt your dockerfile so it can reach sub folders using regular slashes

Another reason might be (which is what I thought when I was trying something similar, albeit when mounting volumes), that backslashes are usually used to escape the following character, which is why it might have worked if you used two backslashes (build\\libs\\licensing-service-0.0.1-SNAPSHOT.jar).

However what @ksuedile says is definitely correct: on unix-based machines path levels are usually separated by a slash (/) rather than a backslash (\) and I'd guess you are using some sort of Linux distro as a basic layer for the image. Perhaps if there was a way to use a Windows system (some version of Windows Core?), this might've worked (I am not sure there's any version of Windows to serve as a base for a container - from what I read somewhere even Microsoft uses Linux in these cases).

[hannah23280]

After lot of struggling, able to get it work. Not because of .dockerignore (i don't have this file), but cos of the slash
Not working
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
docker image build --build-arg JAR_FILE="build\libs\licensing-service-0.0.1-SNAPSHOT.jar" .
Working
docker image build --build-arg JAR_FILE="build/libs/licensing-service-0.0.1-SNAPSHOT.jar" .
Do note that my host machine is windows, 10 pro.
Why would / works, and not \ , until now its still a mystery to me.

@hanct probably because your base image (FROM ...) of your Dockerfile is unix-based. And on unix systems back-slashes ( \ ) is not the norm. That is a fundamental difference between windows systems and unix systems. So just adapt your dockerfile so it can reach sub folders using regular slashes

Another reason might be (which is what I thought when I was trying something similar, albeit when mounting volumes), that backslashes are usually used to escape the following character, which is why it might have worked if you used two backslashes (build\\libs\\licensing-service-0.0.1-SNAPSHOT.jar).

However what @ksuedile says is definitely correct: on unix-based machines path levels are usually separated by a slash (/) rather than a backslash (\) and I'd guess you are using some sort of Linux distro as a basic layer for the image. Perhaps if there was a way to use a Windows system (some version of Windows Core?), this might've worked (I am not sure there's any version of Windows to serve as a base for a container - from what I read somewhere even Microsoft uses Linux in these cases).

Yes. I am using linux distro as basic layer.
Yes. I think there is windows version serving as base for a container. Nanoserver is one of them.

The reason why i initially thought backslash can be used, is because I am running these docker commands using windows command prompt of my host machine (which is window), and hence never thought that the slash in these commands must follow the container's base layer OS, not the host OS

[HackerTheMonkey]

Make sure that the file that you are trying to copy is not part of the .dockerignore file.

You're genius !

On Mac and this did it for me 👌

[peymanebrahimi]

Make sure that the file that you are trying to copy is not part of the .dockerignore file.

I removed **/bin from .dockerignore and worked for me.

FROM mcr.microsoft.com/dotnet/aspnet:5.0

COPY ["ChatApp/bin/Release/net5.0/publish", "App/"]

WORKDIR /App
EXPOSE 80
EXPOSE 443
ENTRYPOINT ["dotnet", "ChatApp.dll"]

[cjishnu]

On the Mac, keeping absolute paths made it work for me:

cd /absolute/path/to/project/folder
docker build -f /absolute/path/to/project/folder/Dockerfile -t name:tag —platform=linux/amd64 /absolute/path/to/project/folder

MacOS Big Sur 11.5.2 Apple M1

[mohittalele]

On Mac Big Sur 11.2.3 , deleting the dockerignore solved the problem.

[tmerten]

On Mac Big Sur 11.2.3 , deleting the dockerignore solved the problem.

I did not really want to try this ;) but works here, too (of course it introduces other problems). Absolute paths did not make any change though.

[Jijii]

Creating an empty .dockerignore worked for me

[mhtsharma948]

I'm on MacOS docker engine v20.10.11 I Tried the following, but nothing helps.

1. Removing .dockerignore
2. Emptying .dockerignore
3. using DOCKER_BUILDKIT=0 docker build...

Can anyone tell how to fix it or on which version, it was working perfectly?
[NOTE: The same script it working on windows but not on linux and mac.]

error with BUILDKIT=0:
COPY failed: stat usr/src/app/dist: file does not exist
error with BUILDKIT=1:
failed to compute cache key: "/usr/src/app/dist" not found: not found

[thaJeztah]

@mhtsharma948 paths in Dockerfiles are relative to, and must be within the build-context (if you run docker build <options> . then the . at the end means the current directory (.) is used as build-context. If your Dockerfile tries to COPY /usr/src/app/dist ..., that means a subdirectoryusr/src/app/dis must exist in your current directory; if not, it will fail.

From your output I suspect you were trying to use an absolute path on your computer, which is not supported, and is not a bug.

If you (and others on this thread) think there's a bug at hand, and have exact steps to reproduce, pleas open a new ticket instead with details and steps to reproduce.

[thaJeztah]

I am locking the conversation on this ticket, because it's collecting many comments that are unrelated to the original issue, but wrote down some pointers in another ticket that may help understanding the error messages that can happen when performing a build; docker/buildx#415 (comment)