Docker fails to launch on Apple Silicon

[0xTim]

Fails to launch on Apple Silicon (unsurprisingly given it's so new). Can't see a similar issue so raising here to track.

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID: Unable to capture diagnostics

Expected behavior

Docker for Mac installs successfully

Actual behavior

Information

Appreciate that it probably isn't expected to work until the fixes to the engine are in, but thought it would be worth having an issue to track anyway.

• macOS Version: 11.0

[stephen-turner]

Thanks. We're working closely on this with Apple. This is expected right now (and indeed there's nothing we can do about it yet) as the new chips don't have virtualisation support yet.

[codercotton]

Are you able to release anything now that today's Big Sur update has the Virtualization Framework available?

https://developer.apple.com/documentation/virtualization

[stephen-turner]

Not yet. The OS has the virtualisation functions, but the Apple chips don't have virtualisation support yet.

[kobaltz]

I don't know too much about the A12Z, but when I run sysctl, it does show VMX listed in the features. I wonder if this means that the A12Z does have some sort of virtualization support, but perhaps it's just not accessible yet?

➜  ~ sysctl -a | grep machdep.cpu.features

machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA
 CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3
 PCLMULQDQ DTSE64 MON DSCPL VMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2
 AES SEGLIM64

[TC72]

This page says Hypervisor.framework is not supported on the DTK

https://twtext.com/article/1279716564187729927

[gongzhang]

I'm interested in how Docker will be implemented on Apple Silicon. Will it support x86 images? Or just support arm images?

[stephen-turner]

ARM images by default, though you should be able to run x86 images using qemu just like you can run ARM images on x86 machines today.

[julianschiavo]

Was wondering if a fix would be expected to be available on current hardware (the Developer Transition Kit) or if it will not be available until new hardware is shipping from Apple?

(Basically, will the DTK ever support Docker?)

[stephen-turner]

The DTK will never support Docker. Virtualisation needs new hardware not just new OS.

[julianschiavo]

The DTK will never support Docker. Virtualisation needs new hardware not just new OS.

Ahh okay, thanks

[joekendal]

what? did Apple tell us this 😱

[saagarjha]

Apple promised this for their consumer hardware, not the DTK. Future chips will support virtualization and presumably Docker will be able to use that.

[Lutzifer]

@saagarjha Just out of interest: Is there any publicly available source for the promise on hardware virtualization support that you can give? have been looking for one some time ago but did not find one.

[saagarjha]

I'm not quite sure what you mean? Apple has shown off that Apple silicon will do virtualization, and there is documentation for it already so developers can get an idea of what'll be shipping later. It's just that they weren't able to put the feature in the A12Z chip that powers the DTK, which they reused from iPad without modification. Presumably they figured this particular thing was not one of the hardware features specific to macOS they needed to ship early but keep disabled on iOS.

[Lutzifer]

@saagarjha thx, that link helped a lot.

[djensen47]

Are we getting Docker support on Apple Silicon when the new hardware launches next week? If not, what's the timeline?

It's hard to tell based on this thread but I'm guessing the Docker team knows? Also, I apologize in advance if this info is announced elsewhere.

Thanks!

[stephen-turner]

Sorry, we don't have a date yet.

[svoop]

@djensen47 More important than a date is to know whether the Apple Silicon chips in the first batch of hardware support (sufficient) virtualization for Docker to work at all.

[woachk]

@svoop Yes, the A14 present in the iPhone this year has virtualisation support in hardware, I wouldn't worry about it.

[hazcod]

@woachk how did you check that? Very curious why they would leave that enabled in an iOS chip.

[woachk]

@hazcod Via undisclosed methods. :)

iOS has later in boot an explicit in the kernel to expose hv_support as off in sysctl (and as such disable virtualization being exposed to user space) if the CPU core part number is 32 or 33. If there wasn't that one, I'd have have demo'd virtual machines running on an iPhone by now.

[saagarjha]

One easy way to check is to disassemble initialization code from start_first_cpu, which on A14 devices touches *_EL2 registers:

$ otool -xv kernel 2> /dev/null | grep -A 10 OSLAR_EL1 | head
fffffff008288400	msr	OSLAR_EL1, xzr
fffffff008288404	msr	DAIFSet, #0xf
fffffff008288408	mrs	x0, CurrentEL
fffffff00828840c	cmp	x0, #0x8
fffffff008288410	b.ne	0xfffffff008288428
fffffff008288414	mrs	x0, HCR_EL2
fffffff008288418	orr	x0, x0, #0x8000000
fffffff00828841c	orr	x0, x0, #0x400000000
fffffff008288420	msr	HCR_EL2, x0
fffffff008288424	msr	CNTVOFF_EL2, xzr

[nebuk89]

Please upvote the public roadmap issue as well :) docker/roadmap#142

[olafgeibig]

Five years ago I changed from Linux to Mac and I did not regret that move. Rocksolid OS, ability to run all the dev tools, docker, etc. that I was used to, combined with the ability to run standard desktop software like MS word. Docker has a very central place in my daily workflows. Very often I don’t install a software but rather write a script that runs this software with docker instead. In our project we have a local development workflow where we run the services with minikube on our Mac workstations. But in the future that seems to be not possible any longer as the CI server builds the Docker images for Intel. OK, there is the upcoming virtualization but I don’t think I’ll be able to run that many pods on my local minikube as before. Using ARM containers for local development would leave an odd feeling because of using different images with a different CPU architecture. What do you think about such a setup? Will it still be feasible with an ARM Mac?

[CpuID]

@olafgeibig I've wondered about the same thing, how workflows will work between local development, CI and staging/production environments etc with Intel/AMD vs ARM architectures being interchanged regularly...

[markjeffries]

We're just about to buy 2 more macbook pros for developers, our dev, CI, staging and production stacks relies heavily on docker. It just seems too risky to buy an ARM Mac right now if you're using docker.

[peterlozano]

Any news on the virtualization support now that the M1 has been unveiled?

[FernandoMiguel]

@olafgeibig the out of the box option is that your prod env also runs on ARM, and you get to benefit from the cost reduction and performance gains

[graycrow]

Let's keep this discussion clear of "smart" advises about which CPU architecture should or should not be used for development.

[xhafan]

My 2c. I'm doing C# development on Windows without docker (too slow to deal with docker locally in my opinion). CI builds Linux docker x86-64 image based on .NET Core, runs unit tests, integration tests and automation tests against the docker image, and deploys to staging environment (AWS ECS). Production deployment (AWS ECS) is done manually by clicking a button. Due to high code coverage by tests, I'm quite confident that what I produce on Windows will work on Linux. I know sometimes there might be some very small issues from time to time, which I then fix locally using Linux docker image. But generally I have no issues at all with this dev setup, I've been doing it like this for about 2 years.

Now, I can easily imagine having this new Mac with Apple Silicon M1 ARM CPU, doing C# development, executing my app locally without docker, when happy committing the code and the CI setup above takes over. I would have Linux docker ARM image based on .NET Core, would fix the small issues in there, and if it really happens something's still broken on Linux docker x86-64 image, well, if I cannot run x86-64 docker images on Apple Silicon ARM CPU in emulation, I would have to remote somewhere - for instance temp AWS EC2 dev x86-64 virtual machine running only when needed.

Basically, due to practising TDD, thus having code covered by tests, I'm pretty confident I would be happy with this setup.

[tuapuikia]

Five years ago I changed from Linux to Mac and I did not regret that move. Rocksolid OS, ability to run all the dev tools, docker, etc. that I was used to, combined with the ability to run standard desktop software like MS word. Docker has a very central place in my daily workflows. Very often I don’t install a software but rather write a script that runs this software with docker instead. In our project we have a local development workflow where we run the services with minikube on our Mac workstations. But in the future that seems to be not possible any longer as the CI server builds the Docker images for Intel. OK, there is the upcoming virtualization but I don’t think I’ll be able to run that many pods on my local minikube as before. Using ARM containers for local development would leave an odd feeling because of using different images with a different CPU architecture. What do you think about such a setup? Will it still be feasible with an ARM Mac?

You can always change back to Linux. 😉

[justinjdickow]

I regret subscribing to this issue lol 👋

[ranneyd]

I regret subscribing to this issue lol 👋

Just in case you mean this literally and not as a commentary

[hakman]

I regret subscribing to this issue lol 👋

I get your pain. I would be interested in the updates, but not in another pointless debate.
@nebuk89 or someone else from Docker, would you mind locking this issue and just post updates when available?

[ranneyd]

I regret subscribing to this issue lol 👋

I get your pain. I would be interested in the updates, but not in another pointless debate.
@nebuk89 or someone else from Docker, would you mind locking this issue and just post updates when available?

I get your point, but a few comments:

1. I personally, mostly, subscribed to this thread to see what the community's thoughts on this M1 thing are and whether or not I was the only one scratching my head about it.
2. Though someone said the M1 chip supports virtualization others said it didn't. Can anyone confirm/deny? Because if it doesn't, there aren't really any "updates" that will accomplish anything other than a hardware update next year, right?

[woachk]

@ranneyd There's quite some disinformation in that thread, which is normal given the fear of new things that people have.

About Apple M1, yes, virtualization is supported. It wasn't just for the developer transition kits.

See https://www.parallels.com/blogs/parallels-desktop-apple-silicon-mac/ for example for details on Parallels starting the public beta process for their software on Apple M1 Macs.

[stephen-turner]

I think this thread got busy because it was posted on HackerNews. But you're right, it's got off topic. I'll lock it until Monday.

[ctwise]

Also keep in mind that using qemu to run x86 containers on ARM will waste at least 75% of the CPU on the emulation itself. That's about the performance penalty I noticed on using qemu to run ARM containers on x86, and I don't expect that vice-versa is any faster. Hence docker based development ends up even slower than a 8 year old macbook..

why would you run x86 images on your ARM Mac? there are plenty of ARM images out there for most common tools

Because work. Because you're developing for release on x86 servers. Because you're building x86 images that will be deployed on x86 servers.

[jdilley]

looks like it's not so bad?
https://www.macrumors.com/2020/11/15/m1-chip-emulating-x86-benchmark/

[piper41955]

We won't really know for some time. AFAIK, Rosetta2 will not address the Docker issue - just directly installed software.

[stephen-turner]

Correct, you can't virtualisation under Rosetta.

[jamesez]

Because work. Because you're developing for release on x86 servers. Because you're building x86 images that will be deployed on x86 servers.

Even with "x86 everywhere" there are still some rough edges: I have a --disable-march-tune-native on some libraries because they break when the pod lands on different AWS CPUs.

Ages ago we used to develop on PPC/M68K Macs and x86 PCs and deploy to UltraSPARC and Apollo and RS6000 servers.

[gpetrov]

First launch on Apple Mac Mini with M1:

Both Docker Stable and Edge - quit with this error. Any ideas?

[homerjam]

https://www.docker.com/blog/apple-silicon-m1-chips-and-docker/

[StefanScherer]

@gpetrov Docker Desktop isn't compatible with Apple Silicon processors at the moment. As a first step we'll improve the error message with a link to https://docs.docker.com/go/apple-silicon/ where we will update content once we start shipping a tech preview. Stay tuned :-)

[rene-mueller]

@StefanScherer can you say how long it will take approximately?

[McPringle]

Please: This is not a chat! Everyone involved will get a message every time someone adds a comment. Please, only add valuable comments that help to resolve this issue. Thank you very much from all subscribers!

[rene-mueller]

@McPringle of course it is not a chat. nevertheless it is important to know how long the update takes!

[stephen-turner]

OK, I think it's time to lock this thread again.

To summarise:
We are working on it, but we don't have an ETA as we have external dependencies and we don't know their timelines. Remember everybody is only getting their hands on the new hardware today.
Our blog post at https://www.docker.com/blog/apple-silicon-m1-chips-and-docker/ contains more details.

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[stephen-turner]

Sorry, we forgot to update this ticket when we issued the dev preview. Hopefully everyone's seen it by now. If not, see https://docs.docker.com/docker-for-mac/apple-m1/.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked