-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker windows mode breaks Hyper-V vEthernet (Default Switch) #1166
Comments
FYI, there is also another default adapter that Hyper-V creates that you have no control over. vEthernet (HvsiIcs) This seems new as well in the latest windows versions. |
Pretty sure I'm hitting or hit something similar to your issue, though by default mine appears to be using a broken nat setup on my machine. When I run the I'm on the latest Windows 10 Pro - 16299.15.amd64fre.rs3_release.170928-1534 These are the networks I see:
I've tried uninstalling Docker and the Container feature and reinstalling various versions but they are all broken network wise and none of the networking troubleshooting docs I've read have been able to get it back to working. |
Ok, I still don't know if I am hitting the same issue but I was finally able to get this working though it took quite of work and I have no idea which thing fixed it. :( Still on Windows 10 Pro 1709 with all the latest updates.
Turned off "Experimental features". This is very likely a separate issue, but wanted to call it out here in case someone hit this when trying out the steps I ran. #1252 Reran the various "tests" that failed before and they all seemed to work fine. |
One additional item. I reran this
Not sure if this error from the docker logs is related. Also |
I also have this problem. Enabling LCOW with 1709+ destroys DNS functionality on the "Default Switch". Unfortunately, the "nat" switch that Docker LCOW creates does not work for non-Docker based VMs either. It would be nice if both could work simultaneously. If I manually set the /etc/resolv.conf to a functioning external DNS server (e.g., 8.8.8.8) then the VM will work until it renews the DHCP and then of course it breaks again. |
I'm also having the same issue same as @AceHack. Since I'm heavy user of docker with windows containers and VM this is really anoying... hope this will be fixed soon. |
Same issue for me as well. I've removed docker for now until this issue is fixed as it otherwise breaks my other labs. |
Reproducible here. I have used LCOW for tests, and now "Default Switch" is not able to offer a DHCP with nameservers set, as the VM will only have Using Get-NetIPInterface -InterfaceAlias "vEthernet (Default Switch)" -AddressFamily IPv4 | Get-NetIPAddress | ForEach-Object { $_.IPAddress } seems to provide me with the correct address for use as DNS (and Gateway) for the VMs on the "Default Switch" |
I experienced the same symptoms. I solved the problem by:
Without these changes, the following would happen:
My setup is: Perhaps this issue is related to the fact that Hyper-V does not support Multiple NAT connections: |
@W1M0R Thanks for the post! I installed Docker on Windows and restarted to no internet connection. Similar setup to yours. |
The problem is not related to the fact Hyper-V does not support Multiple NAT connections. I can make it work very easily with a custom network and a couple tweaks in the Default Switch and the DockerNAT, mostly making the Network "external" and adding (routing) the new network IP address in their scope, but after each reboot, Docker simply UNDO all my changes, which is extremely annoying, aggravating and unacceptable! How stupid this problem is. I hate applications that force the hand on users. When I discuss it and hear from the Docker Team they claim it is for security purposes, but they ignore the fact not all Docker usage is for production environments and this "limitation" prevents developers from set their network environments accordingly with their needs. Also, no one is able to explain why MACVLAN doesn't work in Windows! In Linux it all work, flawlessly, but in Windows nothing does. I am not sure if Docker team is not competent enough to develop proper code in Windows or if they simply do not want to make it available for Windows users. Either way, it is a shame (and this is a voice of frustration speaking out loud). My apologies but I am frustrated mostly because the way changes are reversed after boot. If I change something in my setting I do not want any crappy code touching it!!!! Docker team should know better... BTW, Hyper-V is not a problem because I can create multiple instances of whatever systems I want in there and all will work with the network I set as I want without issues, and without changing it after reboot. Docker is the problem! Notwithstanding, I suggest a little "debug hack" in the "MobiLinuxVM"... Most advanced users will find a set of iptables rules and other conditions that will give you nightmares (thus you will understand why the fstab is never executed in your Linux VM, the one you thought should work exactly as a real server does). |
@jcmarchi I'm not sure why you're thinking this has anything to do with Docker, because NAT and all the other virtual switch types are part of the Microsoft Windows networking stack. Those IP addresses that are changing after each reboot, those are configured and managed by Microsoft Windows services and drivers. I think I might know the problem causing you guys these issues. Can you please reply with the output from running these four Powershell commands? Be sure to run as admin.
|
I heard that before, that is the Windows Network Layer in the way, and for a while, I accepted it because in Linux it is all easily done. However, I started "questioning" the true-truth of this answer when I start digging into the Linux VM Docker create in Hyper-V and noticed a couple of weird behavior, especially in the IPTABLES and FSTAB. Notwithstanding if I create a new VM in Hyper-V and install whatever OS I want into it I can easily and trouble free create my network as I desire, set static IPs for each VM and access (or restrict access) as I please. I can replicate the same infrastructure design (and any other I imagine) either with Hyper-V or DropBox, but not with Docker, with is a shame IMO. But, as per your request:
In this specific machine I have two NIC and one WiFi card. I am only using one NIC and the WiFi is offline. Just FYI. I am deeply curious to see where we will get with it. High hopes again. :) Thanks. |
The problem is your Windows networking configuration. Network traffic will be routed to the network interface having the lowest InterfaceMetric value. If you take a peak at your routing tables, you will notice that route metric and interface metric are summed up together and then used to determine routing. Lower values have priority. Although it may appear your routing tables are correct, because IPs are getting routed correctly but DNS is not, the Host Networking Service (HNS) also needs to configure DNS properly, and it uses InterfaceMetric for choosing routes based on priority. HNS is a brand new Windows service that works alongside WinNAT and VFP drivers, dynamically creating port forwarding rules, mapping and policy for those drivers. HNS is also responsible for the creation and management of virtual switches, address translation (NAT), IP addresses, IP pools, DNS, namespaces, endpoints, ports, filter driver policies, etc. for both Hyper-V and Docker. I think that HNS is actually supposed to be able to recognize MediaConnectionState and re-order routes appropriately when an adapter is in the disconnected or disabled state, but currently this functionality does not exist in HNS, therefore it is required that you manually set the order using InterfaceMetric values when there are multiple physical network adapters present. THE SOLUTION: By default, Windows automatically assigns InterfaceMetric values based on LinkSpeed. Gigabit Ethernet adapters are often assigned the lowest value of '5' and 802.11ac Wi-Fi adapters are often assigned a value of '35'. In this example case, if Wi-Fi is your primary internet-connected network adapter, and Ethernet is disconnected or even disabled, your containers and VMs might have direct IP address communication, but no DNS resolution, and it's likely you would get a "Timed out while waiting for Docker daemon to be ready" error when attempting to start Docker or switch to Windows Containers mode. Run the following in an elevated Powershell session to view current configuration: Then assign your primary interface ('Wi-Fi' in this example) to a lower InterfaceMetric than all others: Make sure InterfaceMetric is changed to the same low value for both IPv4 and IPv6. You can also change adapter options in Windows control panel, instead of Powershell, by un-checking "automatic metric" and entering a value in the field. Note: you should not have any NetNats set up manually, as this will break things currently. Remove all NetNats by running the following in an elevated Powershell session: Also, you will want to remove any external switches, bridges or internet connection sharing until everything else is configured and working properly. Reboot until 'Default Switch' and 'nat' are both in the 172.16.0.0/12 range (172.16.0.0 - 172.31.255.255). Sometimes it takes a couple reboots for HNS to get it right. If your system uses a 'vEthernet (HvsiIcs)' network adapter, it may not appear until after you open Microsoft Edge browser in Application Guard mode first. This adapter will most likely be assigned an IP in the 192.168.0.0/16 range (192.168.0.0 - 192.168.255.255). If HNS is struggling to get it right, or if you want to completely remove all dynamic switches and start with a fresh HNS configuration, you can do the following with elevated permissions:
Keep in mind that the HNS service does it's work dynamically and on-the-fly, so if you are browsing network switches in Hyper-V/Docker, or running Get-Net* commands in Powershell, the HNS service will be queried during these actions and may overwrite your settings, and will likely re-start the HNS and/or Docker services. I recommend completing your tasks and rebooting immediately. |
Issues go stale after 90d of inactivity. Prevent issues from auto-closing with an If this issue is safe to close now please do so. Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. |
/lifecycle frozen |
/remove-lifecycle stale |
This issue also affects Skype for Business on Windows. If the Docker adapter is enabled when Skype for Business starts up, then file transfers, video calling and screen sharing may be impacted. You might see error messages indicating timeouts such as "connection timeout", " failed to accept the invitation" and so on. If other people try to video call or share screen with you, you might not receive any notification that they are trying to do so. Disabling the docker adapter, starting Skype for Business and then re-enabling the Docker adapter fixes the issue. |
@xtremeperf Thanks for that solution. I never would have solved that in a million years. |
Is this bug being worked on at all? Its a pretty painful workaround. |
This workaround did not work for me. I am connected via a physical adapter, can ping ip addresses, but no DNS resolution from inside hyper-v VMs. Physical adapters have an InterfaceMetric that it lower than all others (excepting the DockerNAT?) |
docker swarm init worked for me |
Check for duplicate entries in here: C:\Windows\System32\drivers\etc\hosts.ics |
Expected behavior
Create Hyper-V VM and ping google.com works
Looks like Default Switch is a new built in Hyper-V natted virtual switch that is created by default now when you install Hyper-V in the latest windows version. I don't think docker is expecting this.
Actual behavior
ping google.com results in
Ping request could not find host google.com. Please check the name and try again.
Information
CB980F31-37E6-4F94-8622-E16E6ADBFA2A/2017-10-04_11-57-30
Windows 10 Enterprise Build 16299.rs3_release.170922-1354
Docker 17.09.0-ce, build afdb6d4
Steps to reproduce the behavior
I've attached some diagnostics from the script WindowsContainerNetworking-LoggingAndCleanupAide.ps1 if it offers any help.
FYI, networking inside the containers are working fine, this is just an issues in the Hyper-V VMs.
Thanks.
PreCleanupState_TNDQFBTE.zip
The text was updated successfully, but these errors were encountered: