Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hung downloading ISO (network contention with image pull) #4035

Closed
bcbrockway opened this issue Apr 2, 2019 · 9 comments · Fixed by #4141
Closed

Hung downloading ISO (network contention with image pull) #4035

bcbrockway opened this issue Apr 2, 2019 · 9 comments · Fixed by #4141
Assignees
Labels
area/networking networking issues kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone

Comments

@bcbrockway
Copy link

My minikube hangs when downloading the ISO using Minikube 1.0.0:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64
chmod +x minikube
sudo mv minikube /usr/local/bin
minikube start                                                        
😄  minikube v1.0.0 on linux (amd64)
🤹  Downloading Kubernetes v1.14.0 images in the background ...
🔥  Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
💿  Downloading Minikube ISO ...
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
2019/04/02 11:03:47 No matching credentials were found, falling back on anonymous
 1.11 MB / 142.88 MB [>------------------------------------------]   0.78% 2m33s
  • OS: Ubuntu 18.04
  • Minikube: 1.0.0

Tried with v0.35.0 and I do not have this issue:

sudo cp ~/Downloads/minikube-linux-amd64-0.35.0 /usr/local/bin/minikube
minikube start
😄  minikube v0.35.0 on linux (amd64)
💥  Kubernetes downgrade is not supported, will continue to use v1.14.0
🔥  Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
💿  Downloading Minikube ISO ...
 26.91 MB / 184.42 MB [======>-----------------------------------]  14.59% 2m21s
@jonpharry
Copy link

jonpharry commented Apr 2, 2019

I have the same issue with minikube 1.0.0 running under kvm2 on Centos 7.

I also run without issue using 0.35 (albeit with 0.31 docker-machine-kvm2 to avoid hidden config issue) so this appears to be a 1.0.0 issue.

My Centos 7 is a VM running under VMWare Fusion on Mac (in case that makes any difference).

@jonpharry
Copy link

I was able to work around by manually downloading https://storage.googleapis.com/minikube/iso/minikube-v1.0.0.iso and placing in ~/.minikube/cache/iso

All of the other downloads required during initial startup worked fine. It appears issue is limited to the initial download of the minikube ISO.

@itamarst
Copy link

itamarst commented Apr 3, 2019

This is happening to me on Fedora 29.

@lucasyvas
Copy link

Happening on Ubuntu 18.04 on kvm2. There is no way to get it to work I can find... totally busted.

@tstromberg tstromberg changed the title Minikube hangs when downloading ISO Minikube hangs when downloading ISO (network contention with image pull) Apr 4, 2019
@tstromberg tstromberg changed the title Minikube hangs when downloading ISO (network contention with image pull) Hung downloading ISO (network contention with image pull) Apr 4, 2019
@tstromberg tstromberg added the area/networking networking issues label Apr 4, 2019
@tstromberg
Copy link
Contributor

Thanks for the bug report!

I believe that the root cause here is we're trying to download too much (7 Linux images simultaneously!), and some networks don't deal with this well.

This should eventually unhang.

You can test my root cause theory by using:

minikube start --cache-images=false

This will end up serializing the image downloads.

@tstromberg tstromberg added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Apr 4, 2019
@tstromberg tstromberg added this to the v1.0.1-candidate milestone Apr 4, 2019
@jonpharry
Copy link

jonpharry commented Apr 11, 2019

I can confirm that on a faster network the hang does eventually clear and minikube starts successfully.

@tstromberg tstromberg self-assigned this Apr 11, 2019
@tstromberg
Copy link
Contributor

tstromberg commented Apr 11, 2019

Thanks for all the feedback! Here's the plan:

  • Complete the ISO download first, before container images are pulled. We need it to start the VM anyways. While the VM is starting, we'll be downloading additional container images.
  • Verify that ISO downloads are retryable and resumable

Look for a PR to be sent out in the next day on this.

@locojuhi
Copy link

I've just installed virtualbox, kubectl and minikube... but when I run minikube start it freeze when is downloading the minikube iso

smile minikube v1.0.0 on linux (amd64)
🤹 Downloading Kubernetes v1.14.0 images in the background ...
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
fire Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
cd Downloading Minikube ISO ...
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2019/04/22 22:19:13 No matching credentials were found, falling back on anonymous
2.25 MB / 142.88 MB [>------------------------------------------] 1.57% 9m36s

I haven't found any solution so far... any suggestion?

@grrowl
Copy link

grrowl commented Apr 26, 2019

@locojuhi read above, running minikube start --cache-images=false works around the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/networking networking issues kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants