dep ensure -update doesn't realize when a vendored directory has been removed

[markcampanelli-wf]

What version of dep are you using (dep version)?

$ brew info dep
dep: stable 0.3.1 (bottled), HEAD
Go dependency management tool
https://github.com/golang/dep
/usr/local/Cellar/dep/0.3.1 (7 files, 10.8MB) *
  Poured from bottle on 2017-10-13 at 12:58:59
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/dep.rb
==> Dependencies
Required: go ✔

$ dep version
dep:
 version     : devel
 build date  :
 git hash    :
 go version  : go1.9
 go compiler : gc
 platform    : darwin/amd64

What dep command did you run?

After a dep ensure, I moved vendor/github.com/gopherjs/gopherjs outside of my project directory (gopherjs doesn't support being vendored), then I tried to regenerate it in the vendor directory using

dep ensure -update github.com/gopherjs/gopherjs

What did you expect to see?

The reappearance of vendor/github.com/gopherjs/gopherjs in my project directory.

What did you see instead?

Nothing new appeared. Running in verbose mode suggests that dep determines that nothing should be done. However, running a full dep ensure does regenerate vendor/github.com/gopherjs/gopherjs.

[sdboyer]

hi, welcome! sorry you ran into this - it's quite odd. also, i can't replicate it on tip - could you try it with the latest dep and see if you can replicate, to make sure i'm not suffering from the curse of knowledge on this one?

also, a side note - all variants of dep ensure {,-add,-update} should always fully repopulate the vendor directory. passing -update github.com/gopherjs/gopherjs has an unrelated effect - it's trying to update Gopkg.lock to the latest version of github.com/gopherjs/gopherjs that's allowed by the constraint in Gopkg.toml, then writing out vendor. if you were already at the latest allowed (which i imagine you were), then no change happens in Gopkg.lock, so you don't observe a difference there. (what's weird is that vendor is apparently not being correctly updated afterwards)

[markcampanelli-wf]

It still behaves this way for me using the bleeding edge version of dep from dep ensure -update github.com/gopherjs/gopherjs. (Note that the dep version result looks the same for the brew stable version 0.3.1 or this latest version, but I did brew uninstall dep and checked that the new dep was indeed from the $GOPATH/bin after go get -u github.com/golang/dep/cmd/dep.)

Not sure if this helps, but this is how we constrain gopherjs in the Gopkg.toml:

[[constraint]]
  revision = "415225646bb92c4449bb484646f2c95a98402f6f"
  name = "github.com/gopherjs/gopherjs"

giving this in Gopkg.lock

[[projects]]
  name = "github.com/gopherjs/gopherjs"
  packages = ["js"]
  revision = "415225646bb92c4449bb484646f2c95a98402f6f"

[markcampanelli-wf]

Also, thanks for clarifying the intent and behavior of the -update switch versus not using any switch. Because we are pinning to a specific commit, it appears that the -update switch is indeed superfluous, because dep ensure will change the commit as needed.

[nathanielc]

I am getting the same behavior on tip as well.

My Gopkg.lock file has this entry:

[[projects]]
  name = "github.com/pkg/errors"
  packages = ["."]
  revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
  version = "v0.8.0"

My Gopkg.toml has no constraints for this particular dependency.

My vendor dir does not have the vendor/github.com/pkg/errors directory:

$ ls -la vendor/github.com/pkg/
total 0
drwxr-xr-x 1 nathanielc nathanielc  0 Oct 23 16:12 .
drwxr-xr-x 1 nathanielc nathanielc 68 Oct 23 16:11 ..

Running dep ensure completes but the errors package is still not present in the vendor dir. However if I delete the entire vendor dir and rerun dep ensure everything is correctly placed back in the vendor dir.

[nathanielc]

Hmm, its fixed now on tip, I must not have actually updated to tip when I tried a few minutes ago. Sorry for the confusion.

[sdboyer]

@nathanielc well, i'm glad we're back down to just one instance of this :)

and, ahh, i've now replicated it. ok, we definitely have a bug.

[sdboyer]

ahh, ok, i see what's going on here.

right now, we're inferring from Gopkg.lock being unchanged that vendor is also probably unchanged, and therefore not regenerating it. this is unsafe, as the workflow here demonstrates, and we won't really properly be able to tell if it's safe until we have #121.

[sdboyer]

hmm, so, dep ensure -vendor-only will repopulate vendor correctly in this case. i realize it's less than ideal, but i'd kind of like to avoid disrupting the main workflow for this case, especially given that we know this case won't exist anymore once we have #121.

thoughts?

[markcampanelli-wf]

I'm fine with addressing this in #121. Hopefully available soon :)

[sdboyer]

@markcampanelli-wf yeah, the contributors for it have dropped off a bit recently, but it's one of my highest priority things. i'll have to step in by end of Nov if we haven't seen it through, i think.