Can't dep ensure github.com/google/google-api-go-client

[markmandel]

What version of Go are you using (go version)?
1.8

What operating system and processor architecture are you using?
Linux, Debian Jessie

What did you do?
dep ensure github.com/google/google-api-go-client

What did you expect to see?
github.com/google/google-api-go-client vendored in my ./vendor directory

What did you see instead?
Nothing was there. Looks like nothing gets pulled down. No error is thrown either.

I'm wondering if it's because there is no go files in the root of the repository?

Since there is no support for ensuring subdirectories within a repository - this repo can't be ensured with dep 😢

[sdboyer]

Thanks for the issue report!

Yeah, we need to clarify the docs on this - but this is by design. Yes, it's because of this:

I'm wondering if it's because there is no go files in the root of the repository?

With no go files, it means there's no imports. Imports are what actually determine that a dependency is required. A constraint in the manifest, or stating it as an argument to dep ensure, isn't.

This is intended to enable a workflow where you stay in your editor, add the import paths you need, then just dep ensure, and it'll work it out. It does create this awkwardness in the ensure subcommand, though. There's some discussion of a possible way we might make it match your immediate expectations here better in #36. We're re-speccing the commands in #277, and will definitely deal with this case.

Meantime, though, closing this, as it's working as intended...even if that is confusing, we have issues open already to address that 😄

[markmandel]

Oh that is interesting. Thanks for the clarification - I appreciate it.

[mgutz]

@sdboyer Isn't this the chicken-egg problem? Unused imports cause go errors and most smart editors will remove them. So if I want to try a package, I first have to go get then write code that uses it, then dep ensure?

Most dependency managers I have used in other languages will just get the dependency and put into the "vendor" directory instead of trying to be clever based on whether it is actually used in source.

[markmandel]

So once I understood the behaviour, I just used a import _ "google.golang.org/api/compute/v1" so it wouldn't be removed, and then did a dep ensure.

I do get stuck here if I wanted a specific branch/tag of that repository, as I (assume) can't manually dep ensure it - which is a bit of problem.

But otherwise, I quite like the workflow.

[mgutz]

The workflow seems off. The manifest and lock file should be the source of truth. Factoring in source files is an extra level of complexity that brings about quirks like this.

[sdboyer]

@mgutz

Isn't this the chicken-egg problem? Unused imports cause go errors and most smart editors will remove them. So if I want to try a package, I first have to go get then write code that uses it, then dep ensure?

Yes, there's certainly risk of getting in an awkward situation like that (though as @markmandel, and others, have noted, it seems to work well in practice). But still, this is why - as discussed in #36 - we may take the approach where you can do something like dep ensure github.com/foo/bar, and that'll show up in your lock & vendor, even if it's not imported. Code completion, at least, becomes easy then.

Most dependency managers I have used in other languages will just get the dependency and put into the "vendor" directory instead of trying to be clever based on whether it is actually used in source.

No argument that that's what most dependency managers do. It's also true that most dependency managers:

1. Don't have such descriptive names (e.g. FQDNs) available in the source code
2. Don't have access to static analysis for this problem that is both fast and complete

This isn't so much being clever as it is taking applying Go's unique characteristics to this problem domain.

The manifest and lock file should be the source of truth. Factoring in source files is an extra level of complexity

I definitely agree on the principle that there should be one source of truth, and that having more than one adds complexity. That's exactly why it's suboptimal to treat the manifest as the source of truth. Like it or not, because of that property that (basically) DNS gives import paths meaning, rather than a name system defined by the package manager, import paths are already a source of truth.

that brings about quirks like this.

It's an opinionated tool - no question. Those opinions have quirks, and this issue is one of them (though I definitely think we can work this one through). But I'd say there are other quirks it alleviates - like having to repeat yourself by typing in an import in BOTH a source file, then in a CLI command.

More importantly, though, Go's particular characteristics mean that treating the manifest as the source of truth would end up creating several new circles of dependency hell. (Think about how manifest-as-truth plays out if k8s is one of your dependencies, as it already is for many dep testers)

@markmandel

I do get stuck here if I wanted a specific branch/tag of that repository, as I (assume) can't manually dep ensure it - which is a bit of problem.

dep ensure github.com/foo/bar@<constraint> 😄 Again, ensure is going to be changing soon, but I expect we'll keep something shaped at least generally like that.

[markmandel]

@sdboyer (so I just testing this) - this won't work in the instance of this bug - because there are no .go files in github.com/google/google-api-go-client

So if I do a dep ensure github.com/google/google-api-go-client@master nothing is downloaded. So there is no way for me to specify a constraint on this dependency. That was my point - apologies for the lack of clarity.

[sdboyer]

Ah yeah, that's the project vs. package question. Another thing to figure out :)