fix: Avoid panic when s3 URL is invalid

[liamg]

Gracefully handle when S3 URLs have an unexpected number of path segments.

Currently we expect s3.amazonaws.com/bucket/path, but something like s3.amazonaws.com/bucket will cause a panic, e.g.

panic: runtime error: index out of range [2] with length 2

github.com/hashicorp/go-getter.(*S3Getter).parseUrl(,)
    /go/pkg/mod/github.com/hashicorp/[email protected]/get_s3.go:272
github.com/hashicorp/go-getter.(*S3Getter).Get(, {,},)
    /go/pkg/mod/git...

[hashicorp-cla-app]

All committers have signed the CLA.

[crw]

Hi @liamg, out of curiosity, what is the context in which you discovered this issue? Is there an associated issue in go-getter or in one of the product repositories? Thanks!

[liamg]

Hey, we discovered this internally when using go-getter to pull down terraform modules by URL. Somebody had omitted the object path from the module URL and 💥 .

It's reproducible with the latest terraform by doing this:

module "uhoh" {
  source = "s3.amazonaws.com/bucket"
}

and then a terraform init:

Initializing the backend...
Initializing modules...
Downloading s3::https://s3.amazonaws.com/bucket for uhoh...

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

runtime error: index out of range [2] with length 2
goroutine 1 [running]:
runtime/debug.Stack()
	/Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
	/Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x1c
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x164
panic({0x107e52f40?, 0x1400007a300?})
	/Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x26c
github.com/hashicorp/go-getter.(*S3Getter).parseUrl(0x14000850bac?, 0x140004d6ea0)
	/Users/runner/go/pkg/mod/github.com/hashicorp/[email protected]/get_s3.go:272 +0x808

I'll raise an issue there too.

[crw]

Related issue: hashicorp/terraform#35515

[radeksimko]

Hi @liamg
Thanks for the PR - this looks like a reasonable patch and something certainly worth fixing!

However, before we do so - can you add some tests that reproduce the panic without the patch and pass after the patch? This can be a new test under https://github.com/hashicorp/go-getter/blob/main/get_s3_test.go

[liamg]

@radeksimko Sure thing, I should have done that already 😅

I've added a few test cases now.

[radeksimko]

LGTM, thanks!

[liamg]

Thanks @radeksimko - do you know if we can expect a release any time soon so we can grab this fix?

[radeksimko]

Took a while due to some confusion around versioning and releasing, but I just released v1.7.6: https://github.com/hashicorp/go-getter/releases/tag/v1.7.6

My understanding is that v1 is used by Terraform, so that should eventually address the bug there. We could cherry-pick the patch to v2 as well though.