Go: support Protobuf

[Eric-Arellano]

No description provided.

[tdyas]

Tentatively implemented by #13985 although that PR will need to change once #14258 is implemented.

[Eric-Arellano]

Some exciting progress!

1. Support for generating raw .go files landed in [internal] Add Go Protobuf compiler #13985 thanks to @tdyas
2. We figured out how to support codegen for compiled languages with [internal] Add plugin hook for Go codegen support #14707

There are no longer any blockers for this! Only time to implement the last part: implementing this rule

pants/src/python/pants/backend/codegen/protobuf/go/rules.py

Lines 57 to 61 in 09ba1b2

	@rule
	async def setup_build_go_package_request_for_protobuf(
	_: GoCodegenBuildProtobufRequest,
	) -> BuildGoPackageRequest:
	raise NotImplementedError()

An example rule:

pants/src/python/pants/backend/go/util_rules/build_pkg_target_test.py

Lines 51 to 82 in 09ba1b2

	@rule
	async def generate_from_file(request: GoCodegenBuildFilesRequest) -> BuildGoPackageRequest:
	content = dedent(
	"""\
	package gen
	import "fmt"
	import "github.com/google/uuid"
	func Quote(s string) string {
	uuid.SetClockSequence(-1) // A trivial line to use uuid.
	return fmt.Sprintf(">> %s <<", s)
	}
	"""
	)
	digest = await Get(Digest, CreateDigest([FileContent("codegen/f.go", content.encode())]))
	deps = await Get(Addresses, DependenciesRequest(request.target[Dependencies]))
	assert len(deps) == 1
	assert deps[0].generated_name == "github.com/google/uuid"
	thirdparty_dep = await Get(FallibleBuildGoPackageRequest, BuildGoPackageTargetRequest(deps[0]))
	assert thirdparty_dep.request is not None
	return BuildGoPackageRequest(
	import_path="codegen.com/gen",
	digest=digest,
	dir_path="codegen",
	go_file_names=("f.go",),
	s_file_names=(),
	direct_dependencies=(thirdparty_dep.request,),
	minimum_go_version=None,
	)

I think this can be implemented in several stages.

1. Get compilation of a single trivial Protobuf file working.
   • For this compilation to work, it'll need the Protobuf third-party package as a dependency of the protobuf_source target. For now, that can be added as an explicit dependency in the BUILD file. The rule will need to call await Get(BuildGoPackageRequest, BuildGoPackageTargetRequest()) on that go_third_party_package target to get its compilation set up properly.
2. Make sure that single trivial Protobuf file can be depended upon by first-party go_package code.
   • This will require us figuring out how to calculate the import path the right way so that the generated code is importable.
3. Figure out how to handle multiple Protobuf files that should belong to the same package.
   • Protobuf is file-based, whereas Go is directory based. With Go, the whole package must be present.
   • I think the rule should automatically find all protobuf_source targets that belong to the same package
4. Figure out how generated packages depending on other generated packages should be handled.

Then there is polish:

• Automatically add the dependency on the Protobuf go_third_party_package target to each protobuf_source target via dependency injection
• Add dependency inference for Go imports of generated code.

If anyone is interested in helping with implementing this, I'd be happy to mentor e.g. better instructions or pair programming! I need to switch to adding the codegen plugin hook for JVM to unblock some plugin authors, so I'm not likely to get to any of the above for a few days, possibly longer.

[tdyas]

WIP PR: #14714