cuepls: initial version of "vendor" script

Our approach for creating an initial version of cuepls (named after
gopls) is to start with the gopls implementation, then cut it down,
refactor rename.

This approach (that found in this stack of changes) is the third such
iteration.

The first also started with a working gopls server, but cut
it down and refactored things in a more drastic way. This ultimately
caused us to get bogged down in too much aggressive refactoring early
on.

The second attempt started from a blank page, with "as-required" copying
and adapting of existing gopls code to build out a new LSP server bit by
bit. This meant the "time to a working server" was significantly longer.

This approach again starts with a verbatim copy of gopls but using
"vendored" code from golang.org/x/...

This CL prepares for that "vendor" to define a script that will help
with the vendoring.

This script will be used to vendor parts of golang.org/x/..., after
which we will cut things down by hand.

This is a very basic hard-coded script for now. It might be that we
only use it once. The goal for now is not reuse, just getting the job
done.

It's also unclear whether we will ever re-run the script. Again, the
focus is on the initial import, not later reuse.

For #142

Signed-off-by: Paul Jolly <[email protected]>
Change-Id: Ibdd55f25a66772e06fcab65a8678950e40d81857
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1174163
Unity-Result: CUE porcuepine <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
Reviewed-by: Daniel Martí <[email protected]>

internal/golangorgx/revendorToolsInternal.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+
+set -eu
+shopt -s extglob
+
+# revendorInternal.sh is a script for "vendoring" (internal) parts of the
+# golang.org/x repos for use locally. It currently fixes on versions of
+# x/tools, x/tools/gopls and x/telemetry. It takes a list of target packages,
+# vendors their transitive dependencies, then copies the resulting set of
+# packages under a local directory. Import paths are adjusted to the new
+# location, go:generate directives are stripped out, *_test.go and testdata
+# directories are removed.
+#
+# Whilst this script could be adapted to be run regularly, it's most likely useful
+# as a one-shot wrapper (which sort of suggests the module versions, package listsj)
+
+# Save location of root of repo and change there to start
+SCRIPT_DIR="$( command cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"/
+cd $SCRIPT_DIR
+
+# golang.org/x/tools(/gopls) version
+toolsVersion=v0.17.1-0.20240207023750-c11269ccb0bf
+goplsVersion=v0.0.0-20240207023750-c11269ccb0bf
+telemetryVersion=v0.0.0-20240201224847-0a1d30dda509
+
+# Establish a temporary module to collect and vendor
+# our internal requirements
+td=$(mktemp -d)
+trap "rm -rf $td" EXIT
+cd $td
+cp $SCRIPT_DIR/vendoring/* .
+go mod vendor
+
+tools_regex='s+golang.org/x/tools/internal+cuelang.org/go/internal/golangorgx/tools+g'
+gopls_regex='s+golang.org/x/tools/gopls/internal+cuelang.org/go/internal/golangorgx/gopls+g'
+telemetry_regex='s+golang.org/x/telemetry+cuelang.org/go/internal/golangorgx/telemetry+g'
+
+# Adjust imports
+find ./ -name "*.go" -exec sed -i $tools_regex {} +
+find ./ -name "*.go" -exec sed -i $gopls_regex {} +
+find ./ -name "*.go" -exec sed -i $telemetry_regex {} +
+
+# Strip go:generate directives
+find ./ -name "*.go" -exec sed -i '/^\/\/go:generate/d' {} +
+
+# Remove frontend json files
+find ./ -name "*.json" -exec rm {} +
+
+cd $SCRIPT_DIR
+
+# Force-sync to original
+rsync -a --delete --chmod=D0755,F0644 $td/vendor/golang.org/x/tools/internal/ ./tools
+rsync -a --delete --chmod=D0755,F0644 $td/vendor/golang.org/x/tools/gopls/internal/ ./gopls
+rsync -a --delete --chmod=D0755,F0644 $td/vendor/golang.org/x/telemetry/ ./telemetry
+
+# Cleanup; ensure all copied files are well formatted, and our module is tidy.
+go mod tidy
+gofmt -w .

internal/golangorgx/vendoring/deps.go

@@ -0,0 +1,6 @@
+package deps
+
+import (
+	_ "golang.org/x/tools/gopls"
+	_ "golang.org/x/tools/gopls/internal/test/integration"
+)

internal/golangorgx/vendoring/go.mod

@@ -0,0 +1,21 @@
+module test
+
+go 1.23
+
+require golang.org/x/tools/gopls v0.0.0-20240213143801-1b39a8b6a9ba
+
+require (
+	github.com/BurntSushi/toml v1.2.1 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	golang.org/x/exp/typeparams v0.0.0-20221212164502-fae10dda9338 // indirect
+	golang.org/x/mod v0.15.0 // indirect
+	golang.org/x/sync v0.6.0 // indirect
+	golang.org/x/sys v0.17.0 // indirect
+	golang.org/x/telemetry v0.0.0-20240209200032-7b892fcb8a78 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	golang.org/x/tools v0.18.1-0.20240213143801-1b39a8b6a9ba // indirect
+	golang.org/x/vuln v1.0.1 // indirect
+	honnef.co/go/tools v0.4.6 // indirect
+	mvdan.cc/gofumpt v0.6.0 // indirect
+	mvdan.cc/xurls/v2 v2.5.0 // indirect
+)

internal/golangorgx/vendoring/go.sum

@@ -0,0 +1,40 @@
+github.com/BurntSushi/toml v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
+github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
+github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
+github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/safehtml v0.1.0 h1:EwLKo8qawTKfsi0orxcQAZzu07cICaBeFMegAU9eaT8=
+github.com/google/safehtml v0.1.0/go.mod h1:L4KWwDsUJdECRAEpZoBn3O64bQaywRscowZjJAzjHnU=
+github.com/jba/templatecheck v0.6.0 h1:SwM8C4hlK/YNLsdcXStfnHWE2HKkuTVwy5FKQHt5ro8=
+github.com/jba/templatecheck v0.6.0/go.mod h1:/1k7EajoSErFI9GLHAsiIJEaNLt3ALKNw2TV7z2SYv4=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
+github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
+golang.org/x/exp/typeparams v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y=
+golang.org/x/exp/typeparams v0.0.0-20221212164502-fae10dda9338/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
+golang.org/x/mod v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
+golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
+golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
+golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
+golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/telemetry v0.0.0-20240209200032-7b892fcb8a78 h1:vcVnuftN4J4UKLRcgetjzfU9FjjgXUUYUc3JhFplgV4=
+golang.org/x/telemetry v0.0.0-20240209200032-7b892fcb8a78/go.mod h1:KG1lNk5ZFNssSZLrpVb4sMXKMpGwGXOxSG3rnu2gZQQ=
+golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
+golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+golang.org/x/tools v0.18.1-0.20240213143801-1b39a8b6a9ba h1:dqaVAhw3eYc3J6guBxOyY29fyIOFizYw5/tcUPRSUh8=
+golang.org/x/tools v0.18.1-0.20240213143801-1b39a8b6a9ba/go.mod h1:GL7B4CwcLLeo59yx/9UWWuNOW1n3VZ4f5axWfML7Lcg=
+golang.org/x/tools/gopls v0.0.0-20240213143801-1b39a8b6a9ba h1:0yK4o6+yh4+eP6GNT1gA0G05ulCrYHk7rJYOFfwu3Wc=
+golang.org/x/tools/gopls v0.0.0-20240213143801-1b39a8b6a9ba/go.mod h1:9dmTbCQDoRX+8lRKKi7ScnI9b/i6XsKLdl+L8jOMWvg=
+golang.org/x/vuln v1.0.1 h1:KUas02EjQK5LTuIx1OylBQdKKZ9jeugs+HiqO5HormU=
+golang.org/x/vuln v1.0.1/go.mod h1:bb2hMwln/tqxg32BNY4CcxHWtHXuYa3SbIBmtsyjxtM=
+honnef.co/go/tools v0.4.6 h1:oFEHCKeID7to/3autwsWfnuv69j3NsfcXbvJKuIcep8=
+honnef.co/go/tools v0.4.6/go.mod h1:+rnGS1THNh8zMwnd2oVOTL9QF6vmfyG6ZXBULae2uc0=
+mvdan.cc/gofumpt v0.6.0 h1:G3QvahNDmpD+Aek/bNOLrFR2XC6ZAdo62dZu65gmwGo=
+mvdan.cc/gofumpt v0.6.0/go.mod h1:4L0wf+kgIPZtcCWXynNS2e6bhmj73umwnuXSZarixzA=
+mvdan.cc/xurls/v2 v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
+mvdan.cc/xurls/v2 v2.5.0/go.mod h1:yQgaGQ1rFtJUzkmKiHYSSfuQxqfYmd//X6PxvholpeE=