Enhance Renovate configuration (#314)

* Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update Dockerfile.tmpl * Update Dockerfile * Update variables.sh * Delete go.sum.tmpl * Update generate_gateway.sh * Update Dockerfile * Update renovate.json5 * Update renovate.json5 * Update renovate.json5 * Update Dockerfile * Update go.mod.tmpl * Update renovate.json5 * chore(deps): update dependency google.golang.org/grpc to v1.47.0 * Update Dockerfile.tmpl * Update renovate.json5 * Update variables.sh * Add readme for Renovate * readme fixes * Update .github/README.md Co-authored-by: Steve Ognibene <[email protected]> * Update .github/README.md Co-authored-by: Steve Ognibene <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Steve Ognibene <[email protected]>
namely · Jul 19, 2022 · ddcb1d2 · ddcb1d2
1 parent f8c9fff
commit ddcb1d2
Show file tree

Hide file tree

Showing 8 changed files with 167 additions and 608 deletions.
diff --git a/.github/README.md b/.github/README.md
@@ -0,0 +1,38 @@
+# Renovate Configuration
+
+## What is Renovate?
+
+[Renovate](https://www.mend.io/free-developer-tools/renovate/) is an open-source dependency manager.
+The `Renovate` Github app is configured to automatically open PRs in an attempt to update configured dependencies of different kinds.
+The configuration is managed in [renovate.json5](./renovate.json5).
+
+## How does Renovate work?
+
+`Renovate` is able to automatically identify dependencies of different kinds in different files and start opening PRs once a basic configuration file is found in the repo and the app is enabled. `Renovate` has a lot of different configuration options beyond the supported dependencies which are well documented [here](https://docs.renovatebot.com/).
+
+However, in cases such as this repo where the dependencies are managed in proprietary files, additional configuration is required.
+
+## How is Renovate used in this repo?
+
+In this repo, we have a variety of dependencies pinned to specific versions, primarly to create reproducible, testable builds.\
+Most of these dependencies are installed in the built docker images and are referenced in [Dockerfile](../Dockerfile). However, their versions are set separately in [variables.sh](../variables.sh), which means most PRs opened by `Renovate` will attempt to update this file.
+Because it is a proprietary file, we leverage `Renovate`'s support of using Regular Expressions to find & replace dependencies versions.
+
+The different sources used to find dependencies include `npm`, `Go`, and `Github Releases`.
+
+### Dependencies PR grouping
+
+One of great features of `Renovate` is the ability to group dependency updates into mutual PRs according to different criteria to avoid spamming the repo with too many PRs (1 PR per dependency).
+
+In this repo, we have several groups configured according to specific dependencies and/or dependency types:
+
+*   Major/Minor (main) gRPC updates - these dependencies' updates will be opened in one PR. Merging PRs for this dependency group will be followed by releasing new major version in this repo.
+
+*   All other minor/patch dependency updates - Usually as long as the tests pass for this PR we should be able to merge & release a new patch version in this repo.
+
+*   Go gRPC Gateway module - This dependency is singled out since we cannot update it yet due to a breaking change. Once we do update it, we can move this dependency to the same group as the other non gRPC dependencies.
+
+## How to keep maintaining Renovate in this repo?
+
+If and when adding a new dependency, we should pin its version in `variables.sh` (if applicable), configure it
+in `Renovate`'s configuration file and ensure it is grouped correctly.
diff --git a/.github/renovate.json5 b/.github/renovate.json5
@@ -4,17 +4,25 @@
   baseBranches: [
     "master"
   ],
+  constraints: {
+    "go": "1.17"
+  },
   extends: [
     // A lot of default settings that you can extend can be found in the presets section: https://docs.renovatebot.com/presets-compatibility/
-    ":ignoreModulesAndTests"
+    "config:base"
   ],
+  ignorePaths: [],
   enabledManagers: [
     // Since we only update proprietry files (e.g variables.sh)we should use 'regex' here (see the matchManagers in the packageRules section below)
     // If you need more, see https://docs.renovatebot.com/modules/manager/
     "regex", 
     "github-actions",
-    "gomod"
+    "gomod", 
+    "npm"
   ],
+  "gomod":{
+    "fileMatch": ["(^|/)go.mod.tmpl$"]
+  }, 
   regexManagers: [
     {
       fileMatch: [
@@ -47,25 +55,22 @@
     },
     {
       fileMatch: [
-        "gwy/templates/go.mod.tmpl$"
+        "variables.sh$"
       ],
       matchStrings: [
-        "google.golang.org/grpc\\s+(?<currentValue>.*?)\\n+"
+        "GRPC_WEB_VERSION=\\$\\{GRPC_WEB_VERSION:-(?<currentValue>.*?)\\}\\n"
       ],
-      depNameTemplate: "grpc/grpc-go",
-      datasourceTemplate: "github-releases",
-      versioningTemplate: "regex:^v(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$",
+      depNameTemplate: "grpc-web",
+      datasourceTemplate: "npm"
     },
     {
       fileMatch: [
         "variables.sh$"
       ],
       matchStrings: [
-        // notice we update the same version in two locations perhaps we don't need two separate variables?)
-        "GRPC_WEB_VERSION=\\$\\{GRPC_WEB_VERSION:-(?<currentValue>.*?)\\}\\n",
         "NODE_PROTOC_GEN_GRPC_WEB_VERSION=\\$\\{NODE_PROTOC_GEN_GRPC_WEB_VERSION:-(?<currentValue>.*?)\\}\\n"
       ],
-      depNameTemplate: "grpc-web",
+      depNameTemplate: "protoc-gen-grpc-web",
       datasourceTemplate: "npm"
     },
     {
@@ -79,33 +84,92 @@
       datasourceTemplate: "github-releases",
       extractVersionTemplate: "^v(?<version>\\d+\\.\\d+\\.\\d+)"
     },
+    {
+      fileMatch: [
+        "variables.sh$"
+      ],
+      matchStrings: [
+        "GRPC_GATEWAY_VERSION=\\$\\{GRPC_GATEWAY_VERSION:-(?<currentValue>.*?)\\}\\n"
+      ],
+      depNameTemplate: "github.com/grpc-ecosystem/grpc-gateway/v2",
+      datasourceTemplate: "go",
+    },
+    {
+      fileMatch: [
+        "variables.sh$"
+      ],
+      matchStrings: [
+        "NODE_GRPC_TOOLS_NODE_PROTOC_TS_VERSION=\\$\\{NODE_GRPC_TOOLS_NODE_PROTOC_TS_VERSION:-(?<currentValue>.*?)\\}\\n"
+      ],
+      depNameTemplate: "grpc_tools_node_protoc_ts",
+      datasourceTemplate: "npm",
+    },
+    {
+      fileMatch: [
+        "variables.sh$"
+      ],
+      matchStrings: [
+        "NODE_GRPC_TOOLS_VERSION=\\$\\{NODE_GRPC_TOOLS_VERSION:-(?<currentValue>.*?)\\}\\n"
+      ],
+      depNameTemplate: "grpc-tools",
+      datasourceTemplate: "npm",
+    },
+    {
+      fileMatch: [
+        "^Dockerfile$"
+      ],
+      matchStrings: [
+        "GO111MODULE=on go get google\.golang\.org\/grpc@(?<currentValue>.*?)\\s*\\n"
+      ],
+      depNameTemplate: "google.golang.org/grpc",
+      datasourceTemplate: "go",
+    }
   ],
   packageRules: [
     // here you can define how dependency updates should be grouped in one or multiple PRs:
     // The following means that any major or minor updates to the specified packages will be grouped in one PR,
     // and we would like to cut a new major release for our docker image, e.g 1.42_X => 1.46_0
     {
-      matchPackageNames: ["grpc/grpc", "grpc/grpc-java", "grpc/grpc-go"],
+      matchPackageNames: ["grpc/grpc"],
       groupName: "gRPC Version",
       matchUpdateTypes: [
         "minor",
         "major"
       ],
       groupSlug: "gRPC Major Upgrade"
     },
+    {
+      matchPackageNames: ["grpc/grpc-java", "google.golang.org/grpc"],
+      groupName: "gRPC Version",
+      matchUpdateTypes: [
+        "minor",
+        "major"
+      ],
+      "stabilityDays": 10,
+      groupSlug: "gRPC Major Upgrade"
+    },
     // specifically for grpc-go, a patch version upgrade should be grouped with other misc dependencies, which you can find below.
     // This would most likely mean we want to bump the docker image version without bumping the gRPC version, e.g 1.42_0 => 1.42_1
     {
-      matchPackageNames: ["grpc/grpc-go"],
+      matchPackageNames: ["google.golang.org/grpc"],
       matchUpdateTypes: [
         "patch"
       ],
       groupName: "Misc Non-major dependencies",
       groupSlug: "patch-dependencies"
     },
-    // 
     {
-      matchPackageNames: ["grpc-web", "envoyproxy/protoc-gen-validate"],
+      matchPackageNames: ["grpc-web", "protoc-gen-grpc-web", "envoyproxy/protoc-gen-validate"],
+      matchUpdateTypes: [
+        "minor",
+        "patch"
+      ],
+      groupName: "Misc Non-major dependencies",
+      groupSlug: "minor-dependencies"
+    },
+    {
+      matchDatasources: ["go", "npm"],
+      excludePackageNames: ["github.com/grpc-ecosystem/grpc-gateway/v2", "google.golang.org/grpc"],
       matchUpdateTypes: [
         "minor",
         "patch"
@@ -114,7 +178,7 @@
       groupSlug: "minor-dependencies"
     },
     {
-      matchPackageNames: ["grpc-web", "envoyproxy/protoc-gen-validate"],
+      matchPackageNames: ["grpc-web", "protoc-gen-grpc-web", "envoyproxy/protoc-gen-validate"],
       matchUpdateTypes: [
         "major"
       ],

diff --git a/Dockerfile b/Dockerfile
@@ -74,17 +74,17 @@ RUN ( cd ./grpc-go/cmd/protoc-gen-go-grpc && go install . )
 # Go get go-related bins
 WORKDIR /tmp
 RUN set -e && \
-    GO111MODULE=on go get google.golang.org/grpc@v$grpc_version
+    GO111MODULE=on go get google.golang.org/grpc@v1.47.0
 
 # install protoc-gen-grpc-gateway and protoc-gen-openapiv2
 RUN set -e && \
-    GO111MODULE=on go get -u github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@v${grpc_gateway_version} && \
-    cd /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@v${grpc_gateway_version}/protoc-gen-grpc-gateway && \
+    GO111MODULE=on go get -u github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@${grpc_gateway_version} && \
+    cd /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@${grpc_gateway_version}/protoc-gen-grpc-gateway && \
     go install .
 
 RUN set -e && \
-    GO111MODULE=on go get -u github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@v${grpc_gateway_version} && \
-    cd /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@v${grpc_gateway_version}/protoc-gen-openapiv2 && \
+    GO111MODULE=on go get -u github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@${grpc_gateway_version} && \
+    cd /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@${grpc_gateway_version}/protoc-gen-openapiv2 && \
     go install .
 
 RUN go get -u github.com/gogo/protobuf/protoc-gen-gogo
@@ -177,7 +177,7 @@ COPY --from=build /tmp/grpc_web_plugin /usr/local/bin/grpc_web_plugin
 
 COPY --from=build /tmp/protoc-gen-scala /usr/local/bin/
 
-COPY --from=build /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@v${grpc_gateway_version}/protoc-gen-openapiv2/options /opt/include/protoc-gen-openapiv2/options/
+COPY --from=build /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@${grpc_gateway_version}/protoc-gen-openapiv2/options /opt/include/protoc-gen-openapiv2/options/
 
 COPY --from=build /go/pkg/mod/github.com/envoyproxy/protoc-gen-validate@v${go_envoyproxy_pgv_version}/ /opt/include/