release(changes): include new release pipeline and bug fix (#305)

* Update release_verify_artifacts.yaml * Release 0.5 (#287) * Update release_verify_artifacts.yaml (#286) * Update release_verify_artifacts.yaml * Update README.md * build(deps): bump actions/checkout from 2 to 4 (#289) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump google.golang.org/grpc from 1.62.1 to 1.63.2 (#290) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.62.1 to 1.63.2. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.62.1...v1.63.2) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add Nix flake for Resonate and use Nix extensively in CI (#291) * Add Nix flake for Resonate * Add an envrc file for direnv * Add missing version field to derivation * Add a Docker image output * Add CA certificates to Docker image * Add more comments to Docker image derivation * Add Nix usage docs to CONTRIBUTING.md * Add initial CI configuration * Infer version from Git rev * Add testing and coverage to Nix CI workflow * Add harness to flake and complete nix-cicd workflow * Nixify dst workflow * Fix name of harness binary * Add Nix to scan workflow * Restore test harness binary immediately prior to linearizability check * Nixify release artifacts workflow * Upload checksum for tarballs * Overwrite now-Nixified workflows * Restore explicit permissions to cicd workflow * Remove nix prefix from job names * Rework testing job by removing build job * Streamline check job * Streamline variable names in release workflow * Simplify flake output references in Nix commands * Provide separate step for building server and harness * Ignore Nix in codecov.yml * Add nix flake check to CI * Install Nix prior to flake check * Add gomod2nix support for package build * Add CI check to ensure gomod2nix.toml is up to date * Add gomod2nix docs and mockgen dependency to dev shell * Add docs on direnv * Add additional note on gomod2nix in CI * Opt for more idiomatic diff checking for gomod2nix.toml * Add gomod2nix.toml to codecov.yml ignore (#292) * Remove unnecessary chmod operations in CI (#293) * Standardize all Actions runners to ubuntu-22.04 (#294) * Build static binary on Linux with Nix (#295) * Use go build for dst workflow * Change step name * Add ldflags * Make resonate package static on Linux * Build with glibc on Linux * Update README.md * Update README.md * Add Actions workflow to update gomod2nix.toml (#297) * Add Actions workflow to update gomod2nix.toml * Update name of workflow * Update connections.go (#300) * Update connections.go * Add timeout to http client --------- Co-authored-by: Gabriel Guerra <[email protected]> * fix(release): workflow had small bugs here and there (#304) * Update release_publish_github_artifacts.yaml * Update release_publish_github_artifacts.yaml * Update release_publish_github_artifacts.yaml * Update release_publish_github_artifacts.yaml * Update release_publish_github_artifacts.yaml --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Luc Perkins <[email protected]> Co-authored-by: Gabriel Guerra <[email protected]>
resonatehq · Apr 30, 2024 · 865fc08 · 865fc08
1 parent 0e110a3
commit 865fc08
Show file tree

Hide file tree

Showing 16 changed files with 749 additions and 186 deletions.
diff --git a/.envrc b/.envrc
@@ -0,0 +1 @@
+use flake
diff --git a/.github/workflows/cicd.yaml b/.github/workflows/cicd.yaml
@@ -12,95 +12,83 @@ permissions:
   contents: read
 
 jobs:
-  build:
-    runs-on: ubuntu-latest
-
-    env:
-      TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_HOST: "localhost"
-      TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_PORT: "5432"
-      TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_USERNAME: "username"
-      TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_PASSWORD: "password"
-      TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_DATABASE: "resonate_test"
-
-    services:
-      postgres:
-        image: postgres:15
-        env:
-          POSTGRES_USER: ${{ env.TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_USERNAME }}
-          POSTGRES_PASSWORD: ${{ env.TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_PASSWORD }}
-          POSTGRES_DB: ${{ env.TEST_AIO_SUBSYSTEMS_STORE_CONFIG_POSTGRES_DATABASE }}
-        options: >-
-          --health-cmd pg_isready
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 5
-        ports:
-          - 5432:5432
-
+  checks:
+    runs-on: ubuntu-22.04
     steps:
-      - name: Checkout repository
+      - name: Check out repository
         uses: actions/checkout@v4
 
-      - name: Set up Go
-        uses: actions/setup-go@v5
-        with:
-          go-version-file: go.mod
-          cache: false
+      - name: Check Nixpkgs input
+        uses: DeterminateSystems/flake-checker-action@v5
 
-      - name: Install and run golangci-lint
-        uses: golangci/golangci-lint-action@v4 # nosemgrep
-        with:
-          version: v1.54
-          args: --verbose --timeout=3m
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@v10
+
+      # basic check to verify that all Nix flake outputs are soundly typed
+      - name: Check flake
+        run: |
+          nix flake check \
+            --all-systems \
+            --no-build
+
+      # If running `gomod2nix` changes `gomod2nix.toml`, that means that the
+      # file is not up to date and the build needs to fail. If that happens,
+      # run `gomod2nix` inside the Nix development environment to regenerate
+      # that file and then commit the changes to Git.
+      - name: Ensure gomod2nix dependencies are up to date
+        run: |
+          nix develop --command gomod2nix
+          git diff --exit-code gomod2nix.toml
+
+      - name: Run golangci-lint
+        run: |
+          nix develop --command \
+            golangci-lint run \
+              --verbose \
+              --timeout=3m
 
       - name: Run go test and generate coverage report
         run: |
-          go test -v -coverprofile=coverage.out -coverpkg=./... ./...
+          nix develop --command \
+            go test \
+              -v \
+              -coverprofile=coverage.out \
+              -coverpkg=./... \
+              ./...
 
       - name: Upload coverage report to Codecov
         uses: codecov/codecov-action@v4 # nosemgrep
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           codecov_yml_path: codecov.yml
 
-      - name: Build resonate
-        run: go build -o resonate
-
-      - name: Cache resonate binary
-        uses: actions/cache/save@v4
-        with:
-          path: resonate
-          key: resonate-${{ github.sha }}
-
   test:
-    runs-on: ubuntu-latest
-    needs: [build]
+    runs-on: ubuntu-22.04
     steps:
-      - name: Restore resonate binary
-        uses: actions/cache/restore@v4
-        with:
-          path: resonate
-          key: resonate-${{ github.sha }}
-          fail-on-cache-miss: true
+      - name: Check out repository
+        uses: actions/checkout@v4
 
-      - name: Run resonate server
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@v10
+
+      - name: Use Magic Nix Cache
+        uses: DeterminateSystems/magic-nix-cache-action@v4
+
+      - name: Build server and harness
         run: |
-          chmod +x resonate
-          ./resonate serve --aio-store-sqlite-path :memory: &
+          nix build ".#resonate"
+          cp ./result/bin/resonate resonate
 
-      - name: Checkout test harness
-        uses: actions/checkout@v4
-        with:
-          repository: resonatehq/durable-promise-test-harness
+          nix build ".#durable-promise-test-harness"
+          cp ./result/bin/durable-promise-test-harness durable-promise-test-harness
 
-      - name: Build test harness
+      - name: Run resonate server
         run: |
-          go build -o harness
+          ./resonate serve --aio-store-sqlite-path :memory: &
 
       - name: Run linearizability check
         run: |
-          chmod +x harness
-          ./harness verify -r 1000 -c 10
+          ./durable-promise-test-harness verify -r 1000 -c 10
 
       - uses: actions/upload-artifact@v4
         if: ${{ always() }}
@@ -110,13 +98,22 @@ jobs:
 
   scan:
     name: semgrep
-    needs: [build]
     runs-on: ubuntu-20.04
+    needs: [test]
     env:
       SEMGREP_RULES: "auto semgrep.yml"
-    container:
-      image: returntocorp/semgrep
     if: (github.actor != 'dependabot[bot]')
     steps:
-      - uses: actions/checkout@v4
-      - run: semgrep ci
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@v10
+
+      - name: Use Magic Nix Cache
+        uses: DeterminateSystems/magic-nix-cache-action@v4
+
+      - name: semgrep
+        run: |
+          nix develop --command \
+            semgrep ci
diff --git a/.github/workflows/dependencies.yml b/.github/workflows/dependencies.yml
@@ -0,0 +1,46 @@
+# this approach is indebted to this blog post from GitHub user hallettj:
+# https://sitr.us/2024/03/08/nix-npm-and-dependabot.html
+
+name: Update gomod2nix.toml upon Dependabot changes to Go dependencies
+
+on:
+  push:
+    branches:
+      - "dependabot/go_modules/*"
+
+jobs:
+  update_gomod2nix_toml_file:
+    name: Update gomod2nix.toml if Go dependencies have changed
+    runs-on: ubuntu-22.04
+    if: github.actor == 'dependabot[bot]' || github.actor == 'dependabot-preview[bot]'
+    permissions:
+      contents: write
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@v10
+
+      - name: Use Magic Nix Cache
+        uses: DeterminateSystems/magic-nix-cache-action@v4
+
+      - name: Update gomod2nix.toml (if go.mod has changed)
+        run: |
+          nix develop --command gomod2nix
+      
+      - name: Set up Git config
+        run: |
+          git config user.name "${GITHUB_ACTOR}"
+          git config user.email "${GITHUB_ACTOR}@users.noreply.github.com"
+      
+      - name: Commit changes
+        run: |
+          git add .
+          if [[ $(git status -s) ]]; then
+            git commit -m "build(deps): update gomod2nix.toml [dependabot skip]" --no-verify
+            git push
+            echo "pushed an update to gomod2nix.toml"
+          else
+            echo "go dependencies have not changed"
+          fi
diff --git a/.github/workflows/dst.yaml b/.github/workflows/dst.yaml
@@ -14,34 +14,30 @@ permissions:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     steps:
-    - id: cache
-      uses: actions/cache/restore@v4
-      with:
-        path: resonate
-        key: resonate-${{ github.sha }}
-    - name: Checkout repository
-      if: steps.cache.outputs.cache-hit != 'true'
+    - name: Check out repository
       uses: actions/checkout@v4
-    - name: Set up Go
-      if: steps.cache.outputs.cache-hit != 'true'
-      uses: actions/setup-go@v5
-      with:
-        go-version-file: go.mod
-        cache: false
-    - name: Build resonate
-      if: steps.cache.outputs.cache-hit != 'true'
-      run: go build -o resonate
+
+    - name: Install Nix
+      uses: DeterminateSystems/nix-installer-action@v10
+
+    - name: Use Magic Nix Cache
+      uses: DeterminateSystems/magic-nix-cache-action@v4
+
+    - name: Build resonate binary
+      run: |
+        nix build ".#resonate"
+        cp ./result/bin/resonate resonate
+
     - name: Cache resonate binary
-      if: steps.cache.outputs.cache-hit != 'true'
       uses: actions/cache/save@v4
       with:
         path: resonate
         key: resonate-${{ github.sha }}
 
   seed:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     steps:
     - id: seed
       name: Set random seed
@@ -50,7 +46,7 @@ jobs:
       seed: ${{ inputs.seed || steps.seed.outputs.seed }}
 
   dst:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     needs: [build, seed]
     timeout-minutes: 150
 
@@ -95,7 +91,6 @@ jobs:
         fail-on-cache-miss: true
     - name: Run dst (seed=${{ needs.seed.outputs.seed }}, scenario=${{ matrix.scenario }}, store=${{ matrix.store }})
       run: |
-        chmod +x resonate
         ./resonate dst run --seed ${{ needs.seed.outputs.seed }} --scenario ${{ matrix.scenario }} --aio-store ${{ matrix.store }} > logs.txt 2>&1
     - name: Create issue if dst failed
       env:
@@ -118,7 +113,7 @@ jobs:
         path: logs.txt
 
   diff:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     needs: [build, seed, dst]
 
     strategy:
@@ -162,7 +157,6 @@ jobs:
         GITHUB_TOKEN: ${{ secrets.github_token }}
       if: ${{ failure() }}
       run: |
-        chmod +x resonate
         ./resonate dst issue \
           --seed ${{ needs.seed.outputs.seed }} \
           --scenario ${{ matrix.scenario }} \