Anyone manage to get Hasura to work in a CI/CD pipeline, e.g. with GitHub Actions?

[joshuarobs]

Edit:

I have solved this issue. This is the working GitHub Actions workflow file:

name: Node CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:12.1
        env:
          POSTGRES_USER: postgres
          POSTGRES_DB: postgres
        ports:
        # will assign a random free host port
        - 5432:5432
        # needed because the postgres container does not provide a healthcheck
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
      hasura:
        image: hasura/graphql-engine:latest
        env:
          HASURA_GRAPHQL_DATABASE_URL: postgres://postgres:@postgres:5432/postgres
        ports:
          - 8080:8080

    strategy:
      matrix:
        node-version: [12.x]

    steps:
    - uses: actions/checkout@v1
    - name: Install Hasura cli
      run: curl -L https://github.com/hasura/graphql-engine/raw/master/cli/get.sh | bash
    - name: Start up the Docker containers and migrate to the latest version of the database
      run: |
        docker ps
        cd docker
        hasura migrate status --skip-update-check --endpoint http://localhost:8080
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - name: npm install, build, and test
      run: |
        node index.js
      env:
        CI: true

Working tests are in this repo: https://github.com/joshuarobs/making-hasura-work-with-github-actions

---

This is my original issue (#3638) that I have closed so that I can put it here, in this more centralised issue.

I've spent the last 2 days trying to get Hasura to work with GitHub Actions, so that I can run tests in the cloud after pushing commits. This is so that I can have a proper DevOps workflow, with proper CI/CD.

I've had no luck in being able to get this work. For now I'm giving up, and just going to have to skip cloud testing and pray that my local tests are adequate enough. If someone else wants to take over, good luck.

I have two repos I've tried to make Hasura work in a CI/CD environment:

1. Attempt 1 - https://github.com/joshuarobs/making-hasura-work-with-github-actions
2. Attempt 2 - https://github.com/joshuarobs/test-ci

Attempt 1
This one uses instructions for setup from here: https://docs.hasura.io/1.0/graphql/manual/getting-started/docker-simple.html

This is by running a docker-compose, instead of running docker-run.sh.

Attempt 2
This one uses instructions for setup from here: https://docs.hasura.io/1.0/graphql/manual/deployment/docker/index.html

This is by running a docker-run.sh file, instead of running docker-compose.

If someone has managed to make this work with GitHub Actions, please share how you managed to make it work. Or if you managed to make it work with another build server/service, what did you use and what script are you using?

Also with attempt 1 repo, I found a slight clue as to what might be going wrong here: #3648 But I don't know what to do with this clue

[shahidhk]

@joshuarobs What is the error/issue that you're hitting with this configuration?

[joshuarobs]

@shahidhk
For my first repo attempt (full details here: https://github.com/joshuarobs/making-hasura-work-with-github-actions/commit/359867222135dd31e749b9e81327ad1c610d7843/checks?check_suite_id=391660183)

This is the error: time="2020-01-09T09:21:05Z" level=fatal msg="version check: failed to get version from server: failed making version api call: Get http://localhost:8080/v1/version: EOF"

If you see the link, you can see that calling pg_dump returns no SQL statements. I find it weird, because if I run the same pg_dump on my machine, it returns a lot of SQL stuff.

On my second repo (full details here: https://github.com/joshuarobs/test-ci/commit/07c627eef3d2627ee3d610b4e6efeba3ca3f2fc2/checks?check_suite_id=387692198)

I get this error: ```
time="2020-01-07T09:40:31Z" level=fatal msg="version check: failed to get version from server: failed making version api call: Get http://localhost:8080/v1/version: dial tcp 127.0.0.1:8080: connect: connection refused"
28
##[error]Process completed with exit code 1.

Sometimes running a similar workflow gives different errors. But usually they fail to get the version of Hasura

[samuelcastro]

Hi @joshuarobs Could you share which repository you managed to fix the issue? Since you shared 2 of them I'm not quite sure which one is the one working.

Thanks.

[joshuarobs]

@samuelcastro
https://github.com/joshuarobs/test-ci

[samuelcastro]

Thanks @joshuarobs, quick question, do you know what's the main difference between these 02 different Docker setup? I've posted a question here as well: #3954

Thanks.

[joshuarobs]

@samuelcastro one works and the other doesn't

[browniefed]

@joshuarobs Did you ever get this working? I am stuck on this.

Some added complexity since I've got remote schema too. But running into the same versions issues.
Used your example from your repo, as well as GavinRay97/[email protected] and other actions to apply meta/migrations but no ball.

CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS                   PORTS                    NAMES
b238fcaf3aff        hasura/graphql-engine:v1.3.2   "graphql-engine serve"   3 minutes ago       Up 3 minutes             0.0.0.0:8080->8080/tcp   8db1f60b091e4af9a04176ecad8bd439_hasuragraphqlenginev132_8ac0cb
e561bc0482fe        postgres:12.1                  "docker-entrypoint.s…"   4 minutes ago       Up 3 minutes (healthy)   0.0.0.0:5432->5432/tcp   bcbf762223614507aee0095f81cbcf78_postgres121_53bfb7


ℹ️ Downloading Hasura CLI v1.3.2
,ℹ️ Making Hasura CLI executable
,ℹ️ cd'ing to Hasura project root at ./hasura
,ℹ️ Preparing to apply migrations and metadata
,ℹ️ Migration status:
,{"level":"info","msg":"Help us improve Hasura! The cli collects anonymized usage stats which\nallow us to keep improving Hasura at warp speed. To opt-out or read more,\nvisit https://hasura.io/docs/1.0/graphql/manual/guides/telemetry.html\n","time":"2020-11-03T20:38:31Z"}
{"level":"info","msg":"Checking for update... ","time":"2020-11-03T20:38:31Z"}
{"level":"info","msg":"hasura cli is up to date","time":"2020-11-03T20:38:32Z","version":"1.3.2"}
time="2020-11-03T20:38:33Z" level=fatal msg="version check: failed to get version from server: failed making version api call: Get http://localhost:8080/v1/version: dial tcp 127.0.0.1:8080: connect: connection refused"
❌ Failed getting migration status

[barryskal]

@joshuarobs Did you ever get this working? I am stuck on this.

@browniefed are you still stuck on this? I was working on this for a while as well and I've been using the github action that you wrote as part of my tests. It turns out that this was the issue. When I use your action I get the same message where it says that it cannot connect to 127.0.0.1. When I switch to just directly installing the hasura CLI in the action and calling it directly as @joshuarobs has done it worked immediately.
I think the root cause is probably that the action runs the hasura CLI in a docker container; however, the container is not attached to the same docker network so it cannot communicate with hasura. If I use the CLI directly it can communicate with hasura via localhost so the problem goes away.

[rachel-church]

Thanks for updating with your solution @joshuarobs! I just got Hasura working in a Gitlab CI/CD pipeline with the following configuration. Sharing for anyone who stumbles upon this issue after facing a making http request failed: Get \"http://localhost:8080/v1/version\": dial tcp 172.18.0.3:80: connect: connection refused error or similar in Gitlab.

test-hasura-migrations:
  image: debian:12-slim
  services:
    - name: postgres
      alias: postgres
    - name: hasura/graphql-engine:v2.35.1
      alias: graphql-engine # this is the hostname where the service can be accessed
  variables:
    FF_NETWORK_PER_BUILD: 1 # see https://docs.gitlab.com/ee/ci/services/#connecting-services
    POSTGRES_PASSWORD: "postgrespassword"
    POSTGRES_USER: postgres
    POSTGRES_DB: postgres
    HASURA_GRAPHQL_DATABASE_URL: postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@postgres/$POSTGRES_DB
    PG_DATABASE_URL: postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@postgres/$POSTGRES_DB
    HASURA_GRAPHQL_ENDPOINT: http://graphql-engine:8080
  script:
    - apt update && apt upgrade --yes
    - apt install curl --yes
    - curl -L https://github.com/hasura/graphql-engine/raw/stable/cli/get.sh | bash
    # Apply the migrations to the fresh DB
    - cd hasura
    - hasura metadata apply # at this point, the metadata will be inconsistent, since migrations have not been applied yet
    - hasura migrate apply --database-name $HASURA_DATABASE_NAME
    - hasura migrate status --database-name $HASURA_DATABASE_NAME
    - hasura seeds apply --database-name $HASURA_DATABASE_NAME
    - hasura metadata reload # hasura will check again and metadata should now be consistent with database information.
    - |
      output=$(hasura metadata inconsistency status 2>&1)
      if [[ ! $output =~ "metadata is consistent" ]]; then
        hasura metadata ic list
        exit 1
      fi
  rules:
    # Automatically run on any merge request that updates the hasura directory
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      changes:
        - hasura/migrations/**/*
        - hasura/metadata/**/*
        - hasura/seeds/**/*
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      when: manual # If the hasura files haven't changed allow the user to choose to manually run it
      allow_failure: true # the pipeline continues running even if the manual job is not run