Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use images from docker_base_images repository #34

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"build": {
"dockerfile": "Dockerfile",
"args": {
"docker_image_tag_hash": "b5b06ad736cc8f2df227dabf8ff1da1a2f538939"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An idea could be to also have a 22.04-b5b....939?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean to avoid the long hash?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know how you generate this hash. It looks to me that you use one unique image for all your builds?
In geos, we "play" with the repository name (like ubuntu-22-gcc10...) and the hash is unique and matches versions of the TPLs. It depends on what you want to do for Shiva. Still, it's no big deal to make this evolve in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the hash is from the docker_base_images repository. I would like to standardize the images we use for testing across the project repos.

}
},
"runArgs": [
"--cap-add=SYS_PTRACE",
"--security-opt",
"seccomp=unconfined"
],
"customizations": {
"vscode": {
"extensions": [
"eamodio.gitlens",
"fredericbonnet.cmake-test-adapter",
"GitHub.vscode-github-actions",
"GitHub.vscode-pull-request-github",
"ms-vscode.cpptools-extension-pack",
"redhat.vscode-xml"
]
},
Comment on lines +13 to +23
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this file is mainly if you're using codespaces, which is still not your case?
If you want, you can simplify it a lot, or simply embed the b5...39 hash into your continuousIntegration.yml with some echo "DOCKER_IMAGE_TAG_HASH=b5...39" >> "$GITHUB_OUTPUT".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would want to enable codespaces even though I don't personally use it now....I would like to try it soon.

Copy link
Collaborator

@TotoGaz TotoGaz Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it believe it's a great feature at low (setup and maintenance) cost!

"codespaces": {
"repositories": {
},
"openFiles": []
}
},
"features": {
"ghcr.io/devcontainers/features/sshd:1": {
"version": "latest"
}
},
"postCreateCommand": "bash .devcontainer/postCreateCommand.sh"
}
148 changes: 87 additions & 61 deletions .github/workflows/continuousIntegration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,25 @@ concurrency:

jobs:

get_docker_image_tag_hash:
runs-on: ubuntu-22.04
outputs:
DOCKER_IMAGE_TAG_HASH: ${{ steps.extract_docker_image_tag_hash.outputs.DOCKER_IMAGE_TAG_HASH }}
steps:
- name: Checkout .devcontainer/devcontainer.json
uses: actions/[email protected]
with:
sparse-checkout: |
.devcontainer/devcontainer.json
sparse-checkout-cone-mode: false
submodules: false
lfs: false
fetch-depth: 1
- name: Extract docker image tag hash
id: extract_docker_image_tag_hash
run: |
echo "DOCKER_IMAGE_TAG_HASH=$(jq '.build.args.docker_image_tag_hash' -r .devcontainer/devcontainer.json)" >> "$GITHUB_OUTPUT"

# code_style:
# needs: [check_pull_request_is_not_a_draft]
# runs-on: ubuntu-22.04
Expand All @@ -30,44 +49,50 @@ jobs:

doxygen_check:
runs-on: ubuntu-22.04
needs:
- get_docker_image_tag_hash
steps:
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: Check doxygen
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/clang++
CMAKE_C_COMPILER: /usr/bin/clang
CMAKE_BUILD_TYPE: Release
BUILD_AND_TEST_ARGS: --test-doxygen
run: ./scripts/ci_build_and_test.sh
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: Check doxygen
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04-${{ needs.get_docker_image_tag_hash.outputs.DOCKER_IMAGE_TAG_HASH }}
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/clang++
CMAKE_C_COMPILER: /usr/bin/clang
CMAKE_BUILD_TYPE: Release
BUILD_AND_TEST_ARGS: --test-doxygen
run: ./scripts/ci_build_and_test.sh


code_checks:
runs-on: ubuntu-22.04
steps:
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: code_checks
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/clang++
CMAKE_C_COMPILER: /usr/bin/clang
CMAKE_BUILD_TYPE: Release
BUILD_AND_TEST_ARGS: --code-checks
run: ./scripts/ci_build_and_test.sh
runs-on: ubuntu-22.04
needs:
- get_docker_image_tag_hash
steps:
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: code_checks
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04-${{ needs.get_docker_image_tag_hash.outputs.DOCKER_IMAGE_TAG_HASH }}
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/clang++
CMAKE_C_COMPILER: /usr/bin/clang
CMAKE_BUILD_TYPE: Release
BUILD_AND_TEST_ARGS: --code-checks
run: ./scripts/ci_build_and_test.sh

linux_builds:
name: ${{matrix.name}}
runs-on: ${{matrix.RUNS_ON}}
needs:
- get_docker_image_tag_hash
strategy:
# Set fail-fast to false to ensure that feedback is delivered for all matrix combinations. Consider changing this to true when your workflow is stable.
fail-fast: false
Expand Down Expand Up @@ -174,7 +199,7 @@ jobs:
DOCKER_RUN_ARGS: "--cpus=4 --memory=64g --runtime=nvidia --gpus all"

- name: RHEL8.9-clang17-cuda12-rel (ubi8.9, clang-17.0.6, cuda-12.4.1, Release)
DOCKER_REPOSITORY: geosx/ubi:8.9-cuda12.4.1
DOCKER_REPOSITORY: geosx/ubi:8.9-cuda12.4
RUNS_ON: streak2
CMAKE_CXX_COMPILER: clang++
CMAKE_C_COMPILER: clang
Expand All @@ -185,7 +210,7 @@ jobs:
DOCKER_RUN_ARGS: "--cpus=4 --memory=64g --runtime=nvidia --gpus all"

- name: RHEL8.9-gcc13-cuda12-rel (ubi8.8, gcc 13.2.1, cuda-12.4.1, Release)
DOCKER_REPOSITORY: geosx/ubi:8.9-cuda12.4.1
DOCKER_REPOSITORY: geosx/ubi:8.9-cuda12.4
RUNS_ON: streak2
CMAKE_CXX_COMPILER: /opt/rh/gcc-toolset-13/root/bin/g++
CMAKE_C_COMPILER: /opt/rh/gcc-toolset-13/root/bin/gcc
Expand All @@ -207,7 +232,7 @@ jobs:

- name: Build and test
env:
DOCKER_REPOSITORY: ${{ matrix.DOCKER_REPOSITORY }}
DOCKER_REPOSITORY: ${{ matrix.DOCKER_REPOSITORY }}-${{ needs.get_docker_image_tag_hash.outputs.DOCKER_IMAGE_TAG_HASH }}
CMAKE_CXX_COMPILER: ${{ matrix.CMAKE_CXX_COMPILER }}
CMAKE_C_COMPILER: ${{ matrix.CMAKE_C_COMPILER }}
CMAKE_BUILD_TYPE: ${{ matrix.CMAKE_BUILD_TYPE }}
Expand All @@ -220,39 +245,40 @@ jobs:
run: ./scripts/ci_build_and_test.sh

code_coverage:
runs-on: ubuntu-22.04
# needs:
# - linux_builds
steps:
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: run code coverage
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/g++
CMAKE_C_COMPILER: /usr/bin/gcc
CMAKE_BUILD_TYPE: Debug
BUILD_AND_TEST_ARGS: "--build-exe --code-coverage"
run: ./scripts/ci_build_and_test.sh
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
files: shiva_coverage.info.cleaned
fail_ci_if_error: true
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
runs-on: ubuntu-22.04
needs:
- get_docker_image_tag_hash
steps:
- name: Checkout Repository
uses: actions/[email protected]
with:
submodules: true
lfs: false
- name: run code coverage
env:
DOCKER_REPOSITORY: geosx/ubuntu:22.04-${{ needs.get_docker_image_tag_hash.outputs.DOCKER_IMAGE_TAG_HASH }}
HOST_CONFIG: hostconfigs/environment.cmake
CMAKE_CXX_COMPILER: /usr/bin/g++
CMAKE_C_COMPILER: /usr/bin/gcc
CMAKE_BUILD_TYPE: Debug
BUILD_AND_TEST_ARGS: "--build-exe --code-coverage"
run: ./scripts/ci_build_and_test.sh
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
files: shiva_coverage.info.cleaned
fail_ci_if_error: true
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}


check_that_all_jobs_succeeded:
runs-on: ubuntu-22.04
needs:
- linux_builds
- doxygen_check
- code_checks
- get_docker_image_tag_hash
- linux_builds
- doxygen_check
- code_checks
if: ${{ always() }}
env:
RETURN_VAL: |
Expand Down
25 changes: 0 additions & 25 deletions docker/buildAndPush.bash

This file was deleted.

16 changes: 0 additions & 16 deletions docker/ubi-cuda11/Dockerfile

This file was deleted.

17 changes: 0 additions & 17 deletions docker/ubi-cuda12/Dockerfile

This file was deleted.

14 changes: 0 additions & 14 deletions docker/ubi/Dockerfile

This file was deleted.

16 changes: 0 additions & 16 deletions docker/ubuntu-cuda11/Dockerfile

This file was deleted.

21 changes: 0 additions & 21 deletions docker/ubuntu/Dockerfile

This file was deleted.

Loading