Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build docker images using nix #654

Merged
merged 12 commits into from
Dec 21, 2022
Merged

Build docker images using nix #654

merged 12 commits into from
Dec 21, 2022

Conversation

ch1bo
Copy link
Collaborator

@ch1bo ch1bo commented Dec 14, 2022

This is a follow-up of #646 (which is a follow-up of #625)

☃️ Build our docker images directly using nix instead of "nix inside docker". This will allow us to re-use the nix store better after a build / between builds.

☃️ Keep the docker workflow as is (for now) to do the tagging / labelling as before (with a small workaround).

☃️ The images now only include what is needed and don't have a base (before it was alpine) anymore. Image sizes went down between 20-50%:

image

☃️ NOTE: Building & pushing images from tags is untested and we'll need to see if it works on our next release.

To check before merging:

  • CHANGELOG is up to date
  • Up to date with master

@github-actions
Copy link

github-actions bot commented Dec 14, 2022

Test Results

269 tests  ±0   263 ✔️ ±0   13m 4s ⏱️ - 1m 35s
  92 suites ±0       6 💤 ±0 
    4 files   ±0       0 ±0 

Results for commit 0d54918. ± Comparison against base commit 03f64a7.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 15, 2022

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2022-12-21 07:11:53.185933804 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4832 12.11 4.83 0.50
2 5042 13.05 5.16 0.51
3 5242 13.71 5.38 0.53
5 5656 19.64 7.72 0.61
10 6677 27.85 10.83 0.75
46 14059 99.47 38.44 1.85

Cost of Commit Transaction

Currently only one UTxO per commit allowed (this is about to change soon)

UTxO Tx size % max Mem % max CPU Min fee ₳
1 576 20.63 8.37 0.40

Cost of CollectCom Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 13306 20.68 8.29 0.96
2 13697 37.14 15.05 1.16
3 13915 54.33 22.16 1.36
4 14342 77.80 31.91 1.64
5 14556 99.96 41.17 1.90

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 9581 8.36 3.42 0.66
2 9782 9.53 4.03 0.69
3 9943 10.50 4.55 0.71
5 10275 11.92 5.40 0.74
10 11133 16.21 7.81 0.83
30 14437 31.99 16.91 1.17
69 16355 40.83 15.45 1.31

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 9576 8.17 3.34 0.66
2 9743 8.94 3.79 0.68
3 9941 10.17 4.42 0.70
5 10266 11.91 5.39 0.74
10 11130 16.02 7.73 0.83
30 14433 31.66 16.77 1.16
41 16291 41.37 22.18 1.36

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 13953 25.73 10.91 1.05
2 14278 42.95 18.87 1.26
3 14247 54.14 23.17 1.38
4 14572 77.66 33.97 1.66
5 14608 95.27 41.49 1.86

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 13728 10.26 4.46 0.87
2 13699 11.40 5.19 0.88
3 13864 13.93 6.47 0.92
5 13807 16.26 7.95 0.95
10 14120 24.74 12.69 1.06
50 15431 84.50 47.45 1.85
60 15788 99.75 56.26 2.06

@ch1bo ch1bo force-pushed the ch1bo/nix-docker-builds branch 6 times, most recently from 67b0e2f to 9f12208 Compare December 19, 2022 17:11
This yields smaller closures and hence smaller docker images
This seems to still work as the nix closure for hydra-tui was correctly
including the terminfo files etc.
The workflow will use a dummy Dockerfile to leverage the tagging /
labelling without actually rebuilding the image.
@ch1bo ch1bo force-pushed the ch1bo/nix-docker-builds branch 5 times, most recently from 17a6c50 to f42ff75 Compare December 19, 2022 17:29
As we are not using Dockerfiles, this seems to be the only way how to
tag/label images when using nix to build.
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
docker push ghcr.io/input-output-hk/${{matrix.target}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before we were using ${{ github.repository_owner }} instead of input-output-hk

};
in
rec {
packages = hydraPackages // {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does // means?

is it like the set difference operation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the opposite, it's a union and thus merges the argument set on the right hand side to the one on the left hand side.

docker inspect ghcr.io/input-output-hk/${{matrix.target}}:unstable

- name: 📤 Maybe push
if: ${{ github.ref == 'refs/heads/master' || (github.ref_type == 'tag' && matrix.target != 'hydraw') }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe 'hydraw' can defined inside an exclusion list?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the single exclusion in our repo. The target list is (and always will be) very specific to the project. I don't like to over generalize the job (it becomes harder to read & maintain IMO).

@ch1bo ch1bo merged commit 171a9d7 into master Dec 21, 2022
@ch1bo ch1bo deleted the ch1bo/nix-docker-builds branch December 21, 2022 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants