Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support builds that are reproducible and do not require network access #3369

Open
philipmw opened this issue Jul 23, 2022 · 5 comments
Open

Comments

@philipmw
Copy link
Contributor

Description

I want to build Lighthouse for NixOS. This requires that Lighthouse builds in a reproducible way without needing network access.

Version

2.4.0

Present Behaviour

In several places of the code, Lighthouse downloads files via HTTPS during build.

Expected Behaviour

Lighthouse should support finding the needed files on the filesystem (placed there by the packager) without using the network.

Steps to resolve

Follow the pattern in deposit_contract, which supports specifying a filename via an environment variable.

philipmw added a commit to philipmw/lighthouse that referenced this issue Jul 23, 2022
The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
@ralexstokes
Copy link
Contributor

I had to work through the same issue and this was my solution:

https://gist.github.com/ralexstokes/e9e123ccc9386ad6de1470e3c9011d99

it just applies the result of what the network request would output

I think the output is static which suggests this could even be hardcoded into the repo -- I'd ask the maintainers to reconsider this aspect of the build at some point because it does complicate the build

philipmw added a commit to philipmw/lighthouse that referenced this issue Jul 26, 2022
The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
philipmw added a commit to philipmw/lighthouse that referenced this issue Jul 26, 2022
The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
philipmw added a commit to philipmw/lighthouse that referenced this issue Jul 26, 2022
The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
@philipmw
Copy link
Contributor Author

That's a neat approach. Is anything keeping you from contributing this to Nixpkgs?

philipmw added a commit to philipmw/lighthouse that referenced this issue Jul 26, 2022
The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
@ralexstokes
Copy link
Contributor

Is anything keeping you from contributing this to Nixpkgs?

lack of knowledge and somewhat time -- do I just make a PR to that repo? any links to docs etc? I'm relatively new to Nix world

@philipmw
Copy link
Contributor Author

You already did the hard part! Now the easy part is to just make a pull request to the Nixpkgs repo. Here's a chapter in the NixOS manual that describes the process: https://nixos.org/manual/nixpkgs/stable/#chap-submitting-changes

One difference I noticed between my approach and yours is that I am building the full package, while you seem to be omitting tests. Did you run into any particular difficulties patching the tests, or just didn't bother?

bors bot pushed a commit that referenced this issue Jul 27, 2022
## Issue Addressed

#3369 

## Proposed Changes

The goal is to make it possible to build Lighthouse without network access,
so builds can be reproducible.

This parallels the existing functionality in `common/deposit_contract/build.rs`,
which allows specifying a filename through the environment to avoid downloading
it. In this case, by specifying the version and making it available on the
filesystem, the existing logic will avoid a network download.
@ralexstokes
Copy link
Contributor

Did you run into any particular difficulties patching the tests, or just didn't bother?

yeah I skipped the stuff I did not need, feel free to take this and run w/ it if you want

if you improve what I had, I'd love to see it at some point!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants