You can install the necessary components directly from official sources:
For precise control over versions, it's recommended to use the asdf tool version manager and follow the versions specified in .tool-versions in this repository.
After installing asdf, add the required plugins for managing the tools:
asdf plugin add elixir
asdf plugin add erlang
asdf plugin add golang
asdf plugin add rust
asdf plugin add protocFinally, install the specific versions of these tools as specified in .tool-versions:
asdf installTo create a sandbox environment with all the required tool chains, use Nix. Steps to get Nix working are as follows:
- Install Nix from the official website: https://nixos.org/download.
- To allow experimental features (nix develop and nix-command) you might need to do the following:
mkdir ~/.config/nix
echo "experimental-features = nix-command flakes " > ~/.config/nix/nix.confAlternatively, for a smoother experience you can use the following script from Determinate Systems that takes care of setting up everything for you:
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install- Check if Nix has been successfully installed:
nix --version. - To launch the environment:
nix develop.
There are Makefile targets for these tasks.
Tip
You can list the available targets with make help
make deps # Installs dependencies
make iex # Runs a terminal with the application startedThe iex terminal can be closed by pressing ctrl+c two times.
Warning
The node isn't capable of syncing from genesis yet, and so requires using checkpoint-sync to start (see Checkpoint Sync).
In case checkpoint-sync is needed, make iex will end immediately with an error.
You can also sync from a checkpoint given by a trusted third-party. You can specify a URL to fetch it from with the "--checkpoint-sync-url" flag:
iex -S mix run -- --checkpoint-sync-url <your_url_here>Some public endpoints can be found in eth-clients.github.io/checkpoint-sync-endpoints.
Important
The data retrieved from the URL is stored in the DB once the node is initiated (i.e. the iex prompt shows).
Once this happens, following runs of make iex will start the node using that data.
You can start the application with the Beacon API on the default port 4000 running:
make startYou can also specify a port with the "--beacon-api-port" flag:
iex -S mix run -- --beacon-api --beacon-api-port <your_port_here>Warning
In case checkpoint-sync is needed, following the instructions above will end immediately with an error (see Checkpoint Sync).
Our CI runs tests, linters, and also checks formatting and typing. To run these checks locally:
make test # Runs tests
make spec-test # Runs all spec-tests
make lint # Runs linter and format-checker
make dialyzer # Runs type-checkerSource code can be formatted using make fmt.
This formats not only the Elixir code, but also the code under native/.
The repo includes a Dockerfile for the consensus client. It can be built with:
docker build -t consensus .Then you run it with docker run, adding CLI flags as needed:
docker run consensus --checkpoint-sync <url> --network <network> ...You can run all of them with:
make spec-testOr only run those of a specific config with:
make spec-test-config-`config`
# Some examples
make spec-test-config-mainnet
make spec-test-config-minimal
make spec-test-config-generalOr by a single runner in all configs, with:
make spec-test-runner-`runner`
# Some examples
make spec-test-runner-ssz_static
make spec-test-runner-bls
make spec-test-runner-operationsThe complete list of test runners can be found here.
If you want to specify both a config and a runner:
make spec-test-mainnet-operations
make spec-test-minimal-epoch_processing
make spec-test-general-blsMore advanced filtering (e.g. by fork or handler) will be re-added again, but if you want to only run a specific test, you can always do that manually with:
mix test --no-start test/generated/<config>/<fork>/<runner>.exs:<line_of_your_testcase>You can put a "*" in any directory (e.g. config) you don't want to filter by, although that won't work if adding the line of the testcase.
Note
We specify the --no-start flag to stop ExUnit from starting the application, to reduce resource consumption.
Elixir is a functional programming language that runs atop the Erlang Virtual Machine (BEAM). It offers enhanced readability, syntactic sugar, and reduced boilerplate, enabling developers to achieve more with fewer lines of code compared to Erlang. Like Erlang, Elixir compiles to bytecode that is interpreted by the VM. As a result, it inherits several notable properties, including:
- Fault tolerance for increased reliability
- High availability
- Simplified construction of complex distributed systems
- Predictable latency
Erlang and its VM were originally developed in 1986 for telecommunication systems that demanded unparalleled uptime and reliability. We recognize that these attributes could be immensely beneficial for an Ethereum client, particularly in the realm of consensus. This is why our current focus is on building a consensus layer (CL) rather than an execution layer (EL). Elixir may not be tailored for sheer performance, but it excels in delivering predictable latency and creating systems designed for continuous operation—qualities essential for the CL.
Our aim is to infuse these strengths into the Ethereum consensus client ecosystem with our offering.
We also have for objective to bootstart an Ethereum Elixir community, and to make Elixir a first-class citizen in the Ethereum ecosystem.
Dream of becoming an Ethereum core developer? Eager to shape the protocol that will underpin tomorrow's world? Want to collaborate with a passionate team, learn, grow, and be a pivotal part of the Ethereum Elixir community?
Then you're in the right place! 🚀
-
Prerequisites: Before diving in, ensure you have the necessary tools installed. Check out the Prerequisites section for guidance.
-
Clone the Repository:
git clone [REPO_URL] cd lambda_ethereum_consensus -
Setup: Once you've cloned the repository, follow the steps in the Installing and running section to set up your environment.
To contribute effectively, you'll need a foundational understanding of both the Ethereum protocol and the Elixir language, including the Erlang VM (BEAM). If you're new to these areas, we've curated a list of resources to get you started:
Learning Elixir:
- Videos:
- Blogs:
- Books:
With this foundation you should have a basic understanding of the Elixir language and the Erlang VM. You can then start (or in parallel) learning about the Ethereum protocol.
Learning Ethereum:
- Videos:
- Posts
- Books:
- Blogs:
- Papers:
Learning Ethereum Consensus:
- Books:
- Eth2Book by Ben Edgington. This book is indispensable for understanding the Ethereum consensus protocol. If you can read only one thing, read this.
- Specifications:
While some of the resources listed might appear outdated, it's important to understand that the Ethereum protocol is continuously evolving. As such, there isn't a definitive, unchanging source of information. However, these resources, even if older, provide foundational knowledge that remains pertinent to understanding the protocol's core concepts.
Truly mastering the Ethereum protocol is a complex endeavor. The list provided here is just a starting point, and delving deeper will necessitate exploring a broader range of readings and resources. As you immerse yourself in the project, continuous learning and adaptation will be key.
If you come across any resource that you find invaluable and believe should be added to this list, please don't hesitate to suggest its inclusion.
With your newfound knowledge, explore the various areas of our project. Whether you're interested in the core consensus layer, networking, CLI, documentation, testing, or tooling, there's a place for you.
Start by browsing our issues, especially those tagged as good first issue. These are beginner-friendly and a great way to familiarize yourself with our codebase.
Found an issue you're passionate about? Comment with "I'd like to tackle this!" to claim it. Once assigned, you can begin your work. After completing your contribution, submit a pull request for review. Our team and other contributors will be able to provide feedback, and once approved, your contribution will be merged.
Please adhere to the Conventional Commits specification when crafting PR titles. Also, run make fmt to format source code according to the configured style guide. The repo enforces these automatically via GitHub Actions.
Important
We believe in fostering an inclusive, welcoming, and respectful community. All contributors are expected to adhere to our Code of Conduct. Please familiarize yourself with its contents before participating.
Open communication is key to the success of any project. We encourage all contributors to join our Telegram chat for real-time discussions, updates, and collaboration.
For more structured discussions or proposals, consider opening an issue or a discussion on the GitHub repository.
We value every contribution, no matter how small. All contributors will be recognized in our project's documentation. Additionally, consistent and significant contributors may be offered more formal roles within the project over time.
If you encounter any issues or have questions, don't hesitate to reach out. Our team and the community are here to help. You can ask questions in our Telegram chat or open an issue on GitHub for technical challenges.
Lambda Ethereum Consensus is more than just a project; it's a community-driven initiative to bring the power and reliability of Elixir to the Ethereum ecosystem. With your help, we can make this vision a reality. Dive in, contribute, learn, and let's shape the future of Ethereum together!
Thank you for being a part of our journey. Let's build an amazing future for Ethereum together! 🚀🌍
When running the node, metrics are available at http://localhost:9568/metrics in Prometheus format.
A docker-compose is available at metrics/ with a Grafana-Prometheus setup preloaded with dashboards that disponibilize the data.
To run it, install Docker Compose and execute:
make grafana-upAfter that, open http://localhost:3000/ in a browser.
The default username and password are both admin.
To stop the containers run make grafana-down. For cleaning up the metrics data, run make grafana-clean.
To install QCachegrind via Homebrew, run:
brew install qcachegrindTo build a qcachegrind profile, run, inside iex:
LambdaEthereumConsensus.Profile.build()Options and details are in the Profile package. After the profile trace is generated, you open it in qcachegrind with:
qcachegrind callgrind.out.<trace_name>If you want to group the traces by function instead of process, you can use the following before viewing it in qcachegrind:
grep -v "^ob=" callgrind.out.trace_name > callgrind.out.merged.trace_nameAnother useful tool to quickly diagnose processes taking too much CPU is :etop, similar to UNIX top command. This is installed by default in erlang, and included in the :observer extra application in mix.exs. You can run it with:
:etop.start()In particular, the reds metric symbolizes reductions, which can roughly be interpreted as the number of calls a function got.
This can be used to identify infinite loops or busy waits.
Also of note is the :sort option, that allows sorting the list by, for example, message queue size:
:etop.start(sort: :msg_q)When optimizing code, it might be useful to have a graphic way to determine bottlenecks in the system.
In that case, you can use eFlambè to generate flamegraphs of specific functions.
The following code will capture information from 10 calls to Handlers.on_block/2, dumping it in different files named <timestamp>-eflambe-output.bggg.
:eflambe.capture({LambdaEthereumConsensus.ForkChoice.Handlers, :has_block?, 2}, 10)The files generated can be processed via common flamegraph tools. For example, using Brendan Gregg's stack:
cat *-eflambe-output.bggg | flamegraph.pl - > flamegraph.svgWe, as members, contributors, and leaders of open source communities and projects pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community and project.
Examples of behavior that contributes to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people.
- Being respectful of differing opinions, viewpoints, and experiences.
- Giving and gracefully accepting constructive feedback.
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience.
- Focusing on what is best not just for us as individuals, but for the overall community and project.
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind.
- Trolling, insulting or derogatory comments, and personal or political attacks.
- Public or private harassment.
- Publishing others' private information, such as a physical or electronic address, without their explicit permission.
- Other conduct which could reasonably be considered inappropriate in a professional setting.
Maintainers are responsible for clarifying and enforcing standards of acceptable behavior and will take appropriate and fair corrective action.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for behaviors that they deem inappropriate, threatening, offensive, or harmful.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported with proof to the maintainers through Telegram. All complaints will be reviewed and investigated promptly, fairly and anonymously.
This Code of Conduct is adapted from the Contributor Covenant, version 2.1.
