Skip to content

Parser for the Nix Expressions Language

License

Notifications You must be signed in to change notification settings

kamadorueda/nixel

Repository files navigation

🐉 NixEL

Parser for the Nix Expressions Language.

CI/CD Documentation Coverage Version License

Features

  • ✔️ Fast

    It parses all the files in Nixpkgs in under 25 seconds, single-threaded. 1

    It's written in Rust and a little bit of C++, Flex and GNU Bison.

  • ✔️ Correct

    This library is a copy-paste of the original lexer and parser of Nix, with some types adapted for better ergonomy.

    No parser can get closer to the original implementation than this.

  • ✔️ Reliable

    High coverage, battle-tested, and memory-safe2.

  • ✔️ Useful

    It gives you comments, whitespace, starting and end positions, automatic string un-escaping, multiline string indentation handling, a typed API, and everything you need to parse the Nix language!

Usage

You can check out the documentation at docs.rs/nixel.

This is a full usage example:

let input: String = String::from(
    r#"
        # Greet the user
        "Hello, World!"
        # Bye!
    "#,
);

let parsed: nixel::Parsed = nixel::parse(input);

match &*parsed.expression {
    nixel::Expression::String(string) => {
        assert_eq!(
            &string.span,
            &nixel::Span {
                start: nixel::Position { line: 3, column: 9 }.into(),
                end: nixel::Position { line: 3, column: 24 }.into(),
            }
            .into()
        );
        assert_eq!(
            &parsed.trivia_before(&string.span.start)[1],
            &nixel::Trivia::Comment(nixel::TriviaComment {
                content: "# Greet the user".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 2, column: 9 }.into(),
                    end: nixel::Position { line: 2, column: 25 }.into(),
                }
                .into()
            })
        );
        assert_eq!(
            &string.parts[0],
            &nixel::Part::Raw(nixel::PartRaw {
                content: "Hello, World!".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 3, column: 10 }.into(),
                    end: nixel::Position { line: 3, column: 23 }.into(),
                }
                .into()
            })
        );
        assert_eq!(
            &parsed.trivia_after(&string.span.end)[1],
            &nixel::Trivia::Comment(nixel::TriviaComment {
                content: "# Bye!".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 4, column: 9 }.into(),
                    end: nixel::Position { line: 4, column: 15 }.into(),
                }
                .into()
            })
        );
    },
    expression => unreachable!("Expected a String, got: {expression:#?}"),
}

Or from the CLI using Rust's Debug trait:

$ echo '1 + 2' | nix run github:kamadorueda/nixel -- --format=debug

BinaryOperation(
    BinaryOperation {
        left: Integer(
            Integer {
                value: "1",
                span: Span {
                    start: Position {
                        line: 1,
                        column: 1,
                    },
                    end: Position {
                        line: 1,
                        column: 2,
                    },
                },
            },
        ),
        operator: Addition,
        right: Integer(
            Integer {
                value: "2",
                span: Span {
                    start: Position {
                        line: 1,
                        column: 5,
                    },
                    end: Position {
                        line: 1,
                        column: 6,
                    },
                },
            },
        ),
    },
)

Or from the CLI using JSON format:

$ echo '1 + 2' | nix run github:kamadorueda/nixel -- --format=json

{
  "BinaryOperation": {
    "left": {
      "Integer": {
        "value": "1",
        "span": {
          "start": {
            "line": 1,
            "column": 1
          },
          "end": {
            "line": 1,
            "column": 2
          }
        }
      }
    },
    "operator": "Addition",
    "right": {
      "Integer": {
        "value": "2",
        "span": {
          "start": {
            "line": 1,
            "column": 5
          },
          "end": {
            "line": 1,
            "column": 6
          }
        }
      }
    }
  }
}

You can check out more examples in the tests folder.

Alternatives

License

Please read LICENSE.md.

Footnotes

Footnotes

  1. Running on a machine with:

    • CPU: 4 physical, 4 logical, 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
    • MHz: from 400 to 4700 MHz
    • BogoMips: 5606.40
    • Cache L3: 12 MiB

    The following command takes around 1 minute:

    $ nix build --system x86_64-linux
    $ time find /data/nixpkgs -type f -name '*.nix' \
      -exec ./result/bin/nixel --format=none {} \;
    
    real  0m24.293s
    user  0m15.066s
    sys   0m8.955s
    
  2. Tested under real-life workloads using Valgrind, and by running an infinite loop of parsing cycles over Nixpkgs :).

    $ nix build --system x86_64-linux
    $ valgrind ./result/bin/nixel $file
    
      LEAK SUMMARY:
        definitely lost: 0 bytes in 0 blocks
        indirectly lost: 0 bytes in 0 blocks
          possibly lost: 0 bytes in 0 blocks
             suppressed: 0 bytes in 0 blocks