Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixos/hadoop: package rewrite and module improvements #141143

Merged
merged 3 commits into from
Oct 25, 2021
Merged

nixos/hadoop: package rewrite and module improvements #141143

merged 3 commits into from
Oct 25, 2021

Commits on Oct 22, 2021

  1. hadoop: 2->3 

    - Make hadoop 3 the default hadoop package
- hadoop_3 -> 3.3.1
- hadoop_3_2 -> 3..2.2
- hadoop_2 -> 2.10.1
    @illustris
    illustris committed Oct 22, 2021
    Configuration menu
    Copy the full SHA
    bf84591 View commit details
    Browse the repository at this point in the history

  2. nixos/hadoop: fix yarn, add more service configuration options

    @illustris
    illustris committed Oct 22, 2021
    Configuration menu
    Copy the full SHA
    28e0bad View commit details
    Browse the repository at this point in the history

  3. nixos/hadoop: add better test 

    The existing tests for HDFS and YARN only check if the services come up and expose their web interfaces.
The new combined hadoop test will also test whether the services and roles work together as intended.
It spin up an HDFS+YARN cluster and submit a demo YARN application that uses the hadoop cluster for
storageand yarn cluster for compute.
    @illustris
    illustris committed Oct 22, 2021
    Configuration menu
    Copy the full SHA
    af49da2 View commit details
    Browse the repository at this point in the history