Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing MapReduce from Hadoop causes demo job to fail #793

Open
maltesander opened this issue Jul 29, 2024 · 0 comments
Open

Removing MapReduce from Hadoop causes demo job to fail #793

maltesander opened this issue Jul 29, 2024 · 0 comments

Comments

@maltesander
Copy link
Member

In the hbase-hdfs-load-cycling-data we use a distcp command to copy data in the demo.

This worked prior to 24.7 release, since then we removed the map reduce component which makes this job fail.

Error: Unable to initialize main class org.apache.hadoop.tools.DistCp
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/Job

Solution in the demo is fallback on a 24.3 image, but we need a general solution in terms of shipping with/without map reduce or provide e.g. custom images for demos.

github-merge-queue bot pushed a commit to stackabletech/demos that referenced this issue Nov 15, 2024
* chore(demo/hbase-hdfs-load-cycling-data): Tidy up job scripts

* fix(demo/hbase-hdfs-load-cycling-data): Revert to 24.3 image for distcp (stackabletech/docker-images#793)

* docs(demo/hbase-hdfs-load-cycling-data): Add hints about the active namenode for browsing files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant