-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ONeTable with MINIO Buckets #327
Comments
related: #322 |
@alberttwong did you get it to work with uneatable ? |
still same issue @alberttwong
|
@sagarlakshmipathy is there something we're missing? I feel like the java app doesn't pick up ENV. |
|
okay. I figured it out. You need to modify utilities/src/main/resources/onetable-hadoop-defaults.xml to include additional configs. #337 We need to clean this up so that onetable scans for conf files. also we need to modify the trino schema create command. See StarRocks/demo#54 for all the instructions. |
That's the issue... none of them worked so I'm just listing all the variants I tried. what worked was only for my own demo and I modified my hadoop settings. I couldn't get it working for onetable's docker demo. |
understood let me post the ticket in hudi channel maybe someone can help there |
@the-other-tim-brown is this something you can help here? |
@soumilshah1995 @alberttwong I'm on vacation until Thursday, I can help replicate this issue once I'm back. @the-other-tim-brown feel free to chime in when you get a chance. |
@sagarlakshmipathy Thanks a lot. Please enjoy your vacations this is not urgent or blocking its mostly for POC |
I have not used MinIO before. I will have to spend some time coming up to speed on it. |
@soumilshah1995 have you tried updating the hadoop config like Albert suggested? He put up the configs he used here: https://github.com/apache/incubator-xtable/pull/337/files |
I haven't opted for using a container for my Hadoop setup. Could you kindly suggest the steps to set it up on a Mac?" |
Soumil... I think if slightly change your java run command to |
I think I will take a different route rather using deltastreamer and xtable with MINIO and StarRocks |
Hello there I have MINIO bucket locally huditest
I am trying to use onetable with MINIO
docker-compose file
hudi_job.py
config.yml
Error
How to configure one table to work with MINO buckets ?
I tried s3 and s3a both how would I setup to work with minio buckets
The text was updated successfully, but these errors were encountered: