Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Nessie Rest s3 signer #1028

Closed
guitcastro opened this issue Aug 9, 2024 · 1 comment · Fixed by #1033
Closed

Support for Nessie Rest s3 signer #1028

guitcastro opened this issue Aug 9, 2024 · 1 comment · Fixed by #1033

Comments

@guitcastro
Copy link
Contributor

Apache Iceberg version

main (development)

Please describe the bug 🐞

Nessie s3 signer use a different endpoint than /aws/s3/sign and also includes query parameters in the URI. Therefore, we need to honor their s3.signer.endpoint which is currently static in code.

Additionally, we need to parameterize the X-Iceberg-Access-Delegation, which currently has the value vended-credentials hardcoded. This is necessary because Nessie does not return the s3.signer.endpoint when the value is set to vended-credentials.

An example of Nessie config from get table endpoint:

    "config": {
        "s3.path-style-access": "true",
        "s3.signer.uri": "http://127.0.0.1:19120/iceberg/",
        "s3.endpoint": "http://minio:9000/",
        "io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
        "s3.signer.endpoint": "v1/main/s3-sign/default.datasets?e=1723099768&b=s3%3A%2F%2Fwarehouse&k=f1379b71-f73a-4fcd-8765-4a1aba6a3585&s=22f3f6b52d586c3b9b310eb2df0e2e51d0c8d0b03030f5bb25f8b2548ef406da&w=s3%3A%2F%2Fwarehouse%2Fdefault%2Fdatasets_f244e57f-143a-4e19-b1c6-591cafbe96fe",
        "s3.remote-signing-enabled": "true",
        "client.region": "us-east-1"
    }
@guitcastro guitcastro changed the title Support for nessie s3 signer Support for Nessie Rest s3 signer Aug 9, 2024
@guitcastro
Copy link
Contributor Author

guitcastro commented Aug 9, 2024

Create the PRs #1029 and #1033 to fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant