Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(webhdfs): handle 307 redirection instead of noredirect #1358

Merged
merged 4 commits into from
Feb 18, 2023

Conversation

ClSlaid
Copy link
Contributor

@ClSlaid ClSlaid commented Feb 17, 2023

OpenDAL will handle all redirections, and auto redirection of client is not taken into considerations. So for WebHDFS, handler redirections directly instead of noredirect=true. This also offers better compatibility.

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

fixes: #1355

OpenDAL will handle all redirections, and auto redirection of client is
not taken into considerations. So for WebHDFS, handler redirections
directly instead of `noredirect=true`. This also offers better
compabilities.

Signed-off-by: ClSlaid <[email protected]>
src/services/webhdfs/backend.rs Outdated Show resolved Hide resolved
src/services/webhdfs/backend.rs Outdated Show resolved Hide resolved
src/services/webhdfs/backend.rs Outdated Show resolved Hide resolved
src/services/webhdfs/backend.rs Show resolved Hide resolved
@Xuanwo
Copy link
Member

Xuanwo commented Feb 17, 2023

[2023-02-17T10:54:05Z ERROR opendal::services] service=webhdfs operation=read path=75eb810f-3c68-4371-97c4-3b071361d177 range=0- -> failed: Unexpected (permanent) at read => expected 307 temporary redirect, got 404 Not Found
    
    Context:
        service: webhdfs
        path: 75eb810f-3c68-4371-97c4-3b071361d177
        range: 0-
    
thread 'services_webhdfs_write::test_read_not_exist' panicked at 'assertion failed: `(left == right)`
  left: `Unexpected`,
 right: `ObjectNotFound`', tests/behavior/write.rs:560:5

@Xuanwo
Copy link
Member

Xuanwo commented Feb 17, 2023

It's better to use exisintg parse_error function instead return unexpected errors directly.

@ClSlaid
Copy link
Contributor Author

ClSlaid commented Feb 17, 2023

cc @Xuanwo , most checks except msrv passed, I'll merge it.

@Xuanwo
Copy link
Member

Xuanwo commented Feb 17, 2023

please update with main, already fixed.

@Xuanwo Xuanwo merged commit 91a0733 into apache:main Feb 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: webhdfs doesn't handle redirection correctly
2 participants