Question: Possible to drop authentication header on redirects?

[colinjstief]

I am attempting to make an authenticated GET request that retrieves a presigned URL to file sitting in Amazon S3. However, I am getting a 400 response with the following error:

"Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specified".

I can make the same request in Postman and successfully retrieve my file. Seems like I need to drop my authorization headers on the redirect that is happening. Is this possible with httr?

[cderv]

I don't know specifically about S3 url and auth mechanism. The error you mentioned seems to indicate you did not use any auth mechanism to connect, or too much mechanism.
You can add a authorization header using add_headers.

I think you could ask this kind of question on community.rstudio.com where there is a broad community ready to help and surely some experienced user with this.

Know that there are some 📦 dedicated to work with AWS S3 like aws.s3 that is build upon httr. They may be of help for this kind of usage.

Hope it helps.

[colinjstief]

Auth header is included and valid on initial request, which is successful. Will head over to community, thanks!

[colinjstief]

One other thought here—I'm confident the re-use of auth header on redirect is my issue, so my question is less troubleshooting and more "does this feature exist in the package". It's not obvious to me from the documentation, but maybe I'm missing something.

Here is a screenshot for the toggle for this in Postman:

Maybe in the end this is more of a feature request.

[cderv]

Can you give an example code of what you are using currently and don't work ?

That way I can have a look if authorization header can be drop for redirect

[colinjstief]

Sure, it's super simple:

exportRequest <- GET(
  exportDownloadURL,
  authenticate(user, password, type = "basic")
)

This goes where I need it to and correctly redirects me to a pre-signed URL that I can use to download the file I need. Just for a little context, this is a third-party API that stores their files on AWS, so I have no control over any of that. What I'm looking for is something like this:

exportRequest <- GET(
  exportDownloadURLComplete,
  authenticate(user, password, type = "basic"),
  passAuthOnRedirect = FALSE
)

[cderv]

thanks for the example that clarifies the situation.

What version of curl do you have on your system?
It seems like a bug in curl to pass the headers also into the redirected request.
See https://curl.haxx.se/docs/CVE-2018-1000007.html

May it be related ? Updating curl can help with this it seems - I don't know how to test this and which version curl is using with httr 🤷‍♂
curl --version on my windows says 7.55 however, curl::curl_version() in R says 7.64.1. I don't know which one is used... 🤔
I am on windows and indeed curl has been updated to 7.64

For the second solution advised on the link above, you can catch the redirect then follow without providing the header. Here is a simple example you can surely improve.

library(httr)
# Authorization header is indeed passed through
resp <- GET(
  "https://httpbin.org/redirect-to",
  query = list(
    url="https://httpbin.org/headers"
  ),
  authenticate("user", "pwd", type = "basic")
)
# look at content res = headers received
res <- content(resp)
"Authorization" %in% names(res$headers)
#> [1] TRUE

# You can catch the redirect yourself and do not provide header
resp <- GET(
  "https://httpbin.org/redirect-to",
  query = list(
    url="https://httpbin.org/headers"
  ),
  config(followlocation = FALSE),
  authenticate("user", "pwd", type = "basic")
)
if ("location" %in% names(headers(resp))) {
  redirected_url <- headers(resp)[["location"]]
  resp <- GET(redirected_url)
}
res <- content(resp)
"Authorization" %in% names(res$headers)
#> [1] FALSE

^{Created on 2019-11-10 by the reprex package (v0.3.0)}

From curl doc, it seems with newer version of curl Auth should not be passed on REDIRECT if hostname is different (which in my test is not). You can try yourself with your real example.

• Check you curl version and see if correct
• update and test again.
• maybe try directly with a curl command line if your version is 7.58 or above to check it;

Hope it helps.

[colinjstief]

Thanks @cderv that's very helpful!

[yaswanth-8]

The error message indicates that multiple authentication mechanisms are being used simultaneously. Specifically, the error states that both the X-Amz-Algorithm query parameter (used for pre-signed URLs) and the Authorization header (which is likely being set somewhere in your code) are being included in the request. S3 only allows one authentication mechanism per request.