Version 0.17 breaks GCS blob signing

[dguaraglia]

When trying to update our project to use gcloud 0.17, we found one of our tests which uses the generate_signed_url for GCS blobs started failing with the following error:

project/tests/common.py:302:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
project/gcs.py:159: in get_signed_url
    return self._content.generate_signed_url(delta, method=method, content_type=content_type)
../../../../.virtualenvs/project/lib/python2.7/site-packages/gcloud/storage/blob.py:239: in generate_signed_url
    generation=generation)
../../../../.virtualenvs/project/lib/python2.7/site-packages/gcloud/credentials.py:223: in generate_signed_url
    string_to_sign)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

credentials = <oauth2client.client.GoogleCredentials object at 0x10f435690>, expiration = 1467937858
string_to_sign = 'GET\n\napplication/json\n1467937858\n/project-test-hcpc74v437kuc-ops/file7jcwv4v437kuc'

    def _get_signed_query_params(credentials, expiration, string_to_sign):
        """Gets query parameters for creating a signed URL.

        :type credentials: :class:`oauth2client.client.AssertionCredentials`
        :param credentials: The credentials used to create a private key
                            for signing text.

        :type expiration: int or long
        :param expiration: When the signed URL should expire.

        :type string_to_sign: string
        :param string_to_sign: The string to be signed by the credentials.

        :rtype: dict
        :returns: Query parameters matching the signing credentials with a
                  signed payload.
        """
>       _, signature_bytes = credentials.sign_blob(string_to_sign)
E       AttributeError: 'GoogleCredentials' object has no attribute 'sign_blob'

[daspecster]

Are you on GCE(Compute Engine)?

If so then it seems this would apply...
https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/storage/blob.py#L166

[dguaraglia]

@daspecster we are actually using a JSON credentials file for a service account. I wonder if there's anything in particular about these credentials that might cause the issue.

[daspecster]

I was just able to reproduce the issue locally.

>>> import time
>>> from gcloud import storage
>>> from gcloud.storage import Blob
>>> client = storage.Client()
>>> bucket = client.get_bucket('my-test-bucket')
<Bucket: my-test-bucket>
>>> blob = Blob('ewfwefwefwef', bucket)
>>> blob.generate_signed_url(int(time.time()+5), method='GET')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gcloud/storage/blob.py", line 239, in generate_signed_url
    generation=generation)
  File "gcloud/credentials.py", line 223, in generate_signed_url
    string_to_sign)
  File "gcloud/credentials.py", line 102, in _get_signed_query_params
    _, signature_bytes = credentials.sign_blob(string_to_sign)
AttributeError: 'GoogleCredentials' object has no attribute 'sign_blob'

It looks like it's not detecting that it's a service account and then using ServiceAccountCredentials, which it needs to do, in order to have access to sign_blob().

https://github.com/google/oauth2client/blob/a4e34317d5b2565cd9194471cbc9d806ce9aceae/oauth2client/client.py#L1253

@dhermes @tseaver, any ideas what changed?

[dhermes]

@daspecster your local environment likely has gcloud CLI in it, hence those user credentials are getting picked up.

[dhermes]

http://gcloud-python.readthedocs.io/en/latest/gcloud-auth.html#credential-discovery-precedence

In theory the env. var. takes precedence over the CLI settings, but maybe something in oauth2client has changed.

[daspecster]

@dhermes, that makes sense. It is picking up my JSON credentials, but shouldn't that end up still being an instance of ServiceAccountCredentials? sign_blob() is only available with GCE, AppEngine and service accounts AFAICT.

[jgeewax]

This seems like a pretty bad error though... If it's picking up credentials that aren't capable of signing (ie, it just has a token and not a private key) we should say that... not an AttributeError....

Can we update this to check and then spit out "you need a private key to sign credentials. the credentials you're currently using (insert pointer to those here) just include a token. see (insert link here) for more details." ?

[daspecster]

I'm not sure about the changes in oauth2client but there's an updated error message now per #1966.

Let me know if we should close this now.

[theacodes]

This should no longer be an issue:

1. We raise a better error now.
2. We're no longer using oauth2client.