Cannot create a spanner session when running in compute engine

[mkmg]

Cannot create a session when running in compute engine

An error occurs when session.create() is called from a compute engine vm. The Compute Engine VM is running in the same Google Cloud Project as the Spanner instance.

1. OS type and version

Debian GNU/Linux 9.1 (compute engine vm)

2. Python version

Python 2.7.13

3. google-cloud-python version

google-auth==1.0.2
google-auth-httplib2==0.0.2
google-cloud-bigquery==0.27.0
google-cloud-core==0.27.1
google-cloud-spanner==0.27.0
google-cloud-storage==1.4.0
google-compute-engine==2.6.0

4. Stacktrace

File "/home/mkmg/integration_test.py", line 123, in _GetProgressObjForJobRun
    session.create()
  File "/home/mkmg/venv/local/lib/python2.7/site-packages/google/cloud/spanner/session.py", line 98, in create
    api = self._database.spanner_api
  File "/home/mkmg/venv/local/lib/python2.7/site-packages/google/cloud/spanner/database.py", line 165, in spanner_api
    credentials = credentials.with_scopes((SPANNER_DATA_SCOPE,))
  File "/home/mkmg/venv/local/lib/python2.7/site-packages/google/auth/compute_engine/credentials.py", line 120, in with_scopes
    'Compute Engine credentials can not set scopes. Scopes must be '
NotImplementedError: Compute Engine credentials can not set scopes. Scopes must be set when the Compute Engine instance is created.

5. Steps to reproduce/6. Code example

Create a Spanner instance and database in a Google Cloud Project. In the same project, create a Compute Engine VM and allow full access to all cloud APIs during creation. In the vm, run the following lines: (store the proper values in instance_name and database_name)

client = spanner.Client()
instance = client.instance(instance_name)
database = instance.database(database_name)
session = database.session()
session.create()

[lukesneeringer]

Does the credential already have the SPANNER_DATA_SCOPE?

[tseaver]

@lukesneeringer, @jonparrott We cannot update scopes on GCE credentials, which means that should not be subclassed from google.auth.credentials.Scoped (raising NotImplementedError from the override in a concrete subclass is a Liskov substition principle violation.

[tseaver]

I opened googleapis/google-auth-library-python#194 to track the issue.

[tseaver]

googleapis/google-auth-library-python#195 suggests a new ReadonlyScoped class, from which the GCE Credentials derive.

If that PR is merged and released, the code we have now in Database.spanner_api will work as designed: GCE credentials will no longer derive from Scoped, and thus will bypass the with_scopes call. We would then need to update the minimum version of google-auth required in setup.py to pick up that fix.

[tseaver]

googleapis/google-auth-library-python#195 has been merged.

[vkedia]

What is the core issue here? Why is session creation failing?
@mkmg orthogonal to this, it is better to use the session pooling library instead of managing sessions yourself.

[tseaver]

@vkedia See googleapis/google-auth-library-python#194, and the fix in googleapis/google-auth-library-python#195 which updates GCE credentials such that they no longer advertise (falsely) that they can create instances with added scopes.

[vkedia]

So does this mean that spanner client does not work at all on compute engine? It definitely did work the last time I tried it.

[lukesneeringer]

Note: google-auth 1.1 is on PyPI now.

[erlichmen]

I have the same issue when trying to use the spanner client under plain machine (like my mac).
Same issue as the OP.

[tseaver]

Via #3950.