Exception: Request failed. Error was: Backend Error

[pdknsk]

I'm getting this error for every request, get or put. I don't know why. The API console only knows this.

Methods	Error codes
datastore.datasets.commit	500
datastore.datasets.lookup	500

[dhermes]

Is there an error payload? Can you provide code snippets? What are you doing for auth? What version of the library are you using?

[pdknsk]

I've figured it out. The datastore is tied to an appengine app-id I use for testing, which was disabled in the old dashboard. Once I re-enabled it worked. I guess there is still a bug in the Error not being descriptive.

[pdknsk]

OT, but will this library also be installable through gcloud once it's stable? It's a bit confusing to me how gcloud-python and the other gcloud are related. Well done either way, it's an improvement from using the Remote API.

[dhermes]

@pdknsk

RE: This bug, can you post a stacktrace and some code which caused it? I'll try it out on a disabled project to see if I can get come up with some feedback for the API team. (Our exceptions print the error messages returned in the payload.)

RE: Install. I doubt it will be installable through gcloud but pip / PyPI is typically fine for installing Python libraries. Anything I'm missing?

RE: gcloud. Our implicit auth already relies on oauth2client which shares tokens (locally) with the gcloud CLI tool.

Thanks for the feedback about the confusion, it's good for us to hear these things.

/cc @silvolu

[silvolu]

@pdknsk thanks for your feedback!

RE: Install. Currently there are no plans to make this library installable through the gcloud CLI. As mentioned by @dhermes, making it pip-installable is our preferred approach.

[pdknsk]

What adds to the confusion is yet another gcloud, which is really gcloud-cli-go from what I can tell.

To reproduce this bug, disable a project in Application Settings in the old Appengine Dashboard. This function does not seem to be available in the new Cloud Console. Run the steps to access a Datastore tied to an Appengine project, in particular the Wizard to enable the API.

from gcloud import datastore
a = datastore.get_dataset('app-id', '@developer.gserviceaccount.com', 'path/key')

a.entity('item').save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/entity.py", line 244, in save
    exclude_from_indexes=self.exclude_from_indexes())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 425, in save_entity
    result = self.commit(dataset_id, mutation)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 329, in commit
    datastore_pb.CommitResponse)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 81, in _rpc
    data=request_pb.SerializeToString())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 59, in _request
    raise Exception('Request failed. Error was: %s' % content)
Exception: Request failed. Error was: Backend Error

# project re-enabled
a.entity('item').save()
<Entity[{'kind': u'item', 'id': 5649391675244544L}] {}>

[dhermes]

Thanks for giving me a way to replicate @pdknsk.

I added a debug statement before the error is raised and actually printed the body and headers of the response.

---

With writes disabled, but the app still enabled I got

('The API call datastore_v4.Commit() is temporarily unavailable: Datastore '
 'writes have been disabled by an application administrator. Writes can be '
 're-enabled in the admin console.')
# Word-wrapping by me

{
  "status": "403",
  "alternate-protocol": "443:quic,p=0.02",
  "content-length": "178",
  "x-xss-protection": "1; mode=block",
  "x-content-type-options": "nosniff",
  "transfer-encoding": "chunked",
  "expires": "Fri, 02 Jan 2015 18:28:40 GMT",
  "vary": "Origin, X-Origin",
  "server": "GSE",
  "-content-encoding": "gzip",
  "cache-control": "private, max-age=0",
  "date": "Fri, 02 Jan 2015 18:28:40 GMT",
  "x-frame-options": "SAMEORIGIN",
  "content-type": "text/html; charset=UTF-8"
}

---

With the app disabled (as @pdknsk suggests) I got

'Backend Error'

{
  "status": "503",
  "alternate-protocol": "443:quic,p=0.02",
  "content-length": "13",
  "x-xss-protection": "1; mode=block",
  "x-content-type-options": "nosniff",
  "transfer-encoding": "chunked",
  "expires": "Fri, 02 Jan 2015 18:31:42 GMT",
  "vary": "Origin, X-Origin",
  "server": "GSE",
  "-content-encoding": "gzip",
  "cache-control": "private, max-age=0",
  "date": "Fri, 02 Jan 2015 18:31:42 GMT",
  "x-frame-options": "SAMEORIGIN",
  "content-type": "text/html; charset=UTF-8"
}

---

Do you think we should also print the status code? Here the information was 403 Forbidden when writes aren't allowed vs. 503 Service Unavailable when the app is turned off.

Do you think the backend should return more than just Backend Error if the app is turned off? FWIW I also tried this on apps that don't exist and apps that my credentials can't see and in both cases got a non-descript error and a 503. This may indicate that the error is a security protection.

[pdknsk]

I don't know how this works behind the scenes, but I've been thinking that Datastore should perhaps be able to operate separately from App Engine. Storage does. Every App Engine project gets a default bucket named app-id.appspot.com which can be accessed with this API even when the app is disabled. I just tried.

[dhermes]

@pdknsk Thanks for the feedback.

@silvolu What's the right channel for this?