Cloud Spanner Client should prevent nested transactions

[vkedia]

Cloud Spanner does not support nested transactions. But the client library does allow users to call database.run_in_transaction inside of the callback that is provided to an outer database.run_in_transaction. This is misleading since users might believe that this will behave like a nested transaction but in reality these will be two independent transactions. This is confusing and a source of bugs. Specifically the inner transaction might succeed while the outer might ABORT, in which case the callback will be rerun thus rerunning the inner transaction.
We should prevent this from happening by detecting and raising an error if someone calls database.run_in_transaction inside the callback provided to database.run_in_transaction. Note that this needs to be done before Beta since this is a known breaking change.
Also note that if we come across a use case where we do want to allow calling a nested database.run_in_transaction, we should be able to enable that in future without making a breaking change.

cc @jgeewax @bjwatson

[bjwatson]

@tseaver This can wait until you have time to work on the Spanner integration tests.

[vkedia]

Note that this also is beta blocking.

[bjwatson]

Thanks for the reminder, @vkedia. I'm reviewing the Python Spanner issues, and adding the status: release blocking to any that are Beta blocking. That way we can see a hotlist for remaining issue.

[vkedia]

Can this be reopened? I don't think the fix actually fixes this.

[bjwatson]

Reopened on behalf of @vkedia, so the remaining issues can be discussed further.

[lukesneeringer]

I reviewed the code; it does exactly what it says on the tin. What is the problem exactly?

[vkedia]

The exact requirements are if someone calls database.run_in_transaction inside the callback passed to database.run_in_transaction it should blow up. I don't think that would happen with the fix. You can verify that by writing a test that does this. Infact such a test should have been there in the PR in the first place.

[lukesneeringer]

There is a test that does this.

[vkedia]

It is not the same thing. That test is calling session.transaction. I am talking about database.run_in_transaction. The issue is when you call database.run_in_transaction again, it will checkout a new session so it will keep on working.
Right way to fix this would be somehow track in a thread local context or something analogous that there is an ongoing transaction in this thread and use that to prevent any other transaction from being opened.

[lukesneeringer]

@tseaver I think I agree with Vikas here. The session checkout would still fundamentally allow...

def bad_news():
    database.run_in_transaction(unit_of_work)

database.run_in_transaction(bad_news)

Note: This is so incredibly unusual in Python that I am struggling to figure out how to properly prevent it. Nevermind, I think I know what to do.

[tseaver]

@lukesneeringer FWIW, the callback passed to Database.run_in_transaction ought to be using only the locals it is passed.

[lukesneeringer]

@tseaver Agreed, but we have no way to enforce that.

In fairness, relying on this kind of weird scoping is uncommon in Python, so this entire issue is well into the edge case territory ("if someone aims a gun at their foot...").

Regardless, I will have a PR in a moment.

[tseaver]

Calling database.run_in_transaction from within the unit-of-work callback (e.g., bad_news) will check out a new session, start a new transaction in it, and call the passed other-unit-of-work callback. I don't see how we can prevent that.

[bjwatson]

@lukesneeringer @tseaver I understand that we might have some false negatives, because we cannot catch all cases in which a nested transaction might be initiated. Are we avoiding all false positives (i.e. failing on a non-existent nested transaction)?

I remember this was a concern in Node.js.