sql: creating / dropping the same user concurrently can cause a deadlock

[RichardJCai]

Likely the cause of this issue:
#74271 (comment)

This roachtest here non-deterministically reproduces the deadlock.

https://github.com/RichardJCai/cockroach/blob/create_user_concurrent_test/pkg/cmd/roachtest/tests/create_user_concurrent.go

Only tried on 21.2 since #74271 is failing on 21.2

Update: It repros on master as well.

Jira issue: CRDB-12524

[rafiss]

Could you try repeatedly doing GRANT admin TO user and REVOKE admin FROM user and see if that also has issues? I wonder if it relates to how we update the descriptor version of the system table.

[RichardJCai]

Could you try repeatedly doing GRANT admin TO user and REVOKE admin FROM user and see if that also has issues? I wonder if it relates to how we update the descriptor version of the system table.

It does hang actually.

Also ran into this error

18:57:28 test_impl.go:323: test failure: 	sql_runner.go:63,create_user_concurrent.go:46: error executing 'GRANT admin TO testuser': pq: internal error: deadline below read timestamp is nonsensical; txn has would have no chance to commit. Deadline: 1642791448.730002636,1. Read timestamp: 1642791448.779620133,0 Previous Deadline: 1642791448.730002636,1.
panic: t.Fatal() was called

[rafiss]

The RandomSchemaChange test is also timing out: #75646 (comment)

So I wonder if there's a more general issue.

[RichardJCai]

The RandomSchemaChange test is also timing out: #75646 (comment)

So I wonder if there's a more general issue.

If this is the same issue then it seems like it's been present for a long while.
#30893

[ZhouXing19]

The goroutine dump can be found here: https://gist.github.com/ZhouXing19/79a382f0c28a28fb56f6629f2e14d89b
cc @rafiss