Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: homogeneize / abstract access to db / table descriptors #17188

Closed
knz opened this issue Jul 24, 2017 · 3 comments
Closed

sql: homogeneize / abstract access to db / table descriptors #17188

knz opened this issue Jul 24, 2017 · 3 comments
Assignees
Labels
A-schema-descriptors Relating to SQL table/db descriptor handling. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. S-3-productivity Severe issues that impede the productivity of CockroachDB developers.
Milestone

Comments

@knz
Copy link
Contributor

knz commented Jul 24, 2017

So while polishing #15388 I realize I do not really have an answer to "how do I get a db descriptor by ID" (other than, I know already, constructing the descriptor key and talking to KV manually)

Actually this pulled up some more questions:

  • how can I access "old" versions of database descriptors? (Is that at all possible?)
  • how do I get a table desc by ID?
  • how do I get a cached table desc by ID?
  • when exactly should I care whether I should request a cached version of a desc or not?

In general I am missing an internal descriptor access API. Currently all the code to access descriptors is ad-hoc, not documented in a central place and the schema lease API is not optimized for fast access to descriptors by ID.

What is needed here is:

  • an interface schemaResolver or something, with a stable API documented in a single place
    • make that API able to resolve descriptors either by name or by ID
    • make that API take a boolean flag "historical query" or something usage-oriented which take ownership of answering the question of whether to use a cached descriptor or not
    • make that API stateful with a "local state" object and make it able to cache lookups within the context of a single caller even when not using the lease manager (so that multiple lookups of the same table by the same caller in a row doesn't hit KV too much)
  • have *planner (or whatever) implement that API
    • ensure data structures ensure equally fast by-name and by-ID resolution
@knz knz added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. high priority labels Jul 24, 2017
@knz knz added this to the 1.1 milestone Jul 24, 2017
@vivekmenezes vivekmenezes modified the milestones: 1.2, 1.1 Jul 24, 2017
@vivekmenezes
Copy link
Contributor

while I agree that there is room for improvement here, a lot of work has been done on table descriptors to bring them more inline with how we want them to be used. See the RFC:
https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/table_descriptor_lease.md

for discussions on table descriptor versions and how they should be used.

More work needs to be done to improve the API in the code and I expect to do some of that work for 1.2.

Unfortunately this still doesn't cover database descriptors and we have work to do here too. it has just not been a priority.

@bdarnell bdarnell modified the milestones: 2.0, Later Feb 8, 2018
@knz knz added A-schema-descriptors Relating to SQL table/db descriptor handling. S-3-productivity Severe issues that impede the productivity of CockroachDB developers. and removed A-bulkio-schema-changes labels Apr 27, 2018
@knz
Copy link
Contributor Author

knz commented Apr 27, 2018

This may be fixed by #22275

@vivekmenezes
Copy link
Contributor

closing in favor of #22275

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-schema-descriptors Relating to SQL table/db descriptor handling. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. S-3-productivity Severe issues that impede the productivity of CockroachDB developers.
Projects
None yet
Development

No branches or pull requests

3 participants