Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/improve rpc compile performance (#1824) #1830

Merged
merged 13 commits into from
Oct 14, 2019
Merged

Feature/improve rpc compile performance (#1824) #1830

merged 13 commits into from
Oct 14, 2019

Commits on Oct 14, 2019

  1. Add manifest.expect method

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    6c87bed View commit details
    Browse the repository at this point in the history

  2. loader.GraphLoader -> parser.manifest.ManifestLoader 

    Create new helper function dbt.perf_utils.get_full_manifest
Update task.runnable accordingly
Update RPC server accordingly
    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    d86092a View commit details
    Browse the repository at this point in the history

  3. Refactors for mypy: 

    initial refactoring of adapter factory stuff
Move HasCredentials protocol into connection contract and use that in the base connection
    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    ef16a99 View commit details
    Browse the repository at this point in the history

  4. Refactor internal RPC logic to support just getting parsed manifests 

    RemoteCallableResult -> RPCResult
RemoteCallable -> RemoteMethod
 - move some things from RPCTask -> RemoteMethod
   - recursive_subclasses classmethod
things in core/dbt/rpc now are all based on RemoteMethods, not RPCTasks
    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    773c979 View commit details
    Browse the repository at this point in the history

  5. Manifests are only parsed, not compiled at SIGHUP/startup time 

    The _sql tasks now compile any ref'ed CTE chains at RPC call time

Give RPC tasks their own folder
 - task/rpc_server -> task/rpc/server
 - task/remote -> task/rpc/{project_commands,sql_commands,base}

Linker enhancements:
  - Expose subset graph building so multiple methods can use it
  - Expose a way for the linker to provide an interable of the ephemeral ancestors of a node
     - it's guaranteed to be ordered (so nested CTEs behave)
    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    7206c20 View commit details
    Browse the repository at this point in the history

  6. Refactor RPC tests

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    66ff79d View commit details
    Browse the repository at this point in the history

  7. Make ManifestMetadata a first-class object

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    75c8fea View commit details
    Browse the repository at this point in the history

  8. Make mypy totally happy 

    Some circular import cleanups
remove is_type function, just compare to resource_type
Add type checking for dbt deps
    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    4aa4295 View commit details
    Browse the repository at this point in the history

  9. fix field serialization for hologram+mypy

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    b658f87 View commit details
    Browse the repository at this point in the history

  10. fix unit tests

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    61f8e6d View commit details
    Browse the repository at this point in the history

  11. Avoid deepcopying the manifest 1x per task per sighup

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    14d8683 View commit details
    Browse the repository at this point in the history

  12. add register_adapter to dbt debug

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    7591675 View commit details
    Browse the repository at this point in the history

  13. remove inscrutable __getattr__ override

    Jacob Beck committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    43daea0 View commit details
    Browse the repository at this point in the history