Feature/logbook structured rpc logging (#1703, #1704) #1715
Conversation
- fix a bug where failed connections caused an AttributeError - fix an issue where the rpc server and its child processes secretly shared mutable state
Log rpc as json, preserve existing log behavior Force werkzeug logs into logbook handling Make rpc a package and set up mypy checking on it fix a lot of logging-related bugs Provide a log manager to toggle various log functionalities add disabled flag to file handler for "dbt debug" and friends Test changes to support logbook lots of fiddly process/thread handling
There was a problem hiding this comment.
Couple of questions / comments here, but overall this looks very good! I did some quick testing locally and I was really impressed with how well everything worked! Logbook looks really incredible - I think that's a perfect library to use here.
One quick follow up question: You mentioned that you "stumbled on a pair of odd connection related bugs that I fixed" - can you say more about that?
Really nice work, Jake!
The first I fixed in this commit - on failed connections, we would have The second was in the integration tests - in some failure paths, we did a similar thing where we tried to rollback a handle, but the handle might be None, which obscured the actual failing error message. |
Fixes #1703
Fixes #1704
As part of this PR, I stumbled on a pair of odd connection related bugs that I fixed.
dbt.log, and return that in thestatusendpoint. (In the RPC server, have the "status" method return the logs from the last compile #1703)logbookpackage, spit out json indbt rpc(Rationalize logging in the rpc server #1704)logsfield of rpc responsesdbt.loggernow provides alog_managerthat lets you toggle the various global logging settings:dbt --debug ...)dbt rpc)dbt ls)dbt/rpc.pyinto a packagedbt/rpc, added it tomypy.ini.This will break anything that hooks in to the dbt logger :)