End to end documentation

[johalnes]

One of the main advantages of DBT is the amazing flow diagram and documentation of all data created. And dbt-duckdb does an amazing job by taking the usefulness of this one step further - including python functions where it makes sense and still get the nice overview of the flow that DBT gives. But what about python documentation?

Until now I've been a fan of using DBT for data transformations only, and then including the DBT webpage as a static site into the main site. As main site I've been using mkdocs and includes more general information like how to guides and of course documentation for Python functions. This works and I love mkdocs, but the DBT site is not searchable and it feels like it's two completely separate projects(partly because until now it has been).

Does anyone have a clever idea to get one spot for documentation that contains both DBT flows, as well as Python reference? Any ways to either extend DBT documentation site with separate markdown files?

Or the other way around, would it be hard to parse the DBT manifest into a graphic in mkdocs? For instance for each folder within Models directory create one page giving the graph with nodes first as an overview and the list all tables below?

[johalnes]

And also an interesting point - how important is documentation for a medium sized company? Is it just a dream that one can remove key personnel risk by documenting well enough? Does it make sense to use valuable time on creating how to guides that explains GIT, code structure, tables etc? Or is just good structure and consistency in the team the only way of keeping control of all data flows and models, and thus improving code quality should get priority?