You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the table names generated in a database destination (tested with ClickHouse) will be a function of dataset_name and dataset_table_seperator. If we use default values for those configuration options, most likely one will end with a table name generated as
{pipeline_name}_dataset__{resource_name}
This feature request is about adding an option to override the generation of the table name entirely bypassing any internal templating.
Are you a dlt user?
Adopting
Use case
Drop in replacement for other data movement tools which do not expect the downstream pipeline to adapt to naming conventions but give the user full control over destination table names.
Proposed solution
perhaps expose a table naming template string user can specify in the pipeline definition, i.E.:
The text was updated successfully, but these errors were encountered:
pulsar256
changed the title
Allow to override the final table name in the a database destination
Allow to override the final table name in a database destination
Oct 24, 2024
@pulsar256 we do it to simulate schemas on destinations that do not support it. typically we let users to pass empty dataset name to pipeline to get rid of the naming layout but I see that this is disabled for clickhouse. we can enable it in the next release, seems like not much work
Feature description
Currently the table names generated in a database destination (tested with ClickHouse) will be a function of
dataset_name
anddataset_table_seperator
. If we use default values for those configuration options, most likely one will end with a table name generated asThis feature request is about adding an option to override the generation of the table name entirely bypassing any internal templating.
Are you a dlt user?
Adopting
Use case
Drop in replacement for other data movement tools which do not expect the downstream pipeline to adapt to naming conventions but give the user full control over destination table names.
Proposed solution
perhaps expose a table naming template string user can specify in the pipeline definition, i.E.:
The text was updated successfully, but these errors were encountered: