[Feature] Add Support for Mapping dbt Objects to AWS Glue Jobs/Workflows #1199
Labels
area:parsing
Related to parsing DAG/DBT improvement, issues, or fixes
enhancement
New feature or request
parsing:dbt_manifest
Issues, questions, or features related to dbt_manifest parsing
triage-needed
Items need to be reviewed / assigned to milestone
Description
Currently, astronomer-cosmos provides an abstraction layer to map dbt objects to Airflow operators, allowing seamless integration between dbt and Airflow for orchestrating data pipelines. We propose to extend this functionality to AWS Glue by creating a similar abstraction layer that maps dbt objects to AWS Glue jobs or workflows.
Feature Proposal:
Key Components:
Benefits:
Potential Challenges:
Use case/motivation
A user working in an AWS-centric environment can define their dbt models and leverage AWS Glue’s fully managed service for ETL pipelines. This allows them to automate orchestration without using Airflow, providing greater flexibility for teams leveraging AWS Glue for their data workflows.
Related issues
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: