LoadMode.CUSTOM cannot select tags

[adammarples]

https://github.com/astronomer/astronomer-cosmos/blob/d28d3df7eacac74d0fbb016be70434bcc30c51be/cosmos/dbt/graph.py#L314C17-L314C25

load_via_custom_parser hard codes DbtNode.tags attribute to be an empty list

        for model_name, model in models:
           config = {item.split(":")[0]: item.split(":")[-1] for item in model.config.config_selectors}
           node = DbtNode(
               unique_id=f"{model.type.value}.{self.project.project_name}.{model_name}",
               resource_type=DbtResourceType(model.type.value),
               depends_on=list(model.config.upstream_models),
               file_path=Path(
                   model.path.as_posix().replace(
                       self.render_config.project_path.as_posix(), self.execution_config.project_path.as_posix()
                   )
               ),
               tags=[],
               config=config,
           )

This causes _is_tags_subset to return False every time

    def _is_tags_subset(self, node: DbtNode) -> bool:
      """Checks if the node's tags are a subset of the config's tags."""
      if not (set(self.config.tags) <= set(node.tags)):
          return False
      return True

because set(node.tags) is always empty.

[dosubot]

🤖

Hi @adammarples,

Great to see you again! Thanks for bringing this to our attention. I'm diving into the issue you've raised and will get back to you with a detailed response soon.

[dosubot]

🤖

Hello @adammarples, it's great to see you again! I hope you're doing well.

Thank you for bringing this to our attention. You're correct that the load_via_custom_parser method currently sets the DbtNode.tags attribute to an empty list, which causes _is_tags_subset to return False every time since set(node.tags) is always empty.

This is because the custom parser does not currently support parsing tags from the dbt project. It's a limitation of the custom parser and not a deliberate design decision to always return False for _is_tags_subset.

If you need to use tags in your dbt project with Cosmos, you might want to consider using a different load method that supports tags, such as load_via_dbt_ls or load_from_dbt_manifest, if your project setup allows for it.

In the load_via_dbt_ls and load_from_dbt_manifest methods, the tags are loaded directly from the dbt project's metadata. In the load_via_dbt_ls method, the tags are loaded from the output of the dbt ls command, as seen in the parse_dbt_ls_output function:

tags=node_dict["tags"],

In the load_from_dbt_manifest method, the tags are loaded from the dbt project's manifest file:

tags=node_dict["tags"],

On the other hand, the load_via_custom_parser method does not handle the DbtNode.tags attribute at all. In this method, a DbtNode instance is created for each model in the dbt project, but the tags attribute is left as an empty list:

tags=[],

This means that any tags associated with the models in the dbt project are not loaded when the load_via_custom_parser method is used.

I hope this clarifies the current behavior of the load_via_custom_parser method in relation to the DbtNode.tags attribute. If you have any further questions or need additional clarification, please let me know.

Sources

• dbt/selectors doesn't work for config tags
• cosmos/dbt/graph.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[tatiana]

@adammarples I just checked on the main branch, and the latest versions of Cosmos, both 1.2.5 and 1.3.0a2 should have this fixed:

astronomer-cosmos/cosmos/dbt/graph.py

Line 112 in 1734365

tags=node_dict["tags"],

Could you upgrade and confirm if it works as expected, please?

[adammarples]

Hi @tatiana this is not a problem with DBT_LS but with load via custom parser

astronomer-cosmos/cosmos/dbt/graph.py

Line 314 in 2878d6a

tags=[],

which is still there in 1.2.5 etc

[dosubot]

🤖

Hi, @adammarples,

I'm helping the Cosmos team manage their backlog and am marking this issue as stale. The issue you raised regarding the load_via_custom_parser function in the graph.py file, which sets the DbtNode.tags attribute to an empty list, has been acknowledged by the bot and discussed by other users. It seems that there was a suggestion for alternative methods that support tags, and there was mention of the issue being fixed in the latest versions of Cosmos. However, you clarified that the problem still persists in the load_via_custom_parser method.

Could you please confirm if this issue is still relevant to the latest version of the Cosmos repository? If it is, please let the Cosmos team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation.

[tatiana]

HI @adammarples , please, could you confirm if this is still an issue in Cosmos 1.4?
Would you like to contribute to the fix?