Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve cosmos logs to always include "astronomer-cosmos" #447

Closed
tatiana opened this issue Aug 8, 2023 · 0 comments · Fixed by #450
Closed

Improve cosmos logs to always include "astronomer-cosmos" #447

tatiana opened this issue Aug 8, 2023 · 0 comments · Fixed by #450
Milestone
1.1.0

Comments

@tatiana
Copy link
Collaborator

tatiana commented Aug 8, 2023

As of 1.0, there is no easy way of isolating logs created by Cosmos from other logs printed in Airflow.

Edgaras Navickas, a community member, proposed in slack that we improve this.

The suggestion is to prefix the project name.

Example of current log message:

Trying to parse the dbt project ……

Suggested log message:

[astronomer-cosmos] Trying to parse the dbt project ……
tatiana added a commit that referenced this issue Aug 9, 2023
@tatiana
Add astronomer-cosmos identifier to project logs
9e9d921 
Closes: #447
tatiana added a commit that referenced this issue Aug 9, 2023
@tatiana
Improve logs to include astronomer-cosmos identifier
b428b16 
Before Cosmos logs looked like standard Airflow logs:
[2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - 13:20:55  Completed successfully

Now they include (astronomer-cosmos):
[2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - (astronomer-cosmos) - 13:20:55  Completed successfully

Closes: #447
@tatiana tatiana mentioned this issue Aug 9, 2023
Merged
tatiana added a commit that referenced this issue Aug 9, 2023
@tatiana
Improve logs to include astronomer-cosmos identifier (#450)
f5b4b7f 
Previously, we were using Airflow standard logging format for Cosmos messages, which looks like:
```
    [2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - 13:20:55  Completed successfully
```

This change introduces a (yellow) "astronomer-cosmos" string into the
project's log messages:
```
    [2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - (astronomer-cosmos) - 13:20:55  Completed successfully
```

It also adds more logs to DBT project parsing to help troubleshoot
issues while rendering the DAG.

Example of how the output looks like:
![Screenshot 2023-08-09 at 15 24
56](https://github.com/astronomer/astronomer-cosmos/assets/272048/cea78586-3f2d-45fc-8209-5975265e9fc8)


Closes: #447
tatiana added a commit that referenced this issue Aug 9, 2023
@tatiana
Improve logs to include astronomer-cosmos identifier (#450)
db946b2 
Previously, we were using Airflow standard logging format for Cosmos messages, which looks like:
```
    [2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - 13:20:55  Completed successfully
```

This change introduces a (yellow) "astronomer-cosmos" string into the
project's log messages:
```
    [2023-08-09T14:20:55.532+0100] {subprocess.py:94} INFO - (astronomer-cosmos) - 13:20:55  Completed successfully
```

It also adds more logs to DBT project parsing to help troubleshoot
issues while rendering the DAG.

Example of how the output looks like:
![Screenshot 2023-08-09 at 15 24
56](https://github.com/astronomer/astronomer-cosmos/assets/272048/cea78586-3f2d-45fc-8209-5975265e9fc8)


Closes: #447
@tatiana tatiana added this to the 1.1.0 milestone Sep 7, 2023
@tatiana tatiana mentioned this issue Sep 15, 2023
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.1.0
Development

Successfully merging a pull request may close this issue.

Improve logs to include astronomer-cosmos identifier astronomer/astronomer-cosmos
1 participant
@tatiana