airflow: implement DagRun listener

[mobuchowski]

This PR implements DagRun listener based on apache/airflow#27113

In effect, OpenLineage integration will now explicitely emit DagRun start and DagRun complete or DagRun failed events. This allows precise tracking of single dags, while without this PR OpenLineage consumer does not understand when the dag finished - can merely guess that it finished because there's lack of running tasks for this dag.

Signed-off-by: Maciej Obuchowski [email protected]

[codecov-commenter]

Codecov Report

Merging #1286 (c185360) into main (6c839f9) will decrease coverage by 11.52%.
The diff coverage is 42.10%.

@@              Coverage Diff              @@
##               main    #1286       +/-   ##
=============================================
- Coverage     79.91%   68.38%   -11.53%     
  Complexity       74       74               
=============================================
  Files            84       84               
  Lines          3833     3868       +35     
  Branches         25       25               
=============================================
- Hits           3063     2645      -418     
- Misses          736     1189      +453     
  Partials         34       34               

Impacted Files	Coverage Δ
...ntegration/airflow/openlineage/airflow/listener.py	73.46% <40.00%> (-10.32%)	⬇️
integration/airflow/openlineage/airflow/adapter.py	83.75% <46.15%> (-7.56%)	⬇️
...flow/openlineage/airflow/extractors/example_dag.py	0.00% <0.00%> (-100.00%)	⬇️
...w/openlineage/airflow/extractors/sftp_extractor.py	25.00% <0.00%> (-69.24%)	⬇️
...rflow/openlineage/airflow/extractors/converters.py	37.50% <0.00%> (-62.50%)	⬇️
...neage/airflow/extractors/redshift_sql_extractor.py	35.89% <0.00%> (-61.54%)	⬇️
...penlineage/airflow/extractors/sql_execute_query.py	40.00% <0.00%> (-60.00%)	⬇️
...flow/openlineage/airflow/extractors/dbapi_utils.py	30.76% <0.00%> (-60.00%)	⬇️
...enlineage/airflow/extractors/bigquery_extractor.py	34.69% <0.00%> (-51.03%)	⬇️
...ow/openlineage/airflow/extractors/sql_extractor.py	38.73% <0.00%> (-50.46%)	⬇️
... and 13 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more