AIP-32 - Airflow REST API - High-level info

[mik-laj]

We currently have one [experimental API|https://airflow.readthedocs.io/en/latest/rest-api-ref.html], but despite its existence for 2 years, it has not reached a stable level.

The Polidea and Google teams together with the community want to make another attempt based on our and community experience. Airflow deserves new stable solutions.

Details are available in the AIP:

This is the first ticket regarding AIP-32 on Github. I will try to update it regularly to inform you about the high-level status of this task.

We have merged spec. You could preview it using the following links:
Swaagger UI:
https://editor.swagger.io/?url=https://raw.githubusercontent.com/apache/airflow/master/airflow/api_connexion/openapi/v1.yaml
Redoc:
https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/apache/airflow/master/airflow/api_connexion/openapi/v1.yaml

This task consists of the following areas:

• API Endpoints
• Docs for REST API

In addition, we have the following tasks:

• Basic OpenAPI spec
• Basic integration Airflow and connexion
• (de)serialization for Python objects
• (de)serialization common objects
• (de)serialziation for SQLAlchemy objects
• HATEOS for API
• CRUD Framework for API
• Authorization and Permissions
• Authentication in API
• Custom WEB UI screen to control permissions
• API security tests

Resources:

AIP-32 on Wiki:
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-32%3A+Airflow+REST+API
Discussion about AIP on the mailing list:
https://lists.apache.org/thread.html/rbcee49452f1e2714fbdb91f12cfcb115d24681d01ccb6a3845a7e699%40%3Cdev.airflow.apache.org%3E
Voting: on the mailing list
https://lists.apache.org/thread.html/rcc379dc7067397e1a631c19b9f194f2f572a2ea7a338648788911c91%40%3Cdev.airflow.apache.org%3E

Contribution

We invite everyone to contribute. We have #sig-api to decisions and to coordinate our work.
Registration link: https://apache-airflow-slack.herokuapp.com/
All changes are labelled "area:API" on Github.
Information about current tasks are available on Github Project: https://github.com/apache/airflow/projects/1

[mik-laj]

Continuation of the API discussion. https://lists.apache.org/thread.html/re01055f493e93ba87dd01b98270446dfc4324adc0356c3a4eda34c4a%40%3Cdev.airflow.apache.org%3E

[ryw]

Hi @mik-laj where is best place to see what has been completed, and what's left to do for the initial release of the new REST API? I see a list of tickets above, is that still comprehensive? If you're doing some internal project tracking, could you share a snapshot of progress here?

[potiuk]

I believe it's this: https://github.com/apache/airflow/projects/1

[ryw]

ah thank you!

[potiuk]

From the very beginning of the project we track it in this project and @mik-laj excellently planned the work and managed the status all the time in the project (Kudos @mik-laj ), so I am quite sure it accurately reflects the state).

[ashb]

Is this done now that the last endpoint is merged @mik-laj ?

[mik-laj]

@ashb Yes. It is complete. Other tasks are optional improvements but are not necessary to complete this project.