Add OpenAPI specification (II) (#8721)

* Add OpenAPI spec (#7549)

* Fix typo in name of pre-commit hook

* Chaange type for DAGID, DAGRunID, TaskID

* Fix typo in summary - POST /pools

* Fix typo in description - FileToken parameter

* Fix typo - singular/plural form - variables

* Make EventLog endpoints read-only

* Use ExcutionDate in DagRuns endpoints

* Use custom action to control task instances

* Typo in  DELETE Task instance

* Remove unused schema - DagStructureCollection

* Fix typo - singular/plural form - import errors

* Add endpoint - POST /dagRuns

* Remove job_id

We do not have endpoints to download jobs, because this is an implementation detail, so this field has big no value.

* Add filters to GET /taskInstances

* Fix typo - upadtePool => updatePool

* Rename "Create a DAG Run" to "Trigger a DAG Run"

* Use Pool name as a parameter

* Add filter to GET /dagRuns

* Remove invalid note ion start_date field

* Uss POST instead of PATCH for custom action

* Remove DELETE /taskInstances endpooint

* Rename Xcom Value to xcom Entry

* Fix typo in XCCOM Entry endpoint

* Change operationID: patchConnection => updaateConnection

* Make execution_date optionall in DAGRun

This field can be filled with the current date when creating the DAG Run

* Unify connection ID

* Use URL with HTTPS and without www.

* Fix typo - at database => in database

* Fix typo = Raange -> Raange

* Fix typo - the specific DAG => a DAG

* Fix typo - getXComVEntry => getXComVEntry

* Unify collection names - xcomEntries

* Move TaskInstance resource under DagRun resource

* Fix typo - change tag - TaskInstance => TaskInstance

Co-authored-by: Ash Berlin-Taylor <[email protected]>

* Use path paramaters for /variables/lookup/ endpoint

* Use consistent names for IDs

* Use new style for filter parameters

* Remove unused path parameter

* Use ~ as a wildcard charaacter

* Add batch endpoints for TaskInstance and DagRuns

* Fix typo - response in trigger dag endpoint

* Fix typo - Qqueue => Queue

* Set dry_run = True in ClearTaskInstance

* Mark all fieldss (expcet state) of DagRun as read-only

* Use __type as a discriminator

* Fix typo - "The Import Error ID." => "The Event Log ID."

* Fix typo - Self referential in EventLogCollection

* Rename fieldss - dttm => when

* remove fields - pool_id

* Fix typo - change request body in PATCH /pools/{pool_name}

* Use DAG Run ID as a primary identifier

* Fix typo - Change type of query to string

* Unify fields names in collections

* Use variable key as a primary id

* Move collection to /variables

* Mark passord as a write only

* Fix typo - updaateConnection => updateConnection

* Change is_paused/is_subdag to boolean

* Fix typo - clearTaskInstaance => clearTaskInstance

* Fix typo - DAAG => DAG

* Fix typo - many => multiple

* Fix typo - missing "a"

* Fix typo - variable by id => variable by key

* Fix typo - updateXComEntries => updateXComEntry

* Fix typo - missing "a"

* Use dag_run_id as a primary ID

* Fix typo - objectss => objects, DAG IDS => DAG IDs

* Allows create DAG Run with custom conf/execution_date/dag_run_id

* Add new trigger rule, fix typo in dag run state

* Add request body to POST/PATCH /dags/{dag_id}

* Rename collection fields - dag_model => dags

* Fix typo - /clearTaskInstanaces -> /clearTaskInstances

* Improve wording - wildcard

* Returns owners as a array

* Return only references in clear task instances

* Remove support for application/x-www-form-urlencoded

* fixup! Use __type as a discriminator

* Add file_token fields

* Move description of variable collections

* Return SUB DAG in DAG structure

* Fix typo - sucess => sucess, Apache Foundation => Apache Software Foundation, Airfow => Apache Airflow

* Improve description of get logs endpoint

* Fix typo - Get all XCom entry => Get all XCom entries

* Add crossreference between /dags/{dag_id}/structure and /dags/{dag_id}

* Remove all form-urllencoded request bodies

* Rename parameter - NoChunking => FullContent

* Improve description of batch endpoints

* Remove request body for GET endpoint

* Use allOf insteaad of oneOf

* Rename key => xcom_key

* Use lowercase letters in query string parameter - Queue -> queue

* Change type of conf to object

* Change allOf into oneOf for ScheduleInterval

Co-authored-by: Ash Berlin-Taylor <[email protected]>

.pre-commit-config.yaml

@@ -175,6 +175,18 @@ metastore_browser/templates/.*\\.html$|.*\\.jinja2"
         entry: koalaman/shellcheck:stable -x -a
         files: ^breeze$|^breeze-complete$|\.sh$|^hooks/build$|^hooks/push$|\.bash$|\.bats$
         exclude: ^airflow/_vendor/.*$
+      - id: lint-openapi
+        name: Lint OpenAPI using speccy
+        language: docker_image
+        entry: wework/speccy lint
+        files: ^openapi.yaml$
+      - id: lint-openapi
+        name: Lint OpenAPI using openapi-spec-validator
+        entry: openapi-spec-validator --schema 3.0.0 openapi.yaml
+        language: python
+        additional_dependencies: ['openapi-spec-validator']
+        pass_filenames: false
+        files: ^openapi.yaml$
       - id: lint-dockerfile
         name: Lint dockerfile
         language: system