-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIRFLOW-5088][AIP-24][BackPort] Persisting serialized DAG in DB for webserver scalability #67
Commits on Oct 23, 2019
-
[AIRFLOW-5088][AIP-24] Add DAG serialization using JSON (apache#5701)
It implements the method proposed in AIP-24 to serialize DAG. It will be used in DAG persistency in DB to solve webserver scalability issue. (cherry picked from commit 2bd1a51ec75f680a6e6e2101bd948a78421a644a)
Configuration menu - View commit details
-
Copy full SHA for d984217 - Browse repository at this point
Copy the full SHA d984217View commit details -
Configuration menu - View commit details
-
Copy full SHA for a3cfc3e - Browse repository at this point
Copy the full SHA a3cfc3eView commit details -
Make _primitive_types Py2 & Py3 compatible
(cherry picked from commit 7ce34b2a959fc1f8322836f38f474a831e4901a1)
Configuration menu - View commit details
-
Copy full SHA for d16df59 - Browse repository at this point
Copy the full SHA d16df59View commit details -
Fix issue with different class for Pendulum Timezones
(cherry picked from commit c068c67c48d294a589b58be0d0ad8b657c361a77) (cherry picked from commit 04fbf2beac57dcf26b118ebbe5a2bf175ce08af8)
Configuration menu - View commit details
-
Copy full SHA for 098ea82 - Browse repository at this point
Copy the full SHA 098ea82View commit details -
Update timezone class & Do not serialize dates in tasks if they have …
…matching date in DAG (cherry picked from commit be412522cb95a19a51b2f208ae8ebea76e8b667a)
Configuration menu - View commit details
-
Copy full SHA for 7d53a03 - Browse repository at this point
Copy the full SHA 7d53a03View commit details -
Change type of data column to JSON & Add metric for dagbag size
(cherry picked from commit d030b10bec9cd0e468f36e97e131d497d5a43fc6)
Configuration menu - View commit details
-
Copy full SHA for 9b628a2 - Browse repository at this point
Copy the full SHA 9b628a2View commit details -
- Code Cleanup for JSON columns - Test code to allow old mysql & sqlite versions (cherry picked from commit 1db8044f9d29edf25f2b8ad4cd21c496c243534a)
Configuration menu - View commit details
-
Copy full SHA for 690e002 - Browse repository at this point
Copy the full SHA 690e002View commit details -
(cherry picked from commit d14497ff28d123d45d626019cabcbd977c5de79d)
Configuration menu - View commit details
-
Copy full SHA for e8003c0 - Browse repository at this point
Copy the full SHA e8003c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 04af56f - Browse repository at this point
Copy the full SHA 04af56fView commit details -
Support dateutil.relativedelta in SerializedDAGs
This was a valid type for schedule_interval already, so we should continue supporting it (cherry picked from commit ec9d705f1a90790bdcb099196269c77d3cc3d53c) (cherry picked from commit 9805b4a183b87976dc33ae80c7e6a209849ba5d7) (cherry picked from commit f00d9237cd9224571e43bda67ad4dddfb009c402)
Configuration menu - View commit details
-
Copy full SHA for b4cc558 - Browse repository at this point
Copy the full SHA b4cc558View commit details -
Configuration menu - View commit details
-
Copy full SHA for ed722f2 - Browse repository at this point
Copy the full SHA ed722f2View commit details -
(cherry picked from commit 92d442d33dd8c81ea73026405d3978d133140807) (cherry picked from commit 7d371d329613c48deef0d8a812c817f2013db8f9)
Configuration menu - View commit details
-
Copy full SHA for 6b70e54 - Browse repository at this point
Copy the full SHA 6b70e54View commit details -
(cherry picked from commit 549c1f9cd9ab0bfeac4f75fa713cbaae842a6e82)
Configuration menu - View commit details
-
Copy full SHA for 742e8ae - Browse repository at this point
Copy the full SHA 742e8aeView commit details -
fix bugs that date/time/IntEnum are not supported in serialization.
(cherry picked from commit d0ce27e3f3b6046016800855ad2e57fa67d8b57f) (cherry picked from commit 50a60b6a026e6d6249f069944be86560d87a67ca)
Configuration menu - View commit details
-
Copy full SHA for 6710b96 - Browse repository at this point
Copy the full SHA 6710b96View commit details -
Deactivate DAGs instead of deleting if their DAG file is deleted
(cherry picked from commit 5a84ca517cef0dacff23f57b360a554b461b5034)
Configuration menu - View commit details
-
Copy full SHA for f7165a9 - Browse repository at this point
Copy the full SHA f7165a9View commit details -
(cherry picked from commit 712ff47cbada7373eaa4fa92bb9220a453c445ae) (cherry picked from commit 52a0e9e39dc006501eb9d8ac0881357900548cf7)
Configuration menu - View commit details
-
Copy full SHA for 4728d2f - Browse repository at this point
Copy the full SHA 4728d2fView commit details -
Just-in-time loading of DagBag in webserver
To save start-up time (and memory) this changes the DabBag to not be populated by the webserver on start up - and when a specific dag is asked for it will be loaded on-demand from the SerializedDAG table. Co-Authored-By: Ash Berlin-Taylor <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 4f05456 - Browse repository at this point
Copy the full SHA 4f05456View commit details -
(cherry picked from commit e91ad24b006823eadd6f3e21fc7cc5c8dd57b0d1)
Configuration menu - View commit details
-
Copy full SHA for 9060783 - Browse repository at this point
Copy the full SHA 9060783View commit details -
Add default args to decorated_fields
(cherry picked from commit 3f08d2f986364315c3e43bde3524f12d069392ae)
Configuration menu - View commit details
-
Copy full SHA for 02f85b3 - Browse repository at this point
Copy the full SHA 02f85b3View commit details -
[AIRFLOW-5636] Allow adding or overriding existing Operator Links (ap…
…ache#6302) (cherry-picked from 1de210b)
Configuration menu - View commit details
-
Copy full SHA for bf94658 - Browse repository at this point
Copy the full SHA bf94658View commit details -
ExtraOperatorLinks are supported if Plugins are registered for them (cherry picked from commit 9cb6e28) (cherry picked from commit 72c75860ecfcd1930f1dedc7a0c713f122ea51a5)
Configuration menu - View commit details
-
Copy full SHA for f61ca29 - Browse repository at this point
Copy the full SHA f61ca29View commit details -
(cherry picked from commit e840616) (cherry picked from commit 6d01d8e5bac1b6e829b9da6fc50c1a4b6d23bcaf)
Configuration menu - View commit details
-
Copy full SHA for 83572c6 - Browse repository at this point
Copy the full SHA 83572c6View commit details -
Move serialization directory out of dags folder
(cherry picked from commit 8a07aee3e5cf133c45ee4ae26aad6104c84502ab)
Configuration menu - View commit details
-
Copy full SHA for 6470e2a - Browse repository at this point
Copy the full SHA 6470e2aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d60f5f - Browse repository at this point
Copy the full SHA 5d60f5fView commit details -
[AIRFLOW-5268] Apply same DAG naming conventions as in literature (ap…
…ache#5874) cherry-picked from apache@8f6ca53
Configuration menu - View commit details
-
Copy full SHA for 0e0037c - Browse repository at this point
Copy the full SHA 0e0037cView commit details -
Configuration menu - View commit details
-
Copy full SHA for cbd41b4 - Browse repository at this point
Copy the full SHA cbd41b4View commit details -
[AIRFLOW-5481] Allow Deleting Renamed DAGs (apache#6101)
(cherry picked from commit 99a5c2e)
Configuration menu - View commit details
-
Copy full SHA for 0429902 - Browse repository at this point
Copy the full SHA 0429902View commit details
Commits on Oct 24, 2019
-
Fix bad merge_conflict resolution
This was incorrectly removed while cherry-picking and resolving conflicts
Configuration menu - View commit details
-
Copy full SHA for 23ea1bc - Browse repository at this point
Copy the full SHA 23ea1bcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7deb805 - Browse repository at this point
Copy the full SHA 7deb805View commit details -
Configuration menu - View commit details
-
Copy full SHA for cde0c0a - Browse repository at this point
Copy the full SHA cde0c0aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1085f2f - Browse repository at this point
Copy the full SHA 1085f2fView commit details