Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prevent large objects from being stored in the RTIF #38094

Merged
merged 16 commits into from
Mar 25, 2024
Merged

Prevent large objects from being stored in the RTIF #38094

merged 16 commits into from
Mar 25, 2024

Commits on Mar 25, 2024

  1. Prevent large objects from being stored in the RTIF 

    There's no control over the size of objects stored in the rendered taskinstance
field. This PR adds control and enable users to be able to customize the size of
data that can be stored in this field

closes: apache#28199
    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    9d171d9 View commit details
    Browse the repository at this point in the history

  2. fixup! Prevent large objects from being stored in the RTIF

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    0e7fd98 View commit details
    Browse the repository at this point in the history

  3. Use len and check the size of the serialized

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    68b639a View commit details
    Browse the repository at this point in the history

  4. Add test to already existing test

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    feb23d3 View commit details
    Browse the repository at this point in the history

  5. Remove xcom db clearing

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    e64b5a4 View commit details
    Browse the repository at this point in the history

  6. Update airflow/config_templates/config.yml 

    Co-authored-by: Jed Cunningham <[email protected]>
    @ephraimbuddy @jedcunningham
    ephraimbuddy and jedcunningham committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    7f126de View commit details
    Browse the repository at this point in the history

  7. Apply review suggestions

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    37914ab View commit details
    Browse the repository at this point in the history

  8. Add test for redacting values and add significant item

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    d2fe6ec View commit details
    Browse the repository at this point in the history

  9. Prefix with Truncated line

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    7ed2217 View commit details
    Browse the repository at this point in the history

  10. Ensure secrets are masked

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    97b4826 View commit details
    Browse the repository at this point in the history

  11. fixup! Ensure secrets are masked

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    4f799e2 View commit details
    Browse the repository at this point in the history

  12. Check the template field length in separate branches for jsonable and… 

    … nonjsonable
    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    6e7d30c View commit details
    Browse the repository at this point in the history

  13. add tests for rendered dataframe

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    dfb6d5f View commit details
    Browse the repository at this point in the history

  14. Apply suggestions from code review

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    0358d7e View commit details
    Browse the repository at this point in the history

  15. update code and tests

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    8aa3962 View commit details
    Browse the repository at this point in the history

  16. fixup! update code and tests

    @ephraimbuddy
    ephraimbuddy committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    84544df View commit details
    Browse the repository at this point in the history