v0.12.4

What's Changed

• Enable raw sync events rule by @philerooski in #150

Full Changelog: v0.12.3...v0.12.4

v0.12.3

What's Changed

• [ETL-694] Specify python-version in actions/setup-python by @philerooski in #148

Full Changelog: v0.12.2...v0.12.3

v0.12.2

What's Changed

• [ETL-611] Raw sync lambda by @philerooski in #141

Full Changelog: v0.12.1...v0.12.2

v0.12.1

What's Changed

• [ETL-688] Handle "Deleted" data types correctly in raw lambda by @philerooski in #143

Full Changelog: v0.12.0...v0.12.1

v0.12.0

What's Changed

JSON to Parquet

• [ETL-675] Update Glue tables to use compressed JSON by @philerooski in #137

Raw JSON Data

• [ETL-636] Raw Lambda by @philerooski in #138

Great Expectations

• [ETL-616] Implement Great Expectations to run on parquet data by @rxu17 in #139

Patches

• Add HealthKitV2Electrocardiogram type to large jobs by @philerooski in #142
• [ETL-683] Increase memory of raw lambda by @philerooski in #140

Full Changelog: v0.11.2...v0.12.0

v0.11.2

What's Changed

• [ETL-674] Add script to compress intermediate JSON by @philerooski in #134
• [ETL-673] Upload compressed JSON as part of S3 to JSON by @philerooski in #135

Full Changelog: v0.11.1...v0.11.2

v0.11.1

What's Changed

Snowflake

• [ETL-676] Create data loading procedure for each parquet datatype by @philerooski in #132
• [ETL-677] Add bootstrap script for Snowflake Parquet tables by @philerooski in #133

Full Changelog: v0.11.0...v0.11.1

v0.11.0

What's Changed

Great Expectations

• [ETL-658] Bucket for hosting static HTML files for GX reports by @BryanFauble in #120

Developer QoL Improvements

• [ETL-654] Clean up before integration test run by @BryanFauble in #121
• [ETL-670] Setup and run pre-commit for all files by @BryanFauble in #122
• [ETL-670] Adjust cleanup job to run on main directory of dev env by @BryanFauble in #126
• [ETL-670] Update code to paginate over all results by @BryanFauble in #128

Bug fixes

• [ETL-671] Update to use older node version on aws actions by @rxu17 in #124

New Infrastructure

• [ETL-635] Add dispatch-to-raw SQS queue by @philerooski in #125

Snowflake

• [ETL-659] Snowflake git integration by @philerooski in #123
• [ETL-659] Enable any branch to be deployed to any environment by @philerooski in #127
• Add cleanup job for Snowflake dev environments by @philerooski in #129
• Fix Snowflake cleanup by @philerooski in #130
• Specify default warehouse in Snowflake config by @philerooski in #131

Full Changelog: v0.10.0...v0.11.0

v0.10.0

What's Changed

New Features:

• [IBCDPE-923] snowflake integration by @thomasyu888 in #112
• [ETL-648] Modify comparsion job to only consider records from recent exports by @rxu17 in #119

Full Changelog: v0.9.0...v0.10.0

v0.9.0

What's Changed

New Features:

• [ETL-634] Implement Dispatch Lambda by @philerooski in #114. This is not a feature by itself, but is related to our work around building data quality infrastructure.

Bug Fix:

• [ETL-647] Only submit most recent ~2 weeks of exports for integration tests by @philerooski in #115
• [ETL-653] Add corresponding deleted data type table for all HealthKit data types by @philerooski in #116
• [ETL-655] Add Device field to fitbitdevices INDEX_FIELD_MAP by @philerooski in #117
• [ETL-656] Limit input-to-intermediate Lambda batch size to 1 by @philerooski in #118

Full Changelog: v0.8.0...v0.9.0