Implement materialization for RedshiftOfflineStore & RedshiftRetrievalJob

[tsotnet]

Signed-off-by: Tsotne Tabidze [email protected]

What this PR does / why we need it: This PR makes it possible to materialize data from Redshift to online stores. Also added half of test_offline_online_store_consistency.py for Redshift (for now we're only asserting online store after materialization from Redshift and don't check offline store by calling get_historical_features, since that functionality is not yet implemented). This required implementing unload/copy logic through S3. RedshiftRetrievalJob is also fully implemented (to_df, to_arrow, to_s3, to_redshift).

Lastly, needed to add iam_role to the redshift config. This role is necessary for Redshift to access S3. It'll need to be setup by the user (both the role creation, and association to Redshift).

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Adds materialization logic for RedshiftOfflineStore

[codecov-commenter]

Codecov Report

Merging #1680 (fa0c494) into master (0d0492d) will decrease coverage by 13.64%.
The diff coverage is 36.79%.

@@             Coverage Diff             @@
##           master    #1680       +/-   ##
===========================================
- Coverage   82.75%   69.10%   -13.65%     
===========================================
  Files          76       76               
  Lines        6754     6817       +63     
===========================================
- Hits         5589     4711      -878     
- Misses       1165     2106      +941     

Flag	Coverage Δ
integrationtests	?
unittests	69.10% <36.79%> (-0.63%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdk/python/tests/test_entity.py	100.00% <ø> (+15.21%)	⬆️
sdk/python/feast/data_source.py	60.15% <6.66%> (-6.52%)	⬇️
sdk/python/feast/type_map.py	41.66% <31.57%> (-0.80%)	⬇️
sdk/python/feast/infra/utils/aws_utils.py	32.60% <32.60%> (ø)
sdk/python/feast/infra/offline_stores/redshift.py	55.93% <34.21%> (ø)
...hon/tests/test_offline_online_store_consistency.py	58.33% <55.55%> (-41.67%)	⬇️
...dk/python/tensorflow_metadata/proto/v0/path_pb2.py	100.00% <100.00%> (ø)
.../python/tensorflow_metadata/proto/v0/schema_pb2.py	100.00% <100.00%> (ø)
...hon/tensorflow_metadata/proto/v0/statistics_pb2.py	100.00% <100.00%> (ø)
sdk/python/tests/test_online_retrieval.py	17.39% <0.00%> (-82.61%)	⬇️
... and 38 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d0492d...fa0c494. Read the comment docs.

[woop]

/lgtm

[feast-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tsotnet, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tsotnet,woop]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment