feat(data quality): update models, add assertions cli with snowflake integration

[mayurinehate]

Define data quality rules in DataHub

Step 1: Create assertions specification file
Step 2: Upsert these assertions in DataHub to view data quality expectations on entity page
Step 3: Compile and schedule these assertions to run on assertion backend of your choice (here - snowflake)
Step 4: Ingest the assertion results back in DataHub to view current status for data quality checks.

Assertions Specification File:

Simple yaml file to define assertions in declarative manner.

version: 1
assertions:
    - type: volume
      entity: urn:li:dataset:(urn:li:dataPlatform:snowflake,test_db.public.test_assertions_all_times,PROD)
      volume_metric: row_count
      condition:
          type: between
          min: 5
          max: 15
      schedule:
          type: on_table_change

Assertions CLI:

• Upsert command to add assertions to DataHub
  datahub assertions upsert -f examples/library/assertions_configuration.yml

• Compile command to compile assertions specification to chosen assertion backend.
  datahub assertions compile -f examples/library/assertions_configuration.yml -p snowflake -x DMF_SCHEMA=test_db.datahub_dmfs

Ingestion

For snowflake, run ingestion with include_assertion_results: true to ingest assertion results back into DataHub

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[jjoyce0510]

Do we need this?

[jjoyce0510]

can we please remove?

[jjoyce0510]

CI is passing! Going to go ahead and merge this in. We can continue to iterate from here. Nice MVP!