Source acceptance tests: allow parametrize SATs with dynamic values via fixtures

[davydov-d]

Goal

Introduce the mechanism to parametrize SATs with the dynamic values, e.g. content of the config.json, expected_records.txt files, or cursor_paths value

Motivation

Static parameters are not always enough to make SATs work. For example, Google Ads Source limits the age of the requested data to 90 days, so sooner or later tests will become outdated and will not pass. To make them work, the start_date parameter in the config.json file has always to be up to date, as well as expected_records.txt (contains records with the date field) and cursor_path which is impossible to define in the .yaml file (since it has a dynamic field in its structure: {"<customer_id>": {"segment.date": "<value>"}})

Implementation options

Pytest fixtures seem to be the best option:

• parametrize input fixture with dependent fixtures like config, cursor_path, expected_records
• add empty fixtures from above to a connector template in the CDK

[davydov-d]

@sherifnada @misteryeo another issue to improve SATs jfyi

[davydov-d]

Discussed this issue with the team and decided it is a kind of overengineering, since it's only needed in one connector and

• expected records contain the only stream of a very simple structure
• dynamic parametrizing of config.json would not bring any benefit
• all the edge cases are covered with the custom integration tests