Create a Tech Spec/Plan for "Seed all streams for GA connectors & enable tests"

[evantahler]

Prior Art:

• Scoping improvements on output guarantees

While discussing Connector Performance Benchmarking & the E2E Testing Tool, we realized for the immediate goal of testing the low-code CDK, we care more about robust SAT test data than we do about a full E2E testing tool. To that end, we should start working on this project sooner rather than later.

Rather than focus on the E2E testing framework for now, we prefer to spend the time ensuring that all sources have a robust SAT test suite which includes expected_records for every stream

Phase 1

1. Identify GA connectors which do not have expected_records for every stream
2. Are there any streams which we cannot seed?
3. Add expected_records for streams which we have test data for

Phase 2

For the streams we cannot seed we need to choose what to do:

• Ask customers for permission to use their streams?
• Make a tool to seed data to streams?
• Mock API data with something like VCR?

Possible Discussion Topics

• Do we downgrade connectors that aren't passing?
• What does it look like while we are adding these tests/streams and a publish is needed?

[alafanechere]

Tech spec ready for review https://docs.google.com/document/d/19hnZslQUbvxuHeFQd9hbzVQmuK4INzDcwiXEA7FwA4U/edit