[EPIC] SSH tunnel support for OLTP DB sources & destinations #5901
Labels
area/connectors
Connector related issues
lang/java
lang/python
priority/high
High priority
type/enhancement
New feature or request
Tell us about the problem you're trying to solve
This epic tracks adding support for connecting to OLTP databases via SSH tunnels.
Must-have DBs for Airbyte Cloud (in priority order)
Postgres source & Destination (already done)Nice-to-have
Investigation of simulating an SSH bastion using Docker for running acceptance tests
Describe the solution you’d like
There's roughly two logical components here:
Depending on whether you are adding a source or a destination the impl is slightly different. See the instructions for how to add SSH support to a connector
Example PR of adding SSH support to the Postgres source
Example of adding SSH support to Postgres destination
Note that both these PRs have way more logic in them than what you'll need to implement for a single connector because they add common helpers and shared libraries and docs
A note on implementation ordering
While testing via Docker (instead of creating actual DB instances on AWS) is highly desirable, it is optional. On the other hand, we must deliver SSH tunnels for the above DBs by the end of the month to support the Airbyte Cloud launch. My recommendation is to work on both pieces simultaneously to minimize delivery risk:
Postgres Source ssh key creds
anddestination postgres test creds with sshkey
. If Terraform is slowing you down too much feel free to create one in the UI.The complicating factor here is, if standing up infra in step #1 takes a long time then maybe it's a bad use of time if we can deliver #2 first. But this approach is just suggested to minimize risk as much as possible.
The text was updated successfully, but these errors were encountered: