Filter config objects matching workspace via database not in Java #10056
Labels
area/platform
issues related to the platform
team/compose
team/platform-move
technical-debt
issues to fix code smell
We currently filter out configuration objects to match the requested workspace in Java code, after loading all configuration objects from the persistence layer (i.e. DB), see e.g. https://github.com/airbytehq/airbyte/blob/master/airbyte-server/src/main/java/io/airbyte/server/handlers/ConnectionsHandler.java#L247 (though it seems we're doing this for all configuration objects).
I suggest we move filtering by workspace ID into the database query instead. Having this in Java is potentially a performance issue while the amount of workspaces grow. This might not be a problem for self hosted instances, where even across all workspaces the amount of configuration objects might be low, but especially in our cloud those configuration objects might incrase significantly. Filtering them in Java will cause us:
This could already as of today potentially be the reason for slowdowns we're seeing only in cloud and not with custom clusters: #7985 (comment)
The text was updated successfully, but these errors were encountered: