add feature-flag client

[colesnodgrass]

What

• fix create a feature-flag client #20830
• add an airbyte-featureflag module
• add a Feature-Flag client; supports
  • local feature-flags (via config file)
  • remote feature-flags (currently via LaunchDarkly)
  • env-var feature-flags (intended to only be supported during the migration phase)
• introduces Kotlin 1.8.0
• Feature-Flag Client is not actually used as part of this PR

How

See the details section below

Todo

• finalize naming of classes
• show how to use this (working on this as a separate PR)
• figure out where flags.yml should exist

Recommended reading order

1. Client.kt
2. Flags.kt
3. Context.kt
4. Factory.kt

Details

There is a lot going on in this PR, I'm going to attempt to break it down into bite-size descriptive chunks below for ease of understanding.

Design

Client

The goal of this feature-flag client is to support both cloud-hosted and non-cloud-hosted configurations.

This has been accomplished by defining a single interface

/**
 * Feature-Flag Client interface.
 */
sealed interface FeatureFlagClient {
  /**
   * Returns true if the flag with the provided context should be enabled. Returns false otherwise.
   */
  fun enabled(flag: Flag, ctx: Context): Boolean
}

with multiple implementations; ConfigFileClient and LaunchDarklyClient.

graph TD
    I(FeatureFlagClient) --> P(ConfigFileClient)
    I --> C(LaunchDarklyClient)
    C --> L(LaunchDarkly)
    P --> Y(flags.yml)
    P -.-> E(Env Vars)
    C -.-> E
    style E stroke-dasharray: 3

Loading

To know the current state of a feature-flag, provide a Flagobject and a Context object (more information about these concepts in their sections below) to the enabled method. The returned boolean indicates the current status of the Flag with the Context.

In order for this Client to determine that status of the requested Flag and Context, the ConfigFileClient relies on a flags.yml configuration file while the LaunchDarklyClient relies on an external service (LaunchDarkly).

As this repo already has some notion of environment based feature-flags, all implementations of the FeatureFlagClient will initially support flags managed by environment-variables. The near-term goal is to replace all environment-variables flags with flags.yml entries and LaunchDarkly flags.

This PR contains a Micronaut @Factory for automatically determining which FeatureFlagClient should be used. If the application configuration airbyte.feature-flag.path is defined, the ConfigFileClient will be utilized. If airbyte.feature-flag.api-key is defined, then LaunchDarklyClient will be utilized.

Note
What should happen if both fields are defined? What about if neither are defined?
Are these both error states?

New Client types cannot be defined outside of the airbyte-featureflag module as the Client type is sealed with no open implementations.

Flags

A Flag is defined as

/**
 * Flag is a sealed class that all feature-flags must inherit from.
 *
 * @param [key] is the globally unique identifier for identifying this specific feature-flag.
 * @param [default] is the default value of the flag.
 * @param [team] is the team that is responsible for the feature-flag, defaults to Unknown.
 */
sealed class Flag(
  internal val key: String,
  internal val default: Boolean = false,
  internal val team: Team = Team.UNKNOWN,
)

Where the key is the unique identifier of the flag (must match the name defined in both the flags.yml and LaunchDarkly). The default value is the value returned if the flag's state cannot be found or determined. The team is the owning team responsible for managing this feature-flag and only exists for ownership and tracking reasons.

There are currently three types of feature flags supported by this client; Permanent, Temporary, and EnvVar.

Permanent flags are those which will always exist and are not necessarily part of a feature. Presently there is only a single permanent flag which is currently controlled by the env-var LOG_CONNECTOR_MESSAGES (this should migrate away from being env-var controlled soon after this PR is merged). Permanent flags should extend the Flag type directly

object LogConnectorMessages : Flag(key = "LOG_CONNECTOR_MESSAGES", team = Team.PLATFORM_MOVE)

Temporary flags will be the majority of feature flags, as these flags will only exist for a finite period of time, being removed entirely once a feature has been completely and successfully released. Temporary flags should extend the Temporary type

object AutoDetectSchema : Temporary(key = "AUTO_DETECT_SCHEMA", team = Team.PLATFORM_MOVE)

There is no real difference between a Permanent and Temporary flag other than an indication of the lifecycle intent of the flag itself.

EnvVar flags should only exist for a small period of time as the current env-var based feature-flag solution is replaced with this more well defined and managed solution. More bluntly the EnvVar class itself will be removed in the near future. EnvVar flag statuses are determined entirely by the environment-variable they are watching and cannot be changed without restarting the services. EnvVar flags must extend the EnvVar type

object NeedStateValidation : EnvVar(envVar = "NEED_STATE_VALIDATION")

Ideally all feature flags would be defined within the `Flags` file, but this will be be enforced (at least initially) for `Temporary` and `EnvVar` flags.  This is why both of those classes are defined as `open`.  Any `Permanent` flag that extends the `Flag` type _must_ be [declared within the `airbyte-featureflag` module](https://kotlinlang.org/docs/sealed-classes.html#location-of-direct-subclasses) as the `Flag` type is `sealed`.

Contexts

A Context adds additional context to the current request of the status of a feature-flag. For example, the user or workspace that is making the request. As feature-flags may only be enabled for a subset of users or workspaces, this additional information is necessary in order to determine the correct state of the feature-flag being requested.

A Context is defined as

sealed interface Context {
  val kind: String
  val key: String
}

There are currently two implementations; User and Workspace. User represents a user as identified by their unique-id and Workspace represents a workspace as identified by its unique-id (and optionally a user-id as well).

Note
I'm not sure if supporting a User context is necessary, as discussions around feature-flags tend to focus on handling them at the Workspace level.

New Context types cannot be defined outside of the airbyte-featureflag module as the Context type is sealed with no open implementations.

flags.yml

The configuration file for determining the status of a feature-flag for non-cloud deployments must be in the format of

flags:
  - name: feature-flag-example-one
    enabled: true
  - name: feature-flag-example-two
    enabled: false

Currently flags defined in the flags.yml do not respect Contexts. This may be added at a later date.

Kotlin

This PR introduces Kotlin to the platform, specifically Kotlin 1.8.0. There has been multiple discussions regarding adding Kotlin into our code base and this new feature-flag client seemed like the opportune time.

Kotlin Features Utilized

There are a number of Kotlin specific features that are utilized within this PR. As this is the first Kotlin PR I'm going to briefly touch on a few of them. If there are any further questions, please reach out to me.

• Multiple Types Defined Within a Single File
  From Kotlin's coding conventions

  Placing multiple declarations (classes, top-level functions or properties) in the same Kotlin source 
  file is encouraged as long as these declarations are closely related to each other semantically, 
  and the file size remains reasonable (not exceeding a few hundred lines)
  

  This is utilized in nearly every file added in this PR. Technically Java does allow this, but only for sealed classes, which we are already utilizing.

• Data Classes
  Data classes are similar to Java's record type, but with more flexibility. There are only a few data classes in this PR.

• init Blocks
  A primary constructor in Kotlin cannot contain any code, init blocks are utilized for these specific purposes.

• Internal Visibility
  Kotlin supports different visibility modifiers than Java and this PR takes advantage of them; specifically private and internal. Where private is only visible within the file its defined in (like these data classes used for parsing the config file) and internal is visible to the entire module (like this primary constructor which can only be called from within the airbyte-featureflag module).

• Default Parameters, Named Parameters, and @jvmoverloads
  Kotlin supports both defaults parameters and named parameters. Both of which reduce the number of methods necessary to support optional parameters or for ensuring that method parameters are passed in a specific order.
  For example, the following Kotlin code

  open class Temporary @JvmOverloads constructor(
    key: String,
    default: Boolean = false,
    team: Team = Team.UNKNOWN,
  ) : Flag(key = key, default = default, team = team)

  can be called from other Kotlin code via Temporary("key") or Temporary(default=true, key="key"). However, as Java does not support default or named parameters, this code has also been annotated with @JvmOverloads to auto generate the appropriate methods for Java callers

  Temporary(String key, boolean default, Team team)
  Temporary(String key, boolean default)   
  Temporary(String key)

• Object Types
  Kotlin has explicit support for singleton types, called objects. These are utilized for defining the feature-flags. When referenced within Kotlin code, they are referenced simply by their object name. However, as Java doesn't have this support natively, any objects referenced from Java code will need to use the object's auto-generated .INSTANCE; i.e. LogConnectorMessages.INSTANCE

  Note
  This will become more apparent in the related how to use this client PR.

• Extension Functions
  Kotlin supports extension functions which are utilized a few times in this PR. One adds a Path.onChange function that executes a lambda anytime that Java Path is modified (for watching the flags.yml file for changes). The other two add helper methods for converting the Context type to LaunchDarkly specific contexts. All three of these extension functions are defined with a private visibility and therefore can only be utilized by code within the Client file.

• thread Block
  Kotlin has a lot of builtin helpers for dealing with common Java patterns. One used in this PR is a thread block that creates a background thread for monitoring file changes.

[github-actions]

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage	26.59%	🍏