Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Config v6 #133

Merged
merged 106 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from 82 commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
3c75446
♻️ Encapsulate config format (#114)
randycoulman Jan 9, 2024
0142a88
♻️ Return entire config (#117)
randycoulman Jan 9, 2024
5d0ce0a
✨ Update EvaluationFormula for v6
randycoulman Dec 16, 2023
f42459b
🚧 Start converting to config v6
randycoulman Dec 18, 2023
c0bad61
✅ Simplify default user tests
randycoulman Dec 18, 2023
5be4f2f
🚧 Update config modules for new format
randycoulman Dec 18, 2023
02b0bdc
🚧 Evaluate new targeting rule format
randycoulman Dec 18, 2023
51d7c48
🚧 Extract typed values from percentage options
randycoulman Dec 18, 2023
eca5a9c
🚧 Use comparison value of proper type
randycoulman Dec 18, 2023
b10feda
🚧 Implement segment evaluation
randycoulman Dec 19, 2023
dc1c991
✨ Extract segments from config
randycoulman Dec 20, 2023
315f6cc
✅ Handle string lists in comparators
randycoulman Dec 20, 2023
2194568
✨ Use salt for hashing sensitive values
randycoulman Dec 20, 2023
6500b05
♻️ Refactor EvaluationFormula.variation_value
randycoulman Dec 20, 2023
26eebb4
♻️ Merge Rollout.Comparator into Config.UserComparator
randycoulman Dec 20, 2023
1b41dcb
♻️ Delete to ValueAndVariationId where possible
randycoulman Dec 20, 2023
18d44e5
♻️ Use Context struct to hold evaluation context
randycoulman Dec 20, 2023
d7b1f4f
✅ Update matrix tests to v2 format
randycoulman Dec 21, 2023
03441a4
♻️ Default to value_type_test
randycoulman Dec 21, 2023
5d901e9
🧪 Add (and skip) new matrix tests
randycoulman Dec 21, 2023
532ec01
♻️ Rename comparators and descriptions
randycoulman Dec 21, 2023
bfc2118
♻️ Rename EvaluationDetails fields
randycoulman Dec 21, 2023
fe18c62
✨ Support before and after comparators
randycoulman Dec 22, 2023
24b9a54
✨ Support equals/not_equals hashed comparators
randycoulman Dec 22, 2023
e379b01
✨ Implement starts/ends with any of hashed comparators
randycoulman Dec 22, 2023
2e49129
🐛 Fix (not_)contains_any_of to work with string lists
randycoulman Dec 22, 2023
e552a1f
✨ Implement array (not) contains any of comparators
randycoulman Dec 22, 2023
20d59d9
✨ Implement equals/not equals comparators
randycoulman Dec 22, 2023
d07ec74
✨ Implement (not) starts/ends with comparators
randycoulman Dec 22, 2023
41ab628
✨ Implement array (not) contains any of comparators
randycoulman Dec 22, 2023
4c634d2
💡 Update skip reasons for matrix tests
randycoulman Dec 22, 2023
b12ff37
✨ Update comment/SDK key for segments_old test to v2
randycoulman Dec 22, 2023
ac33cd8
♻️ Push nil user checks down
randycoulman Dec 22, 2023
e3d1ff9
♻️ Update type conversion code
randycoulman Dec 22, 2023
116b495
✨ Support NaiveDateTime by converting to UTC
randycoulman Dec 22, 2023
280140f
✅ Parse expected values in matrix tests
randycoulman Dec 23, 2023
9aa566c
🐛 Fix handling of Unicode strings
randycoulman Dec 23, 2023
f59406e
✅ Add attribute value conversion tests
randycoulman Dec 23, 2023
8c1f53a
✨ Support nested percentage options
randycoulman Dec 23, 2023
3bf57ac
✨ Support custom percentage attribute
randycoulman Dec 23, 2023
62a802a
🐛 Return immediately on condition evaluation error
randycoulman Dec 23, 2023
8a1f213
♻️ Simplify percentage option evaluation
randycoulman Dec 23, 2023
412fc16
✨ Support prerequisite flag conditions
randycoulman Dec 24, 2023
48768d8
🐛 Move evaluation log up to caller
randycoulman Dec 24, 2023
6c0d0e3
🐛 Detect circular prerequisite flag dependencies
randycoulman Dec 24, 2023
4d0b63a
✨ Check for type mismatches
randycoulman Dec 24, 2023
c617d43
♻️ Rename config modules
randycoulman Dec 27, 2023
97af4a4
✨ Add inline salt/segment fields
randycoulman Dec 27, 2023
9057bb5
✨ Inline salt and segments after loading configs
randycoulman Dec 27, 2023
6325daa
✨ Use inline salt and segments
randycoulman Dec 27, 2023
afa466f
✅ Add override tests for prerequisite flags
randycoulman Dec 27, 2023
b196876
✅ Add override tests for salt/segments
randycoulman Dec 27, 2023
42cb18f
♻️ Extract a base test case template for common functionality
randycoulman Dec 27, 2023
25fcab4
🦺 Validate SDK key format
randycoulman Dec 27, 2023
1979e35
📝 Add more documentation
randycoulman Dec 27, 2023
17e8b70
🧪 Add tests involving evaluation log entries
randycoulman Dec 28, 2023
9cad06b
✨ Begin updating evaluation logging
randycoulman Dec 28, 2023
7f30945
✨ Implement String.Chars for User
randycoulman Dec 28, 2023
66ea953
✨ Continue adding evaluation logging
randycoulman Dec 28, 2023
2c8bc69
✨ Logging for prerequiste flag condition evaluation
randycoulman Dec 28, 2023
229c9be
✨ Add logging for segment condition evaluation
randycoulman Dec 28, 2023
2cee94a
✨ Log user condition evaluation
randycoulman Dec 29, 2023
2b447d1
✅ Only log final result at root
randycoulman Dec 29, 2023
a647844
✨ Format comparison values consistently
randycoulman Dec 29, 2023
6c1b606
✨ Log when converting non-string comparison value to string
randycoulman Dec 29, 2023
07bccd3
✨ Log user value type mismatches
randycoulman Dec 29, 2023
b9802d6
✨ Only log missing/invalid user warnings once
randycoulman Dec 29, 2023
0df5db6
✅ Fix minor spacing issues in logs
randycoulman Dec 29, 2023
27c3eb2
♻️ Use default_variation_id from context
randycoulman Dec 29, 2023
041df8c
✅ Allow for unspecified user serialization order
randycoulman Dec 29, 2023
6af2847
✅ Raise errors on missing values
randycoulman Dec 29, 2023
556c76a
✅ Fix last remaining skipped test
randycoulman Dec 29, 2023
7089e6c
✨ Warn when default value doesn't match setting's type
randycoulman Dec 29, 2023
ef0a042
✅ Run more tests synchronously
randycoulman Dec 30, 2023
50f8ba2
✅ Use capture_log instead of with_log
randycoulman Jan 10, 2024
789ada2
🔧 Use older Logger config in Elixir < 1.15
randycoulman Jan 10, 2024
ac3a332
✅ Use proper log level for different Elixir versions
randycoulman Jan 10, 2024
104b655
🐛 Replace all instances of warning
randycoulman Jan 10, 2024
db4476d
🐛 Use warning in 1.13 and 1.14
randycoulman Jan 10, 2024
0af559f
📝 Minor doc fix
randycoulman Jan 11, 2024
af5c6bc
🔥 Remove commented code
randycoulman Jan 12, 2024
c550cec
✅ Add test to show that cache time is respected
randycoulman Jan 12, 2024
b10323d
🐛 Treat empty string as missing attribute value
randycoulman Jan 28, 2024
c22c00c
🔊 Change wording of type mismatch errors
randycoulman Jan 28, 2024
035cacb
🐛 Trim strings before converting to floats
randycoulman Jan 28, 2024
122bce1
🐛 Convert user attribute values to strings before hashing
randycoulman Jan 28, 2024
5da8c65
✨ Look at nested percentage options for get_key_and_value
randycoulman Jan 28, 2024
5c9b4f7
✨ Warn if setting value is of an unsupported type
randycoulman Jan 28, 2024
4130cee
🔊 Use ConfigCat types instead of Elixir types
randycoulman Jan 28, 2024
4f3ffd8
✨ Check for string contents of JSON list
randycoulman Jan 28, 2024
7e8e2f2
✅ Add config v1 matrix tests
randycoulman Jan 28, 2024
bfa583e
✅ Add more SDK key validation tests
randycoulman Jan 28, 2024
4ca885b
🔊 Add more details to 1103 error message
randycoulman Jan 28, 2024
d07c796
✅ Add special characters tests
randycoulman Jan 28, 2024
057a503
🏷️ Fix typespec on user_value_to_string
randycoulman Jan 28, 2024
cc9361b
✅ Fix case-sensitivity issue in matrix filename
randycoulman Jan 28, 2024
1005612
⬆️ Update deps in sample apps
randycoulman Jan 30, 2024
767ec96
✅ Add new cache key tests
randycoulman Jan 30, 2024
4c5acac
✅ Add tests for value trimming
randycoulman Jan 30, 2024
00b78c3
🚧 WIP: add new log tests
randycoulman Jan 30, 2024
7ed2557
🐛 Fix conversion of string list -> string
randycoulman Jan 30, 2024
00289e5
✨ Ensure list contains all strings
randycoulman Jan 30, 2024
b1194b3
✅ Add (and pass) new string conversion tests
randycoulman Jan 31, 2024
43d9d89
🥅 Raise an error when a setting value doesn't match its key
randycoulman Jan 31, 2024
c37fe0b
♻️ Minor cleanups to logging code
randycoulman Feb 1, 2024
fee604b
✅ Fix remaining log test
randycoulman Feb 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .credo.exs
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@
{Credo.Check.Refactor.IoPuts, []},
{Credo.Check.Refactor.LongQuoteBlocks, []},
{Credo.Check.Refactor.MatchInCondition, []},
{Credo.Check.Refactor.Nesting, []},
{Credo.Check.Refactor.Nesting, [max_nesting: 3]},
{Credo.Check.Refactor.PassAsyncInTestCases, []},
{Credo.Check.Refactor.FilterFilter, []},
{Credo.Check.Refactor.RejectReject, []},
Expand Down
19 changes: 19 additions & 0 deletions config/config.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import Config

if config_env() == :dev do
config :mix_test_interactive, clear: true
end

if config_env() == :test do
config :logger, level: :warning

if Version.compare(System.version(), "1.15.0") == :lt do
config :logger, :console,
colors: [enabled: false],
format: "$level $message\n"
else
config :logger, :default_formatter,
colors: [enabled: false],
format: "$level $message\n"
end
end
6 changes: 3 additions & 3 deletions lib/config_cat/cache_policy/auto.ex
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ defmodule ConfigCat.CachePolicy.Auto do

@impl GenServer
def handle_call(:get, _from, %State{} = state) do
{:reply, Helpers.cached_settings(state), state}
{:reply, Helpers.cached_config(state), state}
end

@impl GenServer
Expand Down Expand Up @@ -204,10 +204,10 @@ defmodule ConfigCat.CachePolicy.Auto do
defp be_initialized(%State{} = state) when initialized?(state), do: state

defp be_initialized(%State{} = state) do
settings = Helpers.cached_settings(state)
config = Helpers.cached_config(state)

for caller <- state.policy_state.callers do
GenServer.reply(caller, settings)
GenServer.reply(caller, config)
end

Helpers.on_client_ready(state)
Expand Down
2 changes: 1 addition & 1 deletion lib/config_cat/cache_policy/behaviour.ex
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ defmodule ConfigCat.CachePolicy.Behaviour do
alias ConfigCat.FetchTime

@callback get(ConfigCat.instance_id()) ::
{:ok, Config.settings(), FetchTime.t()} | {:error, :not_found}
{:ok, Config.t(), FetchTime.t()} | {:error, :not_found}
@callback offline?(ConfigCat.instance_id()) :: boolean()
@callback set_offline(ConfigCat.instance_id()) :: :ok
@callback set_online(ConfigCat.instance_id()) :: :ok
Expand Down
21 changes: 3 additions & 18 deletions lib/config_cat/cache_policy/helpers.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ defmodule ConfigCat.CachePolicy.Helpers do
alias ConfigCat.Cache
alias ConfigCat.CachePolicy
alias ConfigCat.Config
alias ConfigCat.ConfigCache
alias ConfigCat.ConfigEntry
alias ConfigCat.ConfigFetcher.FetchError
alias ConfigCat.FetchTime
Expand Down Expand Up @@ -66,25 +65,11 @@ defmodule ConfigCat.CachePolicy.Helpers do
Hooks.invoke_on_client_ready(state.instance_id)
end

@spec cached_settings(State.t()) ::
{:ok, Config.settings(), FetchTime.t()} | {:error, :not_found}
def cached_settings(%State{} = state) do
with {:ok, %ConfigEntry{} = entry} <- cached_entry(state),
{:ok, settings} <- Config.fetch_settings(entry.config) do
{:ok, settings, entry.fetch_time_ms}
else
:error ->
{:error, :not_found}

error ->
error
end
end

@spec cached_config(State.t()) :: ConfigCache.result()
@spec cached_config(State.t()) ::
{:ok, Config.t(), FetchTime.t()} | {:error, :not_found}
def cached_config(%State{} = state) do
with {:ok, %ConfigEntry{} = entry} <- cached_entry(state) do
{:ok, entry.config}
{:ok, entry.config, entry.fetch_time_ms}
end
end

Expand Down
2 changes: 1 addition & 1 deletion lib/config_cat/cache_policy/lazy.ex
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ defmodule ConfigCat.CachePolicy.Lazy do
@impl GenServer
def handle_call(:get, _from, %State{} = state) do
with {:ok, new_state} <- maybe_refresh(state) do
{:reply, Helpers.cached_settings(new_state), new_state}
{:reply, Helpers.cached_config(new_state), new_state}
end
end

Expand Down
2 changes: 1 addition & 1 deletion lib/config_cat/cache_policy/manual.ex
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ defmodule ConfigCat.CachePolicy.Manual do

@impl GenServer
def handle_call(:get, _from, %State{} = state) do
{:reply, Helpers.cached_settings(state), state}
{:reply, Helpers.cached_config(state), state}
end

@impl GenServer
Expand Down
95 changes: 65 additions & 30 deletions lib/config_cat/client.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,18 @@ defmodule ConfigCat.Client do
use GenServer

alias ConfigCat.CachePolicy
alias ConfigCat.Config
alias ConfigCat.Config.Setting
alias ConfigCat.Config.SettingType
alias ConfigCat.EvaluationDetails
alias ConfigCat.EvaluationLogger
alias ConfigCat.FetchTime
alias ConfigCat.Hooks
alias ConfigCat.OverrideDataSource
alias ConfigCat.Rollout
alias ConfigCat.User

require ConfigCat.ConfigCatLogger, as: ConfigCatLogger
require ConfigCat.Constants, as: Constants

defmodule State do
@moduledoc false
Expand Down Expand Up @@ -96,9 +99,12 @@ defmodule ConfigCat.Client do

@impl GenServer
def handle_call({:get_key_and_value, variation_id}, _from, %State{} = state) do
case cached_settings(state) do
{:ok, settings, _fetch_time_ms} ->
result = Enum.find_value(settings, nil, &entry_matching(&1, variation_id))
case cached_config(state) do
{:ok, config, _fetch_time_ms} ->
result =
config
|> Config.settings()
|> Enum.find_value(nil, &entry_matching(&1, variation_id))

if is_nil(result) do
ConfigCatLogger.error(
Expand Down Expand Up @@ -183,9 +189,9 @@ defmodule ConfigCat.Client do
end

defp do_get_all_keys(%State{} = state) do
case cached_settings(state) do
{:ok, settings, _fetch_time_ms} ->
Map.keys(settings)
case cached_config(state) do
{:ok, config, _fetch_time_ms} ->
config |> Config.settings() |> Map.keys()

_ ->
ConfigCatLogger.error("Config JSON is not present. Returning empty result.",
Expand All @@ -197,29 +203,26 @@ defmodule ConfigCat.Client do
end

defp entry_matching({key, setting}, variation_id) do
value_matching(key, setting, variation_id) ||
value_matching(key, Map.get(setting, Constants.rollout_rules()), variation_id) ||
value_matching(key, Map.get(setting, Constants.percentage_rules()), variation_id)
end

defp value_matching(key, value, variation_id) when is_list(value) do
Enum.find_value(value, nil, &value_matching(key, &1, variation_id))
end

defp value_matching(key, value, variation_id) do
if Map.get(value, Constants.variation_id(), nil) == variation_id do
{key, Map.get(value, Constants.value())}
case Setting.variation_value(setting, variation_id) do
nil -> nil
value -> {key, value}
randycoulman marked this conversation as resolved.
Show resolved Hide resolved
end
end

defp evaluate(key, user, default_value, default_variation_id, %State{} = state) do
user = if user != nil, do: user, else: state.default_user

details =
case cached_settings(state) do
{:ok, settings, fetch_time_ms} ->
%EvaluationDetails{} =
details =
with {:ok, config, fetch_time_ms} <- cached_config(state),
{:ok, _settings} <- Config.fetch_settings(config),
{:ok, logger} <- EvaluationLogger.start() do
try do
%EvaluationDetails{} =
details = Rollout.evaluate(key, user, default_value, default_variation_id, settings)
details =
Rollout.evaluate(key, user, default_value, default_variation_id, config, logger)

check_type_mismatch(details.value, default_value)

fetch_time =
case FetchTime.to_datetime(fetch_time_ms) do
Expand All @@ -228,7 +231,14 @@ defmodule ConfigCat.Client do
end

%{details | fetch_time: fetch_time}
after
logger
|> EvaluationLogger.result()
|> ConfigCatLogger.debug(event_id: 5000)

EvaluationLogger.stop(logger)
end
else
_ ->
message =
"Config JSON is not present when evaluating setting '#{key}'. Returning the `default_value` parameter that you specified in your application: '#{default_value}'."
Expand All @@ -249,23 +259,48 @@ defmodule ConfigCat.Client do
details
end

defp cached_settings(%State{} = state) do
defp cached_config(%State{} = state) do
%{cache_policy: policy, flag_overrides: flag_overrides, instance_id: instance_id} = state
local_settings = OverrideDataSource.overrides(flag_overrides)
local_config = OverrideDataSource.overrides(flag_overrides)

case OverrideDataSource.behaviour(flag_overrides) do
:local_only ->
{:ok, local_settings, 0}
{:ok, local_config, 0}

:local_over_remote ->
with {:ok, remote_settings, fetch_time_ms} <- policy.get(instance_id) do
{:ok, Map.merge(remote_settings, local_settings), fetch_time_ms}
with {:ok, remote_config, fetch_time_ms} <- policy.get(instance_id) do
{:ok, Config.merge(remote_config, local_config), fetch_time_ms}
end

:remote_over_local ->
with {:ok, remote_settings, fetch_time_ms} <- policy.get(instance_id) do
{:ok, Map.merge(local_settings, remote_settings), fetch_time_ms}
with {:ok, remote_config, fetch_time_ms} <- policy.get(instance_id) do
merged = Config.merge(local_config, remote_config)
{:ok, merged, fetch_time_ms}
end
end
end

defp check_type_mismatch(_value, nil), do: :ok

defp check_type_mismatch(value, default_value) do
value_type = SettingType.infer_elixir_type(value)
default_type = SettingType.infer_elixir_type(default_value)
number_types = ["float()", "integer()"]

cond do
value_type == default_type ->
:ok

value_type in number_types and default_type in number_types ->
:ok

true ->
ConfigCatLogger.warning(
"The type of a setting does not match the type of the specified default value (#{default_value}). " <>
"Setting's type was #{value_type} but the default value's type was #{default_type}. " <>
randycoulman marked this conversation as resolved.
Show resolved Hide resolved
"Please make sure that using a default value not matching the setting's type was intended.",
event_id: 4002
)
end
end
end
81 changes: 56 additions & 25 deletions lib/config_cat/config.ex
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,27 @@ defmodule ConfigCat.Config do
@moduledoc """
Defines configuration-related types used in the rest of the library.
"""
alias ConfigCat.RedirectMode
alias ConfigCat.Config.Preferences
alias ConfigCat.Config.Segment
alias ConfigCat.Config.Setting

@typedoc false
@type comparator :: non_neg_integer()

@typedoc "The name of a configuration setting."
@type key :: String.t()

@typedoc "The configuration settings within a Config."
@type settings :: map()
@typedoc false
@type opt :: {:preferences, Preferences.t()} | {:settings, settings()}

@typedoc false
@type salt :: String.t()

@typedoc false
@type settings :: %{String.t() => Setting.t()}

@typedoc "A collection of configuration settings and preferences."
@type t :: map()
@type t :: %{String.t() => map()}

@typedoc false
@type url :: String.t()
Expand All @@ -25,43 +33,66 @@ defmodule ConfigCat.Config do
@typedoc "The name of a variation being tested."
@type variation_id :: String.t()

@feature_flags "f"
@settings "f"
@preferences "p"
@preferences_base_url "u"
@redirect_mode "r"
@segments "s"

@doc false
@spec new([opt]) :: t()
def new(opts \\ []) do
settings = Keyword.get(opts, :settings, %{})
preferences = Keyword.get_lazy(opts, :preferences, &Preferences.new/0)

%{@settings => settings, @preferences => preferences}
end

@doc false
@spec preferences(t()) :: Preferences.t()
def preferences(config) do
Map.get_lazy(config, @preferences, &Preferences.new/0)
end

@doc false
@spec new_with_preferences(url(), RedirectMode.t()) :: t()
def new_with_preferences(base_url, redirect_mode) do
%{
@preferences => %{
@preferences_base_url => base_url,
@redirect_mode => redirect_mode
}
}
@spec segments(t()) :: [Segment.t()]
def segments(config) do
Map.get(config, @segments, [])
end

@doc false
@spec new_with_settings(settings()) :: t()
def new_with_settings(settings) do
%{@feature_flags => settings}
@spec settings(t()) :: settings()
def settings(config) do
Map.get(config, @settings, %{})
end

@doc false
@spec fetch_settings(t()) :: {:ok, settings()} | {:error, :not_found}
def fetch_settings(config) do
case Map.fetch(config, @feature_flags) do
case Map.fetch(config, @settings) do
{:ok, settings} -> {:ok, settings}
:error -> {:error, :not_found}
end
end

@doc false
@spec preferences(t()) :: {url() | nil, RedirectMode.t() | nil}
def preferences(config) do
case config[@preferences] do
nil -> {nil, nil}
preferences -> {preferences[@preferences_base_url], preferences[@redirect_mode]}
end
@spec merge(left :: t(), right :: t()) :: t()
def merge(left, right) do
left_flags = settings(left)
right_flags = settings(right)

Map.put(left, @settings, Map.merge(left_flags, right_flags))
end

@doc false
@spec inline_salt_and_segments(t()) :: t()
def inline_salt_and_segments(config) do
salt = config |> preferences() |> Preferences.salt()
segments = segments(config)

Map.update(
config,
@settings,
%{},
&Map.new(&1, fn {key, setting} -> {key, Setting.inline_salt_and_segments(setting, salt, segments)} end)
)
end
end
Loading
Loading