Module config as a map #3534
Module config as a map #3534
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3534 +/- ##
==========================================
- Coverage 81.10% 81.06% -0.04%
==========================================
Files 419 419
Lines 32296 32282 -14
==========================================
- Hits 26193 26171 -22
- Misses 6103 6111 +8
Continue to review full report at Codecov.
|
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
f968609
to
d0415c9
Compare
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
d0415c9
to
3e286cf
Compare
|
small_tests_24 / small_tests / 3e286cf small_tests_23 / small_tests / 3e286cf dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 3e286cf dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 3e286cf dynamic_domains_mysql_redis_24 / mysql_redis / 3e286cf ldap_mnesia_24 / ldap_mnesia / 3e286cf dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 3e286cf ldap_mnesia_23 / ldap_mnesia / 3e286cf internal_mnesia_24 / internal_mnesia / 3e286cf pgsql_mnesia_24 / pgsql_mnesia / 3e286cf pgsql_mnesia_23 / pgsql_mnesia / 3e286cf mssql_mnesia_24 / odbc_mssql_mnesia / 3e286cf elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 3e286cf mysql_redis_24 / mysql_redis / 3e286cf riak_mnesia_24 / riak_mnesia / 3e286cf |
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
3e286cf
to
20876d2
Compare
|
small_tests_24 / small_tests / 20876d2 small_tests_23 / small_tests / 20876d2 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 20876d2 dynamic_domains_mysql_redis_24 / mysql_redis / 20876d2 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 20876d2 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 20876d2 ldap_mnesia_23 / ldap_mnesia / 20876d2 ldap_mnesia_24 / ldap_mnesia / 20876d2 internal_mnesia_24 / internal_mnesia / 20876d2 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 20876d2 pgsql_mnesia_23 / pgsql_mnesia / 20876d2 pgsql_mnesia_24 / pgsql_mnesia / 20876d2 mysql_redis_24 / mysql_redis / 20876d2 mssql_mnesia_24 / odbc_mssql_mnesia / 20876d2 riak_mnesia_24 / riak_mnesia / 20876d2 |
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
20876d2
to
dcb75df
Compare
|
small_tests_24 / small_tests / dcb75df small_tests_23 / small_tests / dcb75df dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / dcb75df dynamic_domains_mysql_redis_24 / mysql_redis / dcb75df dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / dcb75df ldap_mnesia_23 / ldap_mnesia / dcb75df ldap_mnesia_24 / ldap_mnesia / dcb75df internal_mnesia_24 / internal_mnesia / dcb75df pgsql_mnesia_23 / pgsql_mnesia / dcb75df pgsql_mnesia_24 / pgsql_mnesia / dcb75df elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / dcb75df dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / dcb75df mssql_mnesia_24 / odbc_mssql_mnesia / dcb75df riak_mnesia_24 / riak_mnesia / dcb75df mysql_redis_24 / mysql_redis / dcb75df |
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
dcb75df
to
4bb0778
Compare
- Support key paths for easier nested map processing - Support old key-value list format as well for the transition period - Define types more strictly - Remove set_opt/3 to keep module options constant
Module options should remain constant. Any modification means creating a new data structure, so 'gen_mod' API sholdn't be used anymore for it.
This module should be used only when working with module options.
Instead, maintain the map as the new data structure. The main motivation is that functions are not covered by the mongoose_config:value() type.
Functions are not allowed as config values. This function would never be set in the config - it was impossible. In the future we can change this code to accept e.g. module names instead.
- Make validity_period a map (more straightforward and easy to set defaults)
Also: - Support dynamic domains - Simplify types and XML generation - Support iqdisc Note: attribute order is not specified now, as it is not significant in XML
- Add a helper for testing default values - Update tests for mod_auth_token and mod_extdisco
- Support dynamic domains
- iqdisc for 2 modules - updated option format
chrzaszcz
force-pushed
the
module-opts-as-map
branch
from
4bb0778
to
4fb9606
Compare
|
small_tests_24 / small_tests / 4fb9606 small_tests_23 / small_tests / 4fb9606 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 dynamic_domains_mysql_redis_24 / mysql_redis / 4fb9606 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 ldap_mnesia_23 / ldap_mnesia / 4fb9606 ldap_mnesia_24 / ldap_mnesia / 4fb9606 internal_mnesia_24 / internal_mnesia / 4fb9606 pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 mysql_redis_24 / mysql_redis / 4fb9606 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 4fb9606 pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 mam_SUITE:rdbms_async_pool_muc_light:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,
{{shutdown,
{failed_to_start_child,pm_mam_async_pool_localhost,
{shutdown,
{failed_to_start_child,
'wpool_pool-pm_mam_async_pool_localhost-process-sup',
{shutdown,
{failed_to_start_child,
'wpool_pool-pm_mam_async_pool_localhost-1',
{already_started,<8641.15204.1>}}}}}}},
{child,undefined,pm_mam_sup_async_pool_localhost,
{mongoose_async_pools,start_link,
[<<"localhost">>,pm_mam,
#{batch_name => insert_mam_messages30,batch_size => 30,
flush_callback => fun mod_mam_rdbms_arch_async:flush_pm/2,
flush_extra =>
#{batch_name => insert_mam_messages30,batch_size => 30,
flush_interval => 1,pool_size => 16},
flush_interval => 1,pool_size => 16}]},
transient,false,infinity,supervisor,
[mongoose_async_pools]}}},
#{id => pm_mam_sup_async_pool_localhost,restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,pm_mam,
#{batch_name => insert_mam_messages30,batch_size => 30,
flush_callback => fun mod_mam_rdbms_arch_async:flush_pm/2,
flush_extra =>
#{batch_name => insert_mam_messages30,batch_size => 30,
flush_interval => 1,pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},{line,183}]},
{mo...mam_SUITE:rdbms_async_pool_prefs_cases:messages_filtered_when_prefs_default_policy_is_roster{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok,ok,ok,ok,ok]\n"}}mam_SUITE:rdbms_async_pool_prefs_cases:messages_filtered_when_prefs_default_policy_is_always{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok,ok,ok,ok,ok,ok]\n"}}mam_SUITE:rdbms_async_pool_prefs_cases:messages_filtered_when_prefs_default_policy_is_never{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok,ok,ok]\n"}}mam_SUITE:rdbms_mnesia_muc_light:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,{already_started,<8641.15223.1>}},
#{id => muc_mam_sup_async_pool_localhost,
restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,muc_mam,
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,
flush_callback =>
fun mod_mam_rdbms_arch_async:flush_muc/2,
flush_extra =>
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,flush_interval => 1,
pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},
{line,183}]},
{mod_mam_rdbms_arch_async,'-start/2-lc$^0/1-0-',2,
[{file,
"/home/circleci/project/src/mam/mod_mam_rdbms_arch_async.erl"},
{line,48}]},
{gen_mod,start_module_for_host_type,3,
[{file,"/home/circleci/project/src/gen_mod.erl"},
{line,132}]},
{mongoose_modules,start_module,4,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,95}]},
{mongoose_modules,ensure_started,3,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,75}]},
{mongoo...mam_SUITE:rdbms_async_cache_muc_all:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,{already_started,<8641.15223.1>}},
#{id => muc_mam_sup_async_pool_localhost,
restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,muc_mam,
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,
flush_callback =>
fun mod_mam_rdbms_arch_async:flush_muc/2,
flush_extra =>
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,flush_interval => 1,
pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},
{line,183}]},
{mod_mam_rdbms_arch_async,'-start/2-lc$^0/1-0-',2,
[{file,
"/home/circleci/project/src/mam/mod_mam_rdbms_arch_async.erl"},
{line,48}]},
{gen_mod,start_module_for_host_type,3,
[{file,"/home/circleci/project/src/gen_mod.erl"},
{line,132}]},
{mongoose_modules,start_module,4,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,95}]},
{mongoose_modules,ensure_started,3,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,75}]},
{mongoo...mam_SUITE:rdbms_async_cache_muc_light:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,{already_started,<8641.15223.1>}},
#{id => muc_mam_sup_async_pool_localhost,
restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,muc_mam,
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,
flush_callback =>
fun mod_mam_rdbms_arch_async:flush_muc/2,
flush_extra =>
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,flush_interval => 1,
pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},
{line,183}]},
{mod_mam_rdbms_arch_async,'-start/2-lc$^0/1-0-',2,
[{file,
"/home/circleci/project/src/mam/mod_mam_rdbms_arch_async.erl"},
{line,48}]},
{gen_mod,start_module_for_host_type,3,
[{file,"/home/circleci/project/src/gen_mod.erl"},
{line,132}]},
{mongoose_modules,start_module,4,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,95}]},
{mongoose_modules,ensure_started,3,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,75}]},
{mongoo...mam_SUITE:rdbms_cache_muc_light:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,{already_started,<8641.15223.1>}},
#{id => muc_mam_sup_async_pool_localhost,
restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,muc_mam,
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,
flush_callback =>
fun mod_mam_rdbms_arch_async:flush_muc/2,
flush_extra =>
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,flush_interval => 1,
pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},
{line,183}]},
{mod_mam_rdbms_arch_async,'-start/2-lc$^0/1-0-',2,
[{file,
"/home/circleci/project/src/mam/mod_mam_rdbms_arch_async.erl"},
{line,48}]},
{gen_mod,start_module_for_host_type,3,
[{file,"/home/circleci/project/src/gen_mod.erl"},
{line,132}]},
{mongoose_modules,start_module,4,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,95}]},
{mongoose_modules,ensure_started,3,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,75}]},
{mongoo...mam_SUITE:rdbms_mnesia_cache_muc_light:init_per_group{'EXIT',
{{badrpc,
{'EXIT',
{{start_child_failed,
{error,{already_started,<8641.15223.1>}},
#{id => muc_mam_sup_async_pool_localhost,
restart => transient,
start =>
{mongoose_async_pools,start_link,
[<<"localhost">>,muc_mam,
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,
flush_callback =>
fun mod_mam_rdbms_arch_async:flush_muc/2,
flush_extra =>
#{batch_name => insert_mam_muc_messages30,
batch_size => 30,flush_interval => 1,
pool_size => 16},
flush_interval => 1,pool_size => 16}]},
type => supervisor}},
[{ejabberd_sup,start_child,1,
[{file,"/home/circleci/project/src/ejabberd_sup.erl"},
{line,183}]},
{mod_mam_rdbms_arch_async,'-start/2-lc$^0/1-0-',2,
[{file,
"/home/circleci/project/src/mam/mod_mam_rdbms_arch_async.erl"},
{line,48}]},
{gen_mod,start_module_for_host_type,3,
[{file,"/home/circleci/project/src/gen_mod.erl"},
{line,132}]},
{mongoose_modules,start_module,4,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,95}]},
{mongoose_modules,ensure_started,3,
[{file,"/home/circleci/project/src/mongoose_modules.erl"},
{line,75}]},
{mongoo...riak_mnesia_24 / riak_mnesia / 4fb9606 pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 riak_mnesia_24 / riak_mnesia / 4fb9606 small_tests_24 / small_tests / 4fb9606 small_tests_24 / small_tests / 4fb9606 small_tests_23 / small_tests / 4fb9606 small_tests_23 / small_tests / 4fb9606 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 dynamic_domains_mysql_redis_24 / mysql_redis / 4fb9606 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 dynamic_domains_mysql_redis_24 / mysql_redis / 4fb9606 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 ldap_mnesia_23 / ldap_mnesia / 4fb9606 ldap_mnesia_23 / ldap_mnesia / 4fb9606 ldap_mnesia_24 / ldap_mnesia / 4fb9606 ldap_mnesia_24 / ldap_mnesia / 4fb9606 internal_mnesia_24 / internal_mnesia / 4fb9606 internal_mnesia_24 / internal_mnesia / 4fb9606 pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 pgsql_mnesia_23 / pgsql_mnesia / 4fb9606 pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 pgsql_mnesia_24 / pgsql_mnesia / 4fb9606 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 4fb9606 mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 4fb9606 mysql_redis_24 / mysql_redis / 4fb9606 mysql_redis_24 / mysql_redis / 4fb9606 mssql_mnesia_24 / odbc_mssql_mnesia / 4fb9606 riak_mnesia_24 / riak_mnesia / 4fb9606 riak_mnesia_24 / riak_mnesia / 4fb9606 |
| {extdisco_configured, [sequence], extdisco_configured_tests()}, | ||
| {multiple_extdisco_configured, [sequence], multiple_extdisco_configured_tests()}, | ||
| {extdisco_required_elements_configured, [sequence], extdisco_required_elements_configured_tests()}], | ||
| ct_helper:repeat_all_until_all_ok(G). |
There was a problem hiding this comment.
Nice. 👍 I know we have some flaky tests, but for me we could just remove it from all suites.
There was a problem hiding this comment.
For extdisco and auth_token there should be no flaky tests already, so I removed the repeat.
|
small_tests_24 / small_tests / 503b249 small_tests_23 / small_tests / 503b249 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 503b249 dynamic_domains_mysql_redis_24 / mysql_redis / 503b249 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 503b249 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 503b249 ldap_mnesia_24 / ldap_mnesia / 503b249 ldap_mnesia_23 / ldap_mnesia / 503b249 internal_mnesia_24 / internal_mnesia / 503b249 pgsql_mnesia_24 / pgsql_mnesia / 503b249 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 503b249 mysql_redis_24 / mysql_redis / 503b249 pgsql_mnesia_23 / pgsql_mnesia / 503b249 mssql_mnesia_24 / odbc_mssql_mnesia / 503b249 riak_mnesia_24 / riak_mnesia / 503b249 |
Start the conversion of module options from proplists to maps. Maps are more efficient, easier to work with, and better integrated with
mongoose_config.Changes in
gen_mod:funin TOML, so I think it's best to avoid this.merger_funinmongoose_user_cacheis hardcoded to bemaps:merge/2as it was impossible to set this option anyway.set_optto discourage modifying options at runtime - if we need to do this with maps, the built-in map syntax is better suited for this. The result should be a new type.gen_modAPI not related to modules, e.g. general proplist processing, but only wheredialyzercomplained.Changes in modules:
mod_auth_tokenandmod_extdisco, to use maps.mod_extdiscosupports dynamic domains now - it was easy and the code was already modified a lot.gen_mod:get_opt/2. I am not completely sure if this is always the right way, but it makes sense to leverage the language constructs instead of calling getter functions. However, the getters for nested maps might be useful, so I think that we need to find out what works best as we progress with the remaining modules.