Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unfold global options without the temporary functions #3406

Merged
merged 4 commits into from
Nov 18, 2021

Conversation

chrzaszcz
Copy link
Member

@chrzaszcz chrzaszcz commented Nov 18, 2021

The goal of this PR is to simplify the config processing so that no temporary functions are inserted into the option list.

This makes it easier to rework the internal structure (e.g. using maps) and to maintain and understand the unit tests.
It also simplifies the subsequent unit test rework needed to support config defaults.

Another advantage of the new unit test macros is that we could rework the config not to unfold the global options and most of the unit tests would stay the same.

These functions made it difficult to rework the resulting
internal config structure and test the expected parsed config.

Now the unfolding is explicit.
In the future we might want to get rid of unfolding
and just use {Key, global}.
However, this might mean one extra lookup for each option.
This is needed by the config parser before key unfolding.
I decided not to complicate types by introducing an extra type here
for the unfolded keys.
@chrzaszcz chrzaszcz force-pushed the config-unfold-global branch 2 times, most recently from 03ecf5a to 9610799 Compare November 18, 2021 09:03
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 18, 2021

small_tests_24 / small_tests / b2f3079
Reports root / small


internal_mnesia_24 / internal_mnesia / b2f3079
Reports root/ big
OK: 1587 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 18, 2021

small_tests_24 / small_tests / 03ecf5a
Reports root / small


internal_mnesia_24 / internal_mnesia / 03ecf5a
Reports root/ big
OK: 1587 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0

@codecov
Copy link

codecov bot commented Nov 18, 2021

Codecov Report

Merging #3406 (cd88a63) into master (90d6e47) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3406      +/-   ##
==========================================
- Coverage   80.96%   80.93%   -0.04%     
==========================================
  Files         414      414              
  Lines       32300    32301       +1     
==========================================
- Hits        26153    26142      -11     
- Misses       6147     6159      +12     
Impacted Files Coverage Δ
src/config/mongoose_config.erl 96.87% <ø> (ø)
src/config/mongoose_config_parser_toml.erl 98.48% <100.00%> (+0.01%) ⬆️
src/mod_roster_riak.erl 81.53% <0.00%> (-15.39%) ⬇️
src/mod_last_rdbms.erl 96.15% <0.00%> (-3.85%) ⬇️
...bal_distrib/mod_global_distrib_hosts_refresher.erl 71.69% <0.00%> (-3.78%) ⬇️
...c/global_distrib/mod_global_distrib_server_mgr.erl 75.14% <0.00%> (-2.26%) ⬇️
src/mod_last.erl 86.56% <0.00%> (-1.50%) ⬇️
src/mam/mod_mam_utils.erl 86.53% <0.00%> (-0.33%) ⬇️
src/ejabberd_s2s_out.erl 63.08% <0.00%> (-0.24%) ⬇️
src/mod_muc_room.erl 77.14% <0.00%> (-0.06%) ⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90d6e47...cd88a63. Read the comment docs.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 18, 2021

small_tests_24 / small_tests / 9610799
Reports root / small


internal_mnesia_24 / internal_mnesia / 9610799
Reports root/ big
OK: 1587 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0


small_tests_23 / small_tests / 9610799
Reports root / small


dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 9610799
Reports root/ big
OK: 2722 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 9610799
Reports root/ big
OK: 2722 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0


dynamic_domains_mysql_redis_24 / mysql_redis / 9610799
Reports root/ big
OK: 2705 / Failed: 0 / User-skipped: 203 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 9610799
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 370 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 9610799
Reports root/ big
OK: 1521 / Failed: 2 / User-skipped: 370 / Auto-skipped: 0

connect_SUITE:just_tls:feature_order:auth_bind_compression_session
{error,
  {{badmatch,
     {error,
       {connection_step_failed,
         {{escalus_session,session},
          {client,
            <<"secure_joe_auth_bind_compression_session_91.85893@localhost/escalus-default-resource">>,
            escalus_tcp,<0.12124.0>,undefined,
            [{username,
               <<"secure_joe_auth_bind_compression_session_91.85893">>},
             {server,<<"localhost">>},
             {password,<<"break_me">>},
             {compression,<<"zlib">>},
             {starttls,required},
             {host,<<"localhost">>},
             {stream_id,<<"305a63e1bc541c53">>},
             {resource,<<"escalus-default-resource">>}]},
          [{compression,false},
           {starttls,false},
           {stream_management,false},
           {advanced_message_processing,false},
           {client_state_indication,false},
           {sasl_mechanisms,[]},
           {caps,undefined}]},
         {timeout,session_reply}}}},
   [{connect_SUITE,auth_bind_compression_session,1,
      [{file,"/home/circleci/app/big_tests/tests/connect_SUITE.erl"},
       {line,519}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1784}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1293}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1225}]}]}}

Report log

connect_SUITE:just_tls:feature_order:auth_compression_bind_session
{error,
  {{badmatch,
     {error,
       {connection_step_failed,
         {{escalus_session,maybe_use_compression},
          {client,
            <<"secure_joe_auth_compression_bind_session_91.91955@localhost">>,
            escalus_tcp,<0.12125.0>,undefined,
            [{username,
               <<"secure_joe_auth_compression_bind_session_91.91955">>},
             {server,<<"localhost">>},
             {password,<<"break_me">>},
             {compression,<<"zlib">>},
             {starttls,required},
             {host,<<"localhost">>},
             {stream_id,<<"f8c0dc1c512c2058">>}]},
          [{compression,[<<"zlib">>]},
           {starttls,false},
           {stream_management,true},
           {advanced_message_processing,true},
           {client_state_indication,false},
           {sasl_mechanisms,[<<"PLAIN">>]},
           {caps,undefined}]},
         {timeout,wait_for_features}}}},
   [{connect_SUITE,auth_compression_bind_session,1,
      [{file,"/home/circleci/app/big_tests/tests/connect_SUITE.erl"},
       {line,509}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1784}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1293}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1225}]}]}}

Report log


dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 9610799
Reports root/ big
OK: 2729 / Failed: 1 / User-skipped: 186 / Auto-skipped: 0

sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_stop_c2s
{error,{thrown,{timeout,msg}}}

Report log


mysql_redis_24 / mysql_redis / 9610799
Reports root/ big
OK: 3104 / Failed: 0 / User-skipped: 200 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 9610799
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 9610799
Reports root/ big
OK: 1892 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 9610799
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 9610799
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 9610799
Reports root/ big
OK: 1738 / Failed: 0 / User-skipped: 298 / Auto-skipped: 0

@chrzaszcz chrzaszcz marked this pull request as ready for review November 18, 2021 10:02
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 18, 2021

small_tests_24 / small_tests / cd88a63
Reports root / small


internal_mnesia_24 / internal_mnesia / cd88a63
Reports root/ big
OK: 1587 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0


small_tests_23 / small_tests / cd88a63
Reports root / small


dynamic_domains_mysql_redis_24 / mysql_redis / cd88a63
Reports root/ big
OK: 2705 / Failed: 0 / User-skipped: 203 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / cd88a63
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 370 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / cd88a63
Reports root/ big
OK: 2722 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / cd88a63
Reports root/ big
OK: 2722 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / cd88a63
Reports root/ big
OK: 2722 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / cd88a63
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 370 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / cd88a63
Reports root/ big
OK: 1892 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / cd88a63
Reports root/ big
OK: 3135 / Failed: 2 / User-skipped: 183 / Auto-skipped: 0

sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_send_message
{error,
  {{assertion_failed,assert_many,true,
     [#Fun<sm_SUITE.22.149602>,#Fun<sm_SUITE.22.149602>,
      #Fun<sm_SUITE.22.149602>],
     [{xmlel,<<"presence">>,
        [{<<"from">>,
        <<"alicE_resume_session_state_send_message_97.965476@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alice_resume_session_state_send_message_97.965476@localhost/escalus-default-resource">>},
         {<<"xml:lang">>,<<"en">>}],
        []},
      {xmlel,<<"message">>,
        [{<<"from">>,
        <<"bOb_resume_session_state_send_message_97.892774@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alicE_resume_session_state_send_message_97.965476@localhost">>},
         {<<"xml:lang">>,<<"en">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"msg-1">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
          {<<"stamp">>,<<"2021-11-18T10:33:18.042786Z">>},
          {<<"from">>,<<"localhost">>}],
           [{xmlcdata,<<"SM Storage">>}]}]},
      {xmlel,<<"message">>,
        [{<<"from">>,
        <<"bOb_resume_session_state_send_message_97.892774@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alicE_resume_session_state_send_message_97.965476@localhost">>},
         {<<"xml:lang">>,<<"en">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"msg-2">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay"...

Report log

sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_send_message
{error,
  {{assertion_failed,assert_many,true,
     [#Fun<sm_SUITE.22.149602>,#Fun<sm_SUITE.22.149602>,
      #Fun<sm_SUITE.22.149602>],
     [{xmlel,<<"presence">>,
        [{<<"from">>,
        <<"alicE_resume_session_state_send_message_4.392783@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alice_resume_session_state_send_message_4.392783@localhost/escalus-default-resource">>},
         {<<"xml:lang">>,<<"en">>}],
        []},
      {xmlel,<<"message">>,
        [{<<"from">>,
        <<"bOb_resume_session_state_send_message_4.288227@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alicE_resume_session_state_send_message_4.392783@localhost">>},
         {<<"xml:lang">>,<<"en">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"msg-1">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
          {<<"stamp">>,<<"2021-11-18T10:33:24.484866Z">>},
          {<<"from">>,<<"localhost">>}],
           [{xmlcdata,<<"SM Storage">>}]}]},
      {xmlel,<<"message">>,
        [{<<"from">>,
        <<"bOb_resume_session_state_send_message_4.288227@localhost/escalus-default-resource">>},
         {<<"to">>,
        <<"alicE_resume_session_state_send_message_4.392783@localhost">>},
         {<<"xml:lang">>,<<"en">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"msg-2">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
 ...

Report log


mssql_mnesia_24 / odbc_mssql_mnesia / cd88a63
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / cd88a63
Reports root/ big
OK: 3109 / Failed: 3 / User-skipped: 200 / Auto-skipped: 0

mam_SUITE:rdbms_prefs_cases:prefs_set_cdata_request
{error,{test_case_failed,"ASSERT EQUAL\n\tExpected {prefs_result_iq,<<\"roster\">>,\n                  [<<\"[email protected]\">>,\n                   <<\"[email protected]\">>],\n                  []}\n\tValue {prefs_result_iq,<<\"always\">>,\n                 [<<\"[email protected]\">>,\n                <<\"[email protected]\">>],\n                 []}\n"}}

Report log

s2s_SUITE:both_tls_required:simple_message
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,<<"alicE_simple_message_59.672260@localhost/res1">>,
          escalus_tcp,<0.8988.2>,
          [{event_manager,<0.8983.2>},
           {server,<<"localhost">>},
           {username,<<"alicE_simple_message_59.672260">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.8983.2>},
            {server,<<"localhost">>},
            {username,<<"alicE_simple_message_59.672260">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"alicE_simple_message_59.672260">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"alicE_simple_message_59.672260">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"103fa4947bf49f51">>}]},
        10000],
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {s2s_SUITE,'-simple_message/1-fun-0-',2,
       [{file,"/home/circleci/app/big_tests/tests/s2s_SUITE.erl"},
        {line,131}]},
     {escalus_story,story,4,
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
        {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"}...

Report log

service_domain_db_SUITE:db:db_keeps_syncing_after_cluster_join
{error,{test_case_failed,{[<<"example1.com">>],
              [<<"example1.com">>,<<"example2.com">>]}}}

Report log


pgsql_mnesia_23 / pgsql_mnesia / cd88a63
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / cd88a63
Reports root/ big
OK: 1738 / Failed: 0 / User-skipped: 298 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / cd88a63
Reports root/ big
OK: 3121 / Failed: 0 / User-skipped: 183 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / cd88a63
Reports root/ big
OK: 3110 / Failed: 1 / User-skipped: 200 / Auto-skipped: 0

mam_SUITE:rdbms_async_cache_prefs_cases:messages_filtered_when_prefs_default_policy_is_roster
{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok]\n"}}

Report log

Copy link
Contributor

@Premwoik Premwoik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 😃 Now it is easier to understand what is going on with config. I had only a quick look at the tests but everything seems to be good.

src/config/mongoose_config.erl Show resolved Hide resolved
@Premwoik Premwoik merged commit 4ef5f1e into master Nov 18, 2021
@Premwoik Premwoik deleted the config-unfold-global branch November 18, 2021 12:47
@Premwoik Premwoik modified the milestone: 5.1.0 May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants