Mongoose c2s #3765

NelsonVides · 2022-09-20T13:30:30Z

This is a feature branch for the new mongoose_c2s prototype. The goal is to have all of CI re-enabled and green.

Pending are things that were partially inside of the old ejabberd_c2s:

And tests that used to depend on the internals of ejabberd_c2s, ejabberd_c2s_sup, or the older version of ranch:

fix new version of ranch
Refactor tests using old c2s internals to be compatible with the new one
Fix all remaining disabled test suites
Verify test coverage does not go down

The flow is as follows: init casts a `connect` message, to indicate the process that it needs to establish the tcp connection, outside of the init function that blocks the supervisor. Then all states are handled in a single handle_event. The event connect in state connecting indicates to end negotiating the connection. The event info with payload from tcp or ssl, regardless of the state, simply parses the payload using exml_stream, and adds all parsed packets as internal action events of the OTP's gen_statem behaviour, that is, they will be processed immediately as the next event in the state machine, without involved message passing. Then, as internal events, all packets are processed, according to their running state machine state. One specific event is expected in that state, and an appropriate handle_* helper does the work. States can be compound, so now instead of sticking data of a specific state into the state-machine memory, we can make it part of the state itself, and it will be used in the next event in that state, see `{wait_for_feature, after auth, _}` and `{wait_for_feature, before_auth, _}`. Code that specifies stanzas is moved to a helper module that construct all the verbose #xmlel{} packets, to keep the c2s module smaller and separate protocol from state machine.

As connections are inherently long-lived, we'd want to avoid the possibility of an attacker opening many connections that then only very slowly progress, taking ports and memory from the system. If the connection doesn't make progress fast enough towards the session establishment, it should be dropped. This, of course, is configurable.

Note that gen_hook type now allows to take the type for the function handler, in order to specify stronger types. Note that dialyzer might not be strong enough to identify this polymorphism, but it might probably be in the future, as well as different tools like Gradualizer. Also mongoose_acc uses erlang:system_time/1 instead of the os equivalent, to better resist time warps and also for performance, and likewise such time is later used to calculate a metric for the time the message took to be handled. Now, all c2s handling stanzas will be captured and handled in hooks, and mongoose_acc will be a universal accumulator.

Set a separate module, with its own edoc comments, all events that are to be triggered by mongoose_c2s.

The idea is that hooks now can request the gen_statem engine to do things on return, like putting payloads on the socket to deliver to the user, adding timeouts to the engine, adding ad-hoc events to be triggered by the engine (see gen_statem's next_event action), or pushing a new callback module to take control over the state machine. This accumulator, specific to the statem engine, is embedded into the mongoose_acc map, and then parsed and acted upon on all hook and handle_event returns.

This includes handling fast_tls, cleaning the declared types to avoid confusion between what is a 'state' and what is 'data' in the context of a state machine, implementing the language callbacks, bouncing messages on termination, and adding an interface to handle handlers.

Prepare all knowledge of the tls configuration, whether to use ranch's reuseport extension, and pass to ranch all other parameters that were present in the c2s listener config.

C2s/enable all suites This PR enabled the last test suite: tcp_listener_SUITE. It was testing an issue with a c2s connection breaking while it was being accepted by mongoose_tcp_listener. Now c2s connections are handled by the ranch acceptor, which does not seem to have such an issue (I checked the code). However, s2s and component (service) listeners still might suffer from the issue, so the updated test ensures that both are working correctly.

Routing should be done only for stanzas. Arbitrary messages shouldn't be sent this way - they are already sent directly in multiple places.

The events are tagged to encourage meaningful and consistent categories, and prevent evnts like '{item, _, _}' which are hard to debug.

C2s/Do not route broadcast tuples

Fixing bug in mongoose_metrics_hooks

This was untested, and actually not working with mongoose_c2s. Also: - Do setup/teardown in init/end_per_testcase instead of doing it in the test case. - Do not setup/teardown inside parallel tests.

Previously the handlers were installed with the access rules themselves, what caused: - Possible duplication of handlers in case of same rules. - Running all handlers sequentially, effectively performing an intersection of all rules instead of applying rules for a given listener. Now the handlers are installed with the listener ID's, and for each new session these ID's are compared with the c2s data, resulting in only the right handler being executed. The access rules can be just extracted from the c2s data itself.

C2s/fix listener access hook

mongoose-im · 2023-01-30T08:55:07Z

small_tests_24 / small_tests / 8f20f7f
Reports root / small

small_tests_25 / small_tests / 8f20f7f
Reports root / small

ldap_mnesia_24 / ldap_mnesia / 8f20f7f
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 8f20f7f
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 8f20f7f
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 8f20f7f
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 8f20f7f
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 8f20f7f
Reports root/ big
OK: 4160 / Failed: 6 / User-skipped: 91 / Auto-skipped: 0

inbox_extensions_SUITE:regular:one_to_one:pagination:max_queries_can_be_limited

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"[email protected]/res1">>,
          escalus_tcp,<0.13130.1>,
          [{event_manager,<0.13086.1>},
           {server,<<"domain.example.com">>},
           {username,<<"bOb_max_queries_can_be_limited_1533">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13086.1>},
            {server,<<"domain.example.com">>},
            {username,<<"bOb_max_queries_can_be_limited_1533">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"bob_max_queries_can_be_limited_1533">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"bOb_max_queries_can_be_limited_1533">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"6884d926710fe09c">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl,3,[{file,"lists.erl"},{line,1350}]},
     {in...

Mongoose c2s #3765

Mongoose c2s #3765

Conversation

NelsonVides commented Sep 20, 2022 • edited Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

mongoose-im commented Jan 30, 2023 • edited Loading

chrzaszcz commented Jan 30, 2023

chrzaszcz commented Feb 2, 2023 • edited Loading

DenysGonchar left a comment

Choose a reason for hiding this comment

NelsonVides commented Sep 20, 2022 •

edited

Loading

mongoose-im commented Jan 30, 2023 •

edited

Loading

chrzaszcz commented Feb 2, 2023 •

edited

Loading