Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Service management rework #3620

Merged
merged 10 commits into from
Apr 4, 2022
Merged

Service management rework #3620

merged 10 commits into from
Apr 4, 2022

Conversation

chrzaszcz
Copy link
Member

@chrzaszcz chrzaszcz commented Apr 1, 2022

Rework service configuration and management, following the logic that is already present for modules, and thus making the two more unified.

Main changes:

  • Put services in a map that is always included.
  • Manage services in mongoose_service, like modules are managed in mongoose_modules. There is some discrepancy in naming, but the name is not changed as mongoose_service is itself a behaviour. There is no replace_services logic for tests, because it is would be just unused in the current tests.
  • Resolve/sort dependencies in mongoose_service_deps, similarly to gen_mod_deps. Service deps are always hard and never have opts, which simplifies the logic.
  • Make unit tests for services and their deps similar to the corresponding tests for modules.

See the commit messages for more details.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 1, 2022

small_tests_24 / small_tests / 7ebbfc3
Reports root / small


small_tests_23 / small_tests / 7ebbfc3
Reports root / small


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 7ebbfc3
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 7ebbfc3
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_mysql_redis_24 / mysql_redis / 7ebbfc3
Reports root/ big
OK: 2834 / Failed: 0 / User-skipped: 150 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 7ebbfc3
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 7ebbfc3
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 7ebbfc3
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


internal_mnesia_24 / internal_mnesia / 7ebbfc3
Reports root/ big
OK: 1547 / Failed: 0 / User-skipped: 360 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 7ebbfc3
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 7ebbfc3
Reports root/ big
OK: 1854 / Failed: 0 / User-skipped: 368 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 7ebbfc3
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / 7ebbfc3
Reports root/ big
OK: 3220 / Failed: 0 / User-skipped: 147 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 7ebbfc3
Reports root/ big
OK: 1697 / Failed: 0 / User-skipped: 367 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 7ebbfc3
Reports root/ big

inbox_extensions_SUITE:async_pools:one_to_one:archive_full_archive_can_be_fetched
{error,
 {{inbox_size,ok,
   [{times,1,
   {error,
    {inbox_size,3,
     [{times,1,
     #{respond_iq =>
      {xmlel,<<"iq">>,
       [{<<"from">>,
         <<"alicE_archive_full_archive_can_be_fetched_912@localhost">>},
        {<<"to">>,
         <<"alicE_archive_full_archive_can_be_fetched_912@localhost/res1">>},
        {<<"id">>,<<"d55356146a3b9c8d899ca19f88f5dcbd">>},
        {<<"type">>,<<"result">>}],
       [{xmlel,<<"fin">>,
         [{<<"xmlns">>,<<"erlang-solutions.com:xmpp:inbox:0">>}],
         [{xmlel,<<"active-conversations">>,[],[{xmlcdata,<<"0">>}]},
        {xmlel,<<"count">>,[],[{xmlcdata,<<"2">>}]},
        {xmlel,<<"unread-messages">>,[],[{xmlcdata,<<"0">>}]}]}]},
       respond_messages =>
      [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alicE_archive_full_archive_can_be_fetched_912@localhost">>},
         {<<"to">>,
        <<"alicE_archive_full_archive_can_be_fetched_912@localhost/res1">>},
         {<<"id">>,<<"1648-800900-713129">>}],
        [{xmlel,<<"result">>,
        [{<<"xmlns">>,<<"erlang-solutions.com:xmpp:inbox:0">>},
         {<<"unread">>,<<"0">>},
         {<<"queryid">>,<<"d55356146a3b9c8d899ca19f88f5dcbd">>}],
        [{xmlel,<<"forwarded">>,
          [{<<"xmlns">>,<<"urn:xmpp:forward:0">>}],
          [{xmlel,<<"delay">>,
          [{<<"xmlns">>,<<"urn:xmpp:delay">>},
           {<<"stamp">>,<<"2022-04-01T08:14:58.273585Z">>}],
          []},
           {xmlel,<<"message">>,
          [{<<"from">>,
            <<"alic...

Report log

@codecov
Copy link

codecov bot commented Apr 1, 2022

Codecov Report

Merging #3620 (4a3fec4) into master (7cbb30d) will increase coverage by 0.01%.
The diff coverage is 97.53%.

❗ Current head 4a3fec4 differs from pull request most recent head 1a7a30d. Consider uploading reports for the commit 1a7a30d to get more accurate results

@@            Coverage Diff             @@
##           master    #3620      +/-   ##
==========================================
+ Coverage   80.96%   80.98%   +0.01%     
==========================================
  Files         424      425       +1     
  Lines       32061    32069       +8     
==========================================
+ Hits        25958    25970      +12     
+ Misses       6103     6099       -4     
Impacted Files Coverage Δ
src/config/mongoose_config_spec.erl 99.21% <ø> (ø)
src/mongoose_service_deps.erl 95.23% <95.23%> (ø)
src/mongoose_service.erl 96.36% <97.50%> (+18.75%) ⬆️
src/config/mongoose_config_parser.erl 92.30% <100.00%> (+2.06%) ⬆️
src/config/mongoose_config_parser_toml.erl 99.20% <100.00%> (-0.03%) ⬇️
src/ejabberd_app.erl 94.36% <100.00%> (-0.38%) ⬇️
src/mongoose_modules.erl 100.00% <100.00%> (ø)
...system_metrics/service_mongoose_system_metrics.erl 77.41% <100.00%> (+4.40%) ⬆️
src/rdbms/mongoose_rdbms.erl 62.54% <0.00%> (-1.82%) ⬇️
src/mod_blocking.erl 88.46% <0.00%> (-1.29%) ⬇️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7cbb30d...1a7a30d. Read the comment docs.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 1, 2022

small_tests_24 / small_tests / 4a3fec4
Reports root / small


small_tests_23 / small_tests / 4a3fec4
Reports root / small


dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 4a3fec4
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4a3fec4
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_mysql_redis_24 / mysql_redis / 4a3fec4
Reports root/ big
OK: 2834 / Failed: 0 / User-skipped: 150 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 4a3fec4
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 4a3fec4
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 4a3fec4
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


internal_mnesia_24 / internal_mnesia / 4a3fec4
Reports root/ big
OK: 1547 / Failed: 0 / User-skipped: 360 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 4a3fec4
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 4a3fec4
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 4a3fec4
Reports root/ big
OK: 1854 / Failed: 0 / User-skipped: 368 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / 4a3fec4
Reports root/ big
OK: 3220 / Failed: 0 / User-skipped: 147 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 4a3fec4
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 4a3fec4
Reports root/ big
OK: 1697 / Failed: 0 / User-skipped: 367 / Auto-skipped: 0

Unify service and module processing:
- Resolve deps
- Unfold opts (this is temporary until only maps are used)

Also: move 'build_state' to a more suitable place.
Unify the way services and modules are management

Follow the solution for modules:
- Store the config in a map and do not copy the config to ETS.
  This prevents inconsistency between the two and speeds up access.
- Delegate dependency management to a separate module.
- Start and stop all services in the dependency-related order.
- Provide API for ensuring a service is stopped/started in tests,
  updating the configuration accordingly.
- Drop the check if MongooseIM is running, because it does not seem to
  be a real risk. Anyway, service management code is not special
  and should not take the responsibility of stopping the whole node.
Follow the logic that is already present for modules.
There are two main steps:
1. Dependency resolution.
2. Service sorting according to the dependency graph.

Only maps are supported, but it is not a problem, because no services
have deps anyway and all of them are going to use maps very soon.
- Services are always present
- Service opts are unfolded

Also: remove unused function
Follow the module tests
- Simplify service start/stop tests
- Check a subset of the deps cases for module corresponding to the
  features that are present for services
This check is in gen_mod, so the test should be here as well.

Also: update module tests to use maps.
@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 1, 2022

small_tests_24 / small_tests / 1a7a30d
Reports root / small


small_tests_23 / small_tests / 1a7a30d
Reports root / small


dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 1a7a30d
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 1a7a30d
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


dynamic_domains_mysql_redis_24 / mysql_redis / 1a7a30d
Reports root/ big
OK: 2834 / Failed: 0 / User-skipped: 150 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 1a7a30d
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 1a7a30d
Reports root/ big
OK: 1506 / Failed: 0 / User-skipped: 401 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 1a7a30d
Reports root/ big
OK: 2851 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


internal_mnesia_24 / internal_mnesia / 1a7a30d
Reports root/ big
OK: 1547 / Failed: 0 / User-skipped: 360 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 1a7a30d
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 1a7a30d
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 1a7a30d
Reports root/ big
OK: 1854 / Failed: 0 / User-skipped: 368 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / 1a7a30d
Reports root/ big
OK: 3220 / Failed: 0 / User-skipped: 147 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 1a7a30d
Reports root/ big
OK: 3225 / Failed: 0 / User-skipped: 142 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 1a7a30d
Reports root/ big
OK: 1697 / Failed: 0 / User-skipped: 367 / Auto-skipped: 0

@chrzaszcz chrzaszcz marked this pull request as ready for review April 1, 2022 13:57
Copy link
Contributor

@Premwoik Premwoik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looks good to me :) I like the new service tests, look more clearly.

@Premwoik Premwoik merged commit 6fcb515 into master Apr 4, 2022
@Premwoik Premwoik deleted the services-map branch April 4, 2022 08:30
@Premwoik Premwoik added this to the 5.1.0 milestone May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants