Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix users number estimate in postgresql 14 #3319

Merged
merged 2 commits into from
Oct 5, 2021
Merged

Fix users number estimate in postgresql 14 #3319

merged 2 commits into from
Oct 5, 2021

Conversation

Premwoik
Copy link
Contributor

@Premwoik Premwoik commented Oct 5, 2021 •
edited
Loading

This handles the situation when table users has never yet been vacuumed or analyzed, and because of that, the estimated users number was -1

According to the description of reltuples column:

Number of live rows in the table. This is only an estimate used by the planner. It is updated by VACUUM, ANALYZE, and a few DDL commands such as CREATE INDEX. If the table has never yet been vacuumed or analyzed, reltuples contains -1 indicating that the row count is unknown.

@codecov
Copy link

codecov bot commented Oct 5, 2021 •
edited
Loading

Codecov Report

Merging #3319 (29056db) into master (2352d22) will increase coverage by 0.00%.
The diff coverage is 75.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3319   +/-   ##
=======================================
  Coverage   80.68%   80.69%           
=======================================
  Files         397      397           
  Lines       32426    32430    +4     
=======================================
+ Hits        26163    26169    +6     
+ Misses       6263     6261    -2
Impacted Files Coverage Δ
src/auth/ejabberd_auth_rdbms.erl 57.57% <75.00%> (+0.43%) ⬆️
src/mod_last_rdbms.erl 96.15% <0.00%> (-3.85%) ⬇️
src/domain/mongoose_domain_loader.erl 77.35% <0.00%> (-1.89%) ⬇️
src/mod_last.erl 86.76% <0.00%> (-1.48%) ⬇️
src/rdbms/mongoose_rdbms.erl 60.48% <0.00%> (-0.41%) ⬇️
src/mam/mod_mam_utils.erl 89.40% <0.00%> (-0.34%) ⬇️
src/mod_muc_log.erl 77.88% <0.00%> (ø)
src/ejabberd_c2s.erl 89.22% <0.00%> (ø)
src/global_distrib/mod_global_distrib_receiver.erl 80.00% <0.00%> (ø)
src/mod_muc.erl 74.05% <0.00%> (+0.22%) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2352d22...29056db. Read the comment docs.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 5, 2021 •
edited
Loading

small_tests_24 / small_tests / 3196bea
Reports root / small

internal_mnesia_24 / internal_mnesia / 3196bea
Reports root/ big
OK: 1589 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0

small_tests_23 / small_tests / 3196bea
Reports root / small

dynamic_domains_24 / pgsql_mnesia / 3196bea
Reports root/ big
OK: 2702 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

ldap_mnesia_24 / ldap_mnesia / 3196bea
Reports root/ big
OK: 1486 / Failed: 0 / User-skipped: 400 / Auto-skipped: 0

ldap_mnesia_23 / ldap_mnesia / 3196bea
Reports root/ big
OK: 1485 / Failed: 1 / User-skipped: 400 / Auto-skipped: 0

amp_big_SUITE:offline:offline_failure:error_deliver_to_stranger_test 
{error,
  {function_clause,
    [{exml_query,attr,[undefined,<<"type">>,undefined],[]},
     {amp_big_SUITE,contains_error,3,
       [{file,"/home/circleci/app/big_tests/tests/amp_big_SUITE.erl"},
        {line,916}]},
     {escalus_new_assert,assert,3,
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
        {line,37}]},
     {amp_big_SUITE,'-error_deliver_to_stranger_test/1-fun-0-',2,
       [{file,"/home/circleci/app/big_tests/tests/amp_big_SUITE.erl"},
        {line,626}]},
     {escalus_story,story,4,
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
        {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1784}]},
     {test_server,run_test_case_eval1,6,
       [{file,"test_server.erl"},{line,1293}]},
     {test_server,run_test_case_eval,9,
       [{file,"test_server.erl"},{line,1225}]}]}}

Report log

dynamic_domains_23 / pgsql_mnesia / 3196bea
Reports root/ big
OK: 2702 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 3196bea
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

mysql_redis_24 / mysql_redis / 3196bea
Reports root/ big
OK: 3054 / Failed: 0 / User-skipped: 228 / Auto-skipped: 0

elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 3196bea
Reports root/ big
OK: 1862 / Failed: 0 / User-skipped: 323 / Auto-skipped: 0

pgsql_mnesia_23 / pgsql_mnesia / 3196bea
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

mssql_mnesia_24 / odbc_mssql_mnesia / 3196bea
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

riak_mnesia_24 / riak_mnesia / 3196bea
Reports root/ big
OK: 1709 / Failed: 0 / User-skipped: 326 / Auto-skipped: 0

ldap_mnesia_23 / ldap_mnesia / 3196bea
Reports root/ big
OK: 1486 / Failed: 0 / User-skipped: 400 / Auto-skipped: 0

@Premwoik Premwoik force-pushed the fix-users-number-estimate-pgsql14 branch from 3196bea to 29056db Compare October 5, 2021 10:33
@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 5, 2021 •
edited
Loading

small_tests_24 / small_tests / 29056db
Reports root / small

internal_mnesia_24 / internal_mnesia / 29056db
Reports root/ big
OK: 1589 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0

small_tests_23 / small_tests / 29056db
Reports root / small

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 29056db
Reports root/ big
OK: 2699 / Failed: 3 / User-skipped: 184 / Auto-skipped: 0

service_domain_db_SUITE:db:db_keeps_syncing_after_cluster_join 
{error,{test_case_failed,{[<<"example1.com">>],
              [<<"example1.com">>,<<"example2.com">>]}}}

Report log

service_domain_db_SUITE:db:rest_with_auth:rest_delete_domain_cleans_data_from_mam 
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_rest_delete_domain_cleans_data_from_mam_87.408980@example.org/res1">>,
          escalus_tcp,<0.28481.1>,
          [{event_manager,<0.28475.1>},
           {server,<<"example.org">>},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_87.408980">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.28475.1>},
            {server,<<"example.org">>},
            {username,
              <<"bob_rest_delete_domain_cleans_data_from_mam_87.408980">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_87.408980">>},
           {server,<<"example.org">>},
           {host,<<"localhost">>},
           {port,5232},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_87.408980">>},
           {server,<<"example.org">>},
           {host,<<"localhost">>},
           {password,<<"makota3">>},
           {port,5232},
           {stream_id,<<"9ef4d773c039c096">>}]},
        5000],
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {service_domain_db_SUITE,
       '-rest_delete_domain_cleans_data_from_mam/1-fun-0-',5...

Report log

service_domain_db_SUITE:db:rest_without_auth:rest_delete_domain_cleans_data_from_mam 
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_rest_delete_domain_cleans_data_from_mam_95.387485@example.org/res1">>,
          escalus_tcp,<0.29105.1>,
          [{event_manager,<0.29099.1>},
           {server,<<"example.org">>},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_95.387485">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.29099.1>},
            {server,<<"example.org">>},
            {username,
              <<"bob_rest_delete_domain_cleans_data_from_mam_95.387485">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_95.387485">>},
           {server,<<"example.org">>},
           {host,<<"localhost">>},
           {port,5232},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"bob_rest_delete_domain_cleans_data_from_mam_95.387485">>},
           {server,<<"example.org">>},
           {host,<<"localhost">>},
           {password,<<"makota3">>},
           {port,5232},
           {stream_id,<<"0abcc5e49fcb9777">>}]},
        5000],
       [{file,
          "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {service_domain_db_SUITE,
       '-rest_delete_domain_cleans_data_from_mam/1-fun-0-',5...

Report log

dynamic_domains_mysql_redis_24 / mysql_redis / 29056db
Reports root/ big
OK: 2685 / Failed: 0 / User-skipped: 201 / Auto-skipped: 0

ldap_mnesia_24 / ldap_mnesia / 29056db
Reports root/ big
OK: 1486 / Failed: 0 / User-skipped: 400 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 29056db
Reports root/ big
OK: 2702 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

ldap_mnesia_23 / ldap_mnesia / 29056db
Reports root/ big
OK: 1486 / Failed: 0 / User-skipped: 400 / Auto-skipped: 0

elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 29056db
Reports root/ big
OK: 1862 / Failed: 0 / User-skipped: 323 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 29056db
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

mysql_redis_24 / mysql_redis / 29056db
Reports root/ big
OK: 3054 / Failed: 0 / User-skipped: 228 / Auto-skipped: 0

mssql_mnesia_24 / odbc_mssql_mnesia / 29056db
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

pgsql_mnesia_23 / pgsql_mnesia / 29056db
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 211 / Auto-skipped: 0

riak_mnesia_24 / riak_mnesia / 29056db
Reports root/ big
OK: 1709 / Failed: 0 / User-skipped: 326 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 29056db
Reports root/ big
OK: 2702 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

@Premwoik Premwoik marked this pull request as ready for review October 5, 2021 12:23
@Premwoik Premwoik added this to the 5.0.0 milestone Oct 5, 2021
chrzaszcz
chrzaszcz approved these changes Oct 5, 2021
View reviewed changes
Copy link
Member

@chrzaszcz chrzaszcz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! The only thing I could point out is to check (either manually or automatically) that the query for the estimate can return something else than '-1'. I think you've already done that.

@vkatsuba vkatsuba merged commit 498ec6e into master Oct 5, 2021
@vkatsuba vkatsuba deleted the fix-users-number-estimate-pgsql14 branch October 5, 2021 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vkatsuba vkatsuba vkatsuba approved these changes

@chrzaszcz chrzaszcz chrzaszcz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
5.0.0
Development

Successfully merging this pull request may close these issues.
4 participants
@Premwoik @mongoose-im @chrzaszcz @vkatsuba