Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prepared queries for modules #3039

Merged
merged 88 commits into from
Mar 2, 2021
Merged

Prepared queries for modules #3039

merged 88 commits into from
Mar 2, 2021

Conversation

arcusfelis
Copy link
Contributor

@arcusfelis arcusfelis commented Feb 25, 2021 •
edited by chrzaszcz
Loading

@arcusfelis arcusfelis mentioned this pull request Feb 25, 2021
Closed
@mongoose-im
Copy link
Collaborator

mongoose-im commented Feb 25, 2021 •
edited
Loading

9115.1 / Erlang 23.0.3 / small_tests / 26521e4
Reports root / small

9115.2 / Erlang 23.0.3 / internal_mnesia / 26521e4
Reports root/ big
OK: 1499 / Failed: 0 / User-skipped: 161 / Auto-skipped: 0

9115.3 / Erlang 23.0.3 / odbc_mssql_mnesia / 26521e4
Reports root/ big
OK: 2767 / Failed: 0 / User-skipped: 229 / Auto-skipped: 0

9115.4 / Erlang 23.0.3 / mysql_redis / 26521e4
Reports root/ big
OK: 2768 / Failed: 1 / User-skipped: 234 / Auto-skipped: 0

mam_SUITE:rdbms_cache_prefs_cases:messages_filtered_when_prefs_default_policy_is_never 
{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok,ok]\n"}}

Report log

9115.7 / Erlang 23.0.3 / elasticsearch_and_cassandra_mnesia / 26521e4
Reports root/ big
OK: 331 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0

9115.5 / Erlang 23.0.3 / riak_mnesia / 26521e4
Reports root/ big
OK: 1625 / Failed: 0 / User-skipped: 181 / Auto-skipped: 0

9115.6 / Erlang 23.0.3 / ldap_mnesia / 26521e4
Reports root/ big
OK: 1401 / Failed: 0 / User-skipped: 259 / Auto-skipped: 0

9115.9 / Erlang 22.3 / pgsql_mnesia / 26521e4
Reports root/ big / small
OK: 2780 / Failed: 0 / User-skipped: 216 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Mar 1, 2021 •
edited
Loading

9154.1 / Erlang 23.0.3 / small_tests / ad1b72d
Reports root / small

9154.2 / Erlang 23.0.3 / internal_mnesia / ad1b72d
Reports root/ big
OK: 1499 / Failed: 0 / User-skipped: 161 / Auto-skipped: 0

9154.3 / Erlang 23.0.3 / odbc_mssql_mnesia / ad1b72d
Reports root/ big
OK: 2767 / Failed: 0 / User-skipped: 229 / Auto-skipped: 0

9154.4 / Erlang 23.0.3 / mysql_redis / ad1b72d
Reports root/ big
OK: 2762 / Failed: 0 / User-skipped: 234 / Auto-skipped: 0

9154.7 / Erlang 23.0.3 / elasticsearch_and_cassandra_mnesia / ad1b72d
Reports root/ big
OK: 331 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0

9154.5 / Erlang 23.0.3 / riak_mnesia / ad1b72d
Reports root/ big
OK: 1625 / Failed: 0 / User-skipped: 181 / Auto-skipped: 0

9154.6 / Erlang 23.0.3 / ldap_mnesia / ad1b72d
Reports root/ big
OK: 1401 / Failed: 0 / User-skipped: 259 / Auto-skipped: 0

9154.9 / Erlang 22.3 / pgsql_mnesia / ad1b72d
Reports root/ big / small
OK: 2780 / Failed: 0 / User-skipped: 216 / Auto-skipped: 0

@chrzaszcz chrzaszcz changed the title Prepared queries - DRAFT 3 Prepared queries Mar 1, 2021
chrzaszcz
chrzaszcz approved these changes Mar 1, 2021
View reviewed changes
Copy link
Member

@chrzaszcz chrzaszcz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a huge chunk of great work!

All PR's were reviewed before.
A quick look at the changes shows no issues.

@chrzaszcz chrzaszcz changed the title Prepared queries Prepared queries for modules Mar 1, 2021
@codecov
Copy link

codecov bot commented Mar 1, 2021 •
edited
Loading

Codecov Report

Merging #3039 (a8d9cab) into master (2f51f6d) will increase coverage by 0.47%.
The diff coverage is 91.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3039      +/-   ##
==========================================
+ Coverage   78.02%   78.50%   +0.47%     
==========================================
  Files         374      378       +4     
  Lines       31190    31121      -69     
==========================================
+ Hits        24337    24431      +94     
+ Misses       6853     6690     -163
Impacted Files Coverage Δ
src/mam/mod_mam.erl 89.25% <ø> (ø)
src/muc_light/mod_muc_light_db_mnesia.erl 89.47% <ø> (+4.47%) ⬆️
src/rdbms/rdbms_queries_mssql.erl 100.00% <ø> (ø)
src/rdbms/mongoose_rdbms_odbc.erl 75.78% <45.45%> (-1.86%) ⬇️
src/mam/mod_mam_rdbms_arch.erl 48.45% <47.14%> (+9.18%) ⬆️
src/rdbms/mongoose_rdbms.erl 64.58% <63.33%> (-0.82%) ⬇️
src/mam/mod_mam_rdbms_user.erl 72.58% <84.61%> (+8.19%) ⬆️
src/mam/mod_mam_utils.erl 89.70% <85.71%> (+0.70%) ⬆️
src/mam/mam_filter.erl 91.66% <91.66%> (ø)
src/mam/mod_mam_muc_rdbms_arch.erl 94.73% <92.75%> (+6.22%) ⬆️
... and 35 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f51f6d...a8d9cab. Read the comment docs.

chrzaszcz
chrzaszcz requested changes Mar 1, 2021
View reviewed changes
Copy link
Member

@chrzaszcz chrzaszcz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized we need to merge the MUC PR here, see #3028

@mongoose-im
Copy link
Collaborator

mongoose-im commented Mar 1, 2021 •
edited
Loading

9155.1 / Erlang 23.0.3 / small_tests / 7707c01
Reports root / small

9155.2 / Erlang 23.0.3 / internal_mnesia / 7707c01
Reports root/ big
OK: 1499 / Failed: 0 / User-skipped: 161 / Auto-skipped: 0

9155.4 / Erlang 23.0.3 / mysql_redis / 7707c01
Reports root/ big
OK: 2762 / Failed: 0 / User-skipped: 234 / Auto-skipped: 0

9155.3 / Erlang 23.0.3 / odbc_mssql_mnesia / 7707c01
Reports root/ big
OK: 2767 / Failed: 0 / User-skipped: 229 / Auto-skipped: 0

9155.7 / Erlang 23.0.3 / elasticsearch_and_cassandra_mnesia / 7707c01
Reports root/ big
OK: 331 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0

9155.5 / Erlang 23.0.3 / riak_mnesia / 7707c01
Reports root/ big
OK: 1625 / Failed: 0 / User-skipped: 181 / Auto-skipped: 0

9155.6 / Erlang 23.0.3 / ldap_mnesia / 7707c01
Reports root/ big
OK: 1401 / Failed: 0 / User-skipped: 259 / Auto-skipped: 0

@chrzaszcz
Copy link
Member

@arcusfelis could you update the PR description so that we don't lose track of the PR's merged to this branch before you rebased it and changed the commit history?

@arcusfelis
MAM: Use prepared queries for message retraction #2961 - first commit
0afdd09
@arcusfelis
Make lookups prepared in mod_mam
257b122
@arcusfelis
Reorder function in mod_mam_rdbms_arch.erl
0e37eb0
@arcusfelis
Convert gdpr query to prepared
fccb037
@arcusfelis
Add maybe_reserve helper
90f1c94
@arcusfelis
Compactify code for do_lookup_messages
8a1ce37
@arcusfelis
Adress review commits
e91f797
@arcusfelis
Use lookup_field record in mod_mam_rdbms_arch
5e53699
@arcusfelis
Refactor opt_count logic (less black magic)
c659c15
@arcusfelis
Add Index Hint
c4b3c3a
@arcusfelis
Reorder code
d089389
@arcusfelis
Add db_mappings
a608dbf
@arcusfelis
Remove GDPR helpers - use standard functions
b7fb067
@arcusfelis
Make prepare_value pure by introducing env_vars
8a77ec7
@arcusfelis
Use env_vars for lookups
aeb81dc
@arcusfelis
Use env_vars for retraction
9f7d0eb
@arcusfelis
Use generic lookup for GDPR data extraction in mod_mam
dc5f3c5
@arcusfelis
Shorten calc_count specs
4edd718
@arcusfelis
Add specs for with_env functions
ed85595
@arcusfelis
Use lookup_query for archive_size
8b76586
@arcusfelis
Simplify error handling in mod_roster_rdbms
5b12c22 
Require database jid to be parsable
@arcusfelis
ROSTER: Edit comment for character_to_integer #2985 - final commit
4f3a28b
@arcusfelis
Use prepared queries in mod_privacy_rdbms #3004
c406764
@arcusfelis
MUC-light: Make select_room_id prepared #3011 - first commit
88f4394 
Make select_room_id_and_version prepared too
@arcusfelis
Split create_room function into two
69cbb98 
Prepare query for muc_light_insert_room function
@arcusfelis
Use prepared query for update_room_version and delete_room functions
463f358
@arcusfelis
Use prepared function for select_affs/2 in MUC-light
975cc35
@arcusfelis
Use prepared function for select_affs/1 in MUC-light
d07bad7
@arcusfelis
Add decode_affs/1 function
a63262d
@arcusfelis
User prepared insert_aff function
93d349f
@arcusfelis
User prepared update_aff function
55d581e
@arcusfelis
Use prepared delete_aff/delete_affs functions
84d73f1
@arcusfelis
Make get_config function prepared
456d6cd
@arcusfelis
Use prepared functions for insert, update and delete room configs
f64038e
@arcusfelis
Use prepared queries for muc-light blocking
c5d3348
@arcusfelis
Fix types for affiliations
b8cabbe
@arcusfelis
Fix muc-light queries for ODBC
1ff7f42
@arcusfelis
Fix comment and typespec based on review
d575c88
@arcusfelis
Fix indention in mod_muc_light_db_rdbms
c7783ab
@arcusfelis
Replace find_key with field_to_odbc_type
5935d79
@arcusfelis
MUC-light: Match everything in create_room_transaction LCs #3011 - fi…
2bf2ec3 
…nal commit
@arcusfelis
Use prepared queries for mod_last #3019
d6df02b 
Remove unused rdbms_queries:update/5 function
@arcusfelis
Insert only a diff of privacy_data #3004
cd9ffe7 
Retry privacy update transaction
Update eodbc. Return bigints as numbers, not binaries now.
Smaller data type for the ord field.
@arcusfelis
Use prepared queries for mod_offline #3040
d5ebedc
@arcusfelis
Use prepared queries for mod_muc_db_rdbms #3028
a8d9cab
chrzaszcz
chrzaszcz approved these changes Mar 2, 2021
View reviewed changes
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good after adding MUC

@chrzaszcz chrzaszcz merged commit 272ee8c into master Mar 2, 2021
@chrzaszcz chrzaszcz deleted the mu-prepared-queries3 branch March 2, 2021 07:46
@leszke leszke added this to the 4.2.0 milestone Apr 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chrzaszcz chrzaszcz chrzaszcz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.2.0
Development

Successfully merging this pull request may close these issues.
4 participants
@arcusfelis @mongoose-im @chrzaszcz @leszke