Verify scaling target requirements

[richard-ramos]

waku-org/pm#12
Understand the expected:

• rate of chat messages
• rate of control messages
• rate of store queries
• avg expected bandwidth usage of each of the above

cc: @Menduist, @jm-clius

[richard-ramos]

The tables receivedMessages, receivedMessageAggregate, protocolStatsRate and protocolStatsTotal (which can be accesed via https://superset.infra.status.im/superset/sqllab/) and Grafana can be used to obtain the above data

Useful queries (possible to do after version 0.10.0 is released)

Number of chat messages per day

SELECT date_trunc('day', to_timestamp(createdAt)) "day", count(*) as num_messages
FROM receivedMessages
WHERE messageType IN ('CHAT_MESSAGE', 'EMOJI_REACTION', 'EDIT_MESSAGE', 'DELETE_MESSAGE')
-- USE `NOT IN` to obtain control messages
GROUP BY 1

Average number and byte size of messages per user per day

SELECT day, avg(num_msgs) avg_num_msg_per_user, avg(avg_message_size_bytes) avg_msg_size_bytes FROM (
  SELECT date_trunc('day', to_timestamp(createdAt)) "day", receiverkeyuid, count(*) num_msgs, avg(messagesize) avg_message_size_bytes
  FROM receivedMessages
  WHERE messageType IN ('CHAT_MESSAGE', 'EMOJI_REACTION', 'EDIT_MESSAGE', 'DELETE_MESSAGE')
  -- USE `NOT IN` to obtain control messages
  GROUP BY 1, receiverkeyuid 
) q
GROUP BY day

Avg total bandwidth used by a protocol by an user session per day

SELECT createdAt, protocolname, avg(sumTi), avg(sumOut)
FROM (
  SELECT createdAt, peerId, protocolname, sum(totalin) sumTi, sum(totalout) sumOut FROM protocolstatstotals 
  GROUP BY peerId, protocolName, createdAt
) q
GROUP BY createdAt, protocolName

[rymnc]

adding a query to this thread -

average size of a "BACKUP" msg sent by a user

SELECT chatid, AVG(messagesize)
FROM receivedmessages
WHERE messagetype = 'BACKUP' AND messagesize > 0
GROUP BY chatid
ORDER BY AVG(messagesize) DESC 
LIMIT 25;

[fryorcraken]

Is there much that can be done before dogfooding? What are the next expected steps?