[Gateway] Guild Sharding

[jhgg]

Guild sharding is coming soon, after we test & deploy it internally.

You'll soon be able to specify a shard in the ready packet. This will cause the gateway to operate in shard mode, which will distribute guilds between each shard. The shard key is a tuple of {shard_id, num_shards}, where shard_id is a number starting at 0 and less than num_shards. num_shards must be a number greater than 1 (having 1 as num_shards is kinda pointless as that's the default mode of operation).

In shard mode, the distribution of guilds is calculated by (guild_id >> 22) % num_shards == shard_id. Guilds that are not on your shard will not appear on the shard list, meaning they will not show up in the guilds object in the ready packet, and that you won't receive any events for guilds not on your shard. DMs will only be sent to shard_id 0. This will make processing of DMs a bit tricky on the bot side, but still possible.

Todo List:

• Actually deploy this - @jhgg
• Add to documentation - @b1naryth1ef Gateway Guild Sharding #31
• Implement in airhorn.solutions as an example for reference - @b1naryth1ef
• 2500 guild limit per shard (may be higher, but won't be lower) - will come, but will roll this one out later, as bot libs add support for sharding for larger bots. This will be enforced beginning September 30, 2016 October 10, 2016 Jan 3, 2017

[cryptiklemur]

Should mention that the tuple needs to be a tuple of integers

[jhgg]

where shard_id is a number

[Fabricio20]

I think we should be able to send a region string along with the shard id, so that specific shard would only return guilds that have voice servers on that region, allowing for better audio load balancing/ping rates.

[jhgg]

Unfortunately that's not possible. We don't know of the guild region's ahead of time when sharding.

[Fabricio20]

What about a shard_move endpoint then? I could send a guild id to it and it's shard would be moved, allowing for the same thing I mentioned above about audio distribution.

(If you think that it would cause too much gateway spam, you could also Jake it).

[cookkkie]

The best way to handle that would be to handle it in your bot code. Not that difficult in theory.

[jhgg]

We have no plans to direct certain guilds to certain shards. The guilds are sharded based on the algorithm defined above.

[AraHaan]

or if they have less than 2500 servers the bots would be uneffected however my bot sometimes disconnects randomly and its in less than 150 Servers. RIP 😛

[macdja38]

The limit should probably be added to the official documentation ahead of it's release so that those that don't follow the api server and this github have time to prepair when using the docs.

It may also be helpfull to include the rate at which subsequent shards should attempt to connect in the documentation. (would be nice if this was a bit lower, especially for bot's like aethex that require a ton of shards to support their user count)

[jhgg]

The rate at which you can connect to the gateway is 1 connection per 5 second per account. That means it'd take 60 seconds to connect to 12 shards - which I think is completely acceptable.

Will add these to docs.

[Hinara]

Just a question about that how to know on how many guilds are the bot ?
If I've got true there is actually no way for the bot to know that they are on more than 2500 servers without loading the guilds array on the web socket.

[b1naryth1ef]

@robbix1206 https://discordapp.com/developers/docs/resources/user#get-current-user-guilds

[Hinara]

Thanks ^^ Haven't seen it.

[itslukej]

When is the 2500 guild limit going to be enforced? It's been over two months now

[b1naryth1ef]

@itslukej when we deploy it. Bots should already be respecting this limit, and we will likely give no notification when it eventually goes out.

[jhgg]

To anyone watching this issue, we (finally) plan to roll this out tonight.

[Tesla010]

Is it possible to manually calculate shard number?

[jhgg]

Yes. Given the number of servers your bot is in. And the limit of how many servers can exist in a single shard, it is very easy to calculate the shard count required.

For any further inquiries about this, open a new issue with a specific and detailed question that you need clarification on.