[Question ]Do we implement the auto guild sharding ?

[Hinara]

discord/discord-api-docs#17
Do we let user deal with that or do we automatically do that ?
(I personally it's better that the users deal with that because they can put the programs on multiple computers for big bots like airhorn)

[iopred]

Having a small sharded wrapper might be the way to go. I don't want to put
sharding support in the core library.

On Sep 30, 2016 4:09 AM, "robbix1206" [email protected] wrote:

discord/discord-api-docs#17
discord/discord-api-docs#17
Do we let user deal with that or do we automatically do that ?
(I personally it's better that the users deal with that because they can
put the programs on multiple computers for big bots)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#265, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABdVgn0cneATUlM1DQWgKM-iG35ptG8nks5qvO3ogaJpZM4KK-gt
.

[Hinara]

Maybe one thing is to add something like "function (*Session) setShard(ShardID,ShardCount)"
In order to able people to split their bots and which need to be called beetween Session.New(...) and Session.Open()
And maybe a function to make them able to easily know how many Shards did they need.

[iopred]

Session.ShardID = 0
Session.ShardCount = 5

Isn't too difficult :)

As for number of shards, I'm hoping Discord will provide an API as there's
no good way to calculate this at the moment. (I set it manually on the
command line)

On Fri, Sep 30, 2016 at 9:32 AM, robbix1206 [email protected]
wrote:

Maybe one thing is to add something like "function (*Session)
setShard(ShardID,ShardCount)"
In order to able people to split their bots and which need to be called
beetween Session.New(...) and Session.Open()
And maybe a function to make them able to easily know how many Shards did
they need.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#265 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABdVgsWEmPU3shfkbsj8ToapzgGoMFhjks5qvTmRgaJpZM4KK-gt
.

[Hinara]

Think we have answer to all that questions ^^

[kdrag0n]

Any word on this? I'm considering using discordgo for my bot that's approaching 3000 guilds and already requires sharding. It's currently written in Java and uses JDA, but I know Go and want to rewrite it to be more memory efficient and faster.

[iopred]

What do you need to know? DiscordGo supports sharding and it's quite simple to wrap together multiple Sessions.

[safinn]

Could you provide a small example of sharding using DiscordGo please?

[Hinara]

I reopen it because of the /gateway/bot endpoint.
https://discordapp.com/developers/docs/topics/gateway#get-gateway-bot

This route should be called once when starting up numerous shards, with the response being cached and passed to all sub-shards/processes.

I think we need to change Session.Open() as it calls the /gateway endpoint one time for each bot !
We need to be able to pass the WSS URL to Open or something like that.

Proposals:

1. Create a specific Session.OpenBot() which take the wss url as argument
2. Create a specific Session.OpenBot() which call the bot endpoint and return endpoint information.

BTW Maybe is that better to make Gateway and GatewayBot independent from a Session.
(Easier to manage with a loop as you don't need a specific case for the first one)

[eslindsey]

This is from over a year ago. Is it resolved so that it correctly calls the GET /gateway/bot only once?

[bwmarrin]

I'm not sure if the above is still relevant anymore. The documentation now states

Returns an object based on the information in Get Gateway, plus additional metadata that can help during the operation of large or sharded bots. Unlike the Get Gateway, this route should not be cached for extended periods of time as the value is not guaranteed to be the same per-call, and changes as the bot joins/leaves guilds.

I don't see in the documentation (at the moment) where it states it should only be called once even with multiple shards.

Right now DiscordGo calls the /gateway/ endpoint to get the gateway. That value is cached so reconnects do not have to call it again, but separate shards would each still call the endpoint.

This issue should probably be closed, actually. Are you running into any specific issue with this or just following up (which I do appreciate)?

[Hinara]

Yes the documentation was modified in this commit discord/discord-api-docs@77af69f
This PR was also to add some system to use this /gateway/bot endpoint which was done in commit 9d3cd03
This is good for me

[eslindsey]

@bwmarrin I was just following up because I am developing a bot which might need sharding, and I'd rather design it with that feature from the beginning. But I couldn't find any examples using sharding, and this issue was one of the first results on Google for discordgo sharding.

[bwmarrin]

Ah, well I unfortunately?.. do not have a bot that's popular enough to need sharding :( If you join the Discord Gophers chat server there are a few people there that have made bots doing sharding that might be able to give you some advice.

The basis though, is you need to set Session.ShardID and Session.ShardCount before calling Session.Open(). ShardID should be unique number for each shard, 0 for your first, then 1, and so on. Then ShardCount should be the total number of shards you are running.

You can create multiple DiscordGo.Session instances in the same program or you could launch each shard as a separate program on the same or even different servers.

The Discord documentation has some details here as well that might be worth reading though.
https://discordapp.com/developers/docs/topics/gateway#sharding

Hopefully that helps some :)

[iopred]

The actual process of sharding is really easy, just set ShardID and ShardCount and your session will be sharded.

The trick comes if you want to use your bot like it was in one big Session, in that case you need to kinda strap a bunch of Sessions together, like I did! https://github.com/iopred/bruxism/blob/master/discord.go#L190

This actually works really well, I'm running this with 50 shards on a single (rather big) GCE instance.

If you want to do this properly-properly, you would just have each instance of your application talk to the same backend/database, but nothing in DiscordGo helps you write that, it's your responsibility.

[VetSoftKeith]

Ah, well I unfortunately?.. do not have a bot that's popular enough to need sharding :( If you join the Discord Gophers chat server there are a few people there that have made bots doing sharding that might be able to give you some advice.

The basis though, is you need to set Session.ShardID and Session.ShardCount before calling Session.Open(). ShardID should be unique number for each shard, 0 for your first, then 1, and so on. Then ShardCount should be the total number of shards you are running.

You can create multiple DiscordGo.Session instances in the same program or you could launch each shard as a separate program on the same or even different servers.

The Discord documentation has some details here as well that might be worth reading though.
https://discordapp.com/developers/docs/topics/gateway#sharding

Hopefully that helps some :)

So, I've migrated my bot over to DiscordGo from JDA and was kind of taken when autosharding didn't happen... My bot is in about 35 shards and auto sharding would be super helpful

[bwmarrin]

Understandable - there's always some differences between libraries. "Auto Sharding" wouldn't be within the design goals of this library though. And really, I'd say it shouldn't exist also because there's a number of ways a bot developer might want to do it for their bot. Maybe they want it all in one program, maybe they want to split up the load 50/50 between two separate programs on different AWS/GCE/whatever hosts, maybe they want to do something else entirely.

This library main design goal (for better or worse) is a low level fairly direct mapping of the api itself, with a little "magic" in the middle.

[servusdei2018]

For future reference, I implemented a simple sharding manager here for anyone who needs it: https://github.com/servusDei2018/shards

It has an example mirroring discordgo's pingpong: https://github.com/servusDei2018/shards/tree/master/examples/pingpong

I documented it well and, unlike some other discordgo sharding managers on GitHub, is documented, well-structured, and concurrency safe by default. Moreover, it supports downtime-free restarting/rescaling out of the box.

[fishuke]

Understandable - there's always some differences between libraries. "Auto Sharding" wouldn't be within the design goals of this library though. And really, I'd say it shouldn't exist also because there's a number of ways a bot developer might want to do it for their bot. Maybe they want it all in one program, maybe they want to split up the load 50/50 between two separate programs on different AWS/GCE/whatever hosts, maybe they want to do something else entirely.

This library main design goal (for better or worse) is a low level fairly direct mapping of the api itself, with a little "magic" in the middle.

This library is so popular and you should not adhere whatever you wanted earlier. When it comes to discord.js it has all features whatever people wanted. At least you need to let people make improvements about it. This is what open source is. You let people whatever they want to build.

[switchupcb]

Disgo has implemented a prioritized shard manager.

This comment is made for future reference.