Wishlist for SDL3_net

[icculus]

To be clear, this isn't something I plan to work on any time soon, but it would be nice if we had an idea of what to aim towards if we're finally breaking API/ABI for SDL_net.

SDL_net is not my favorite thing in the world, but I'm willing to be optimistic and say with the correct changes, I could certainly learn to love it.

Here are some wishlist items.

• IPv6 support, but maybe better said...
• ...any network support, really, since this can be abstracted, I think.
• Resolved addresses live in an opaque structure and there will be a function to return a somewhat-human-readable string (like 127.0.0.1 or 2259:1700:2760:bab8:2a20:562b:e796:143d).
• Simplify what's there: remove the select()-like API (SDLNet_SocketSet), and just accept an array of sockets to check for available data during SDLNet_CheckSockets(). Or just make everything non-blocking...
• Make everything non-blocking. Spin a thread for DNS lookups, set O_NONBLOCK on socket handles, etc.
• Split SDLNet_TCP_Open into client and server operations (one becomes a connect() call, the other becomes a listen() and bind()).
• The "server socket" (something you can accept() connections on) is confusing, I'd replace it with some sort of abstraction for answering connections that doesn't use the same TCPsocket type externally (even if it's just the same sort of file descriptor internally)
• Replace references to "TCP" and "UDP" with "Stream" and "Datagram" (or "Reliable" and "Unreliable" or whatever).
• "Binding" a socket to a "channel" is pretty confusing, both because it confuses the lower-level concept of "binding" a datagram socket to a port number, and it makes it look like a socket needs to bind to a single "channel," which will cause people to try to create multiple sockets on the same port for different channels.
• I kind of want to dump all the explicit allocation of UDP packets, and instead just have the send/recv functions handle this internally. The app just provides a buffer to send or a buffer to receive data and be done with it.
• Remove SDL_FORCE_INLINE functions and macros from SDL_net.h, make them real functions in the library.
• Add a way to convert a TCP stream into an SDL_RWops...? That might be stupid, idk.
• Someone asked for built-in encryption, but adding an OpenSSL or whatever dependency is a huge hassle, doubly-so if that may or may not be available in the build on the other end of the connection.

90% of the reason SDL_net exists at all was to manage differences between BSD Sockets, Winsock, and a few other wierdo platforms, but BSD Sockets won out, modulo a few WinSock differences that can be papered over with a handful of #defines, so that need doesn't really exist any more.

So the only reason for an SDL3_net to exist, if we're going to do an SDL3_net, is to make it easier to do these common socket things from a programming point of view, or add features that are less trivial to slot into an app, like encryption or NAT hole punching. There really isn't anything that integrates with SDL3 that needs to be updated from an SDL2 interface, etc.

All I'm talking about here is how to streamline what's there to be a better API, but the question still lingers: is this worth doing? Or should we retire this with SDL2 and tell people to use BSD Sockets?

[sezero]

+1 for retire

[slouken]

Yeah, I think retiring the library at this point is the right call. I don't mind if someone wants to come along and pick it up, but I don't think there's enough value from a simple sockets wrapper to have a whole library based around it at this point.

[icculus]

Just for the intellectual exercise, this is what I would probably make a revised API look like. It's a little easier to use than BSD Sockets but more or less the same level of usefulness, never blocks (except a few WaitFor* functions), and does most of the heavy lifting of memory management.

(This is obsolete and bulky, so I've edited this comment to hide this, but click if you want to expand.)

/* Version checks... */

#define SDL_NET_MAJOR_VERSION   3
#define SDL_NET_MINOR_VERSION   0
#define SDL_NET_PATCHLEVEL      0

#define SDL_NET_VERSION(X)                          \
{                                                   \
    (X)->major = SDL_NET_MAJOR_VERSION;             \
    (X)->minor = SDL_NET_MINOR_VERSION;             \
    (X)->patch = SDL_NET_PATCHLEVEL;                \
}

extern DECLSPEC const SDL_version * SDLCALL SDLNet_Linked_Version(void);


/* must call first/last... */

extern DECLSPEC int SDLCALL SDLNet_Init(void);
extern DECLSPEC void SDLCALL SDLNet_Quit(void);

/* hostname resolution API... */

extern DECLSPEC SDLNet_Address * SDLCALL SDLNet_ResolveHostname(const char *host);  /* does not block! */
extern DECLSPEC void SDLCALL SDLNet_WaitForResolution(SDLNet_Address *address);  /* blocks until success or failure. */
extern DECLSPEC int SDLCALL SDLNet_GetAddressStatus(SDLNet_Address *address);  /* -1: still working, 0: failed, 1: ready */
extern DECLSPEC const char * SDLCALL SDLNet_GetAddressString(SDLNet_Address *address);  /* human-readable string, like "127.0.0.1" or "::1" or whatever. NULL if GetAddressStatus != 1 */
extern DECLSPEC void SDLCALL SDLNet_RefAddress(SDLNet_Address *address);  /* +1 refcount; SDLNet_ResolveHost starts at 1. */
extern DECLSPEC void SDLCALL SDLNet_UnrefAddress(SDLNet_Address *address);  /* when totally unref'd, gets freed. */

extern DECLSPEC SDLNet_Address **SDLCALL SDLNet_GetLocalAddresses(int *num_addresses);  /* returns NULL-terminated array of SDLNet_Address*, of all known interfaces. */
extern DECLSPEC void SDLCALL SDLNet_FreeLocalAddresses(SDLNet_Address **addresses);  /* unrefs each address, frees array. */

/* Streaming (TCP) API... */

typedef struct SDLNet_StreamSocket SDLNet_StreamSocket;  /* a TCP socket. Reliable transmission, with the usual pros/cons */

/* Clients connect to servers, and then send/receive data on a stream socket. */
extern DECLSPEC SDLNet_StreamSocket * SDLCALL SDLNet_CreateClient(SDLNet_Address *address, Uint16 port);  /* Start connection to address:port. does not block! */
extern DECLSPEC void SDLCALL SDLNet_WaitForClientConnection(SDLNet_StreamSocket *sock);  /* blocks until success or failure */

/* Servers listen for and accept connections from clients, and then send/receive data on a stream socket. */
typedef struct SDLNet_Server SDLNet_Server;   /* a listen socket internally. Binds to a port, accepts connections. */
extern DECLSPEC SDLNet_Server * SDLCALL SDLNet_CreateServer(SDLNet_Address *addr, Uint16 port);  /* Specify NULL for any/all interfaces, or something from GetLocalAddresses */
extern DECLSPEC int SDLCALL SDLNet_GetServerStatus(SDLNet_Server *server);  /* 0: nothing available at the moment, 1: connection ready to accept */
extern DECLSPEC SDLNet_StreamSocket * SDLCALL SDLNet_AcceptClient(SDLNet_Server *server);  /* NULL: nothing available at the moment */
extern DECLSPEC void SDLCALL SDLNet_DestroyServer(SDLNet_Server *server);

/* Use a connected socket, whether from a client or server. */
extern DECLSPEC SDLNet_Address * SDLNet_GetStreamSocketAddress(SDLNet_StreamSocket *sock);  /* Get the address of the other side of the connection */
extern DECLSPEC int SDLCALL SDLNet_GetConnectionStatus(SDLNet_StreamSocket *sock);  /* -1: connecting, 0: failed/dropped, 1: okay */
extern DECLSPEC int SDLNet_WriteToStreamSocket(SDLNet_StreamSocket *sock, const void *buf, int buflen);  /* always queues what it can't send immediately. Does not block, -1 on out of memory, dead socket, etc */
extern DECLSPEC int SDLNet_ReadStreamSocket(SDLNet_StreamSocket *sock, void *buf, int buflen);  /* read up to buflen bytes. Does not block, -1 on dead socket, etc, 0 if no data available. */
extern DECLSPEC void SDLNet_SimulateStreamPacketLoss(SDLNet_DatagramSocket *sock, int percent_loss);  /* since streams are reliable, this holds back data for some amount of time. */
extern DECLSPEC void SDLNet_CloseStreamSocket(SDLNet_StreamSocket *sock);  /* Close your sockets when finished with them. Does not block, handles shutdown internally. */

/* Datagram (UDP) API... */

typedef struct SDLNet_DatagramSocket SDLNet_DatagramSocket;  /* a UDP socket. Unreliable, packet-based transmission, with the usual pros/cons */

typedef struct SDLNet_Datagram
{
    SDLNet_Address *fromaddr;  /* this is unref'd by SDLNet_FreeDatagram. You only need to ref it if you want to keep it. */
    Uint16 fromport;  /* these do not have to come from the same port the receiver is bound to. */
    Uint8 *buf;
    int buflen;
    int channel;
} SDLNet_Datagram;

extern DECLSPEC int SDLNet_GetMaxDatagramSize(void);  /* Probably just hardcode to 1500? */
extern DECLSPEC SDLNet_DatagramSocket * SDLCALL SDLNet_CreateDatagramSocket(Uint16 port);  /* always binds to ANY address. */
extern DECLSPEC int SDLCALL SDLNet_SendDatagram(SDLNet_DatagramSocket *sock, int channel, SDLNet_Address *address, const void *buf, int buflen);  /* always queues what it can't send immediately. Does not block, -1 on out of memory, dead socket, etc. Fails immediately if > MTU size! */
extern DECLSPEC SDLNet_Datagram * SDLCALL SDLNet_ReceiveDatagram(SDLNet_DatagramSocket *sock);  /* Get next available packet. Does not block, NULL if none available. */
extern DECLSPEC void SDLCALL SDLNet_FreeDatagram(SDLNet_Datagram *dgram);  /* call this on return value from SDLNet_ReceiveDatagram when you're done with it. */
extern DECLSPEC void SDLNet_SimulateDatagramPacketLoss(SDLNet_DatagramSocket *sock, int percent_loss);
extern DECLSPEC void SDLNet_CloseDatagramSocket(SDLNet_DatagramSocket *sock);  /* Close your sockets when finished with them. Does not block. */

[icculus]

Ok, as a weekend project, I knocked out something sort of like what I proposed above, and stuck it in a pull request.

The question is, I think, "does SDL_net provide value in a world where everything is BSD sockets anyway?" and having done the intellectual exercise here, the answer is--to my surprise--actually yes.

• BSD Sockets are kinda a pain to use directly and have several parts that are confusing. A much cut-down API, like I'm proposing here, provides the vast majority of what you would ever want in a significantly simplified form.
• Even writing this library as a simplified wrapper over BSD sockets, which I am familiar with and have programmed to before, I was constantly going back to the manpages, looking up obscure What Ifs on stackoverflow, etc, so telling people "just use BSD sockets" may be a disservice, even if they're largely portable.
• This hides the WinSock/BSD differences; although not major, they are things developers need to think about.
• This library offers asynchronous DNS lookups, BSD sockets doesn't (outside of the "getaddrinfo_a" GNU extension). It keeps two worker threads for DNS lookups, and can spin up to ten threads to manage requests, dynamically growing the thread pool as necessary to handle the backlog, and shrinking it back down when the request load drops.
• Even in a pure socket library, it was powerful to be able to count on SDL3 for the C runtime, for thread management and locking, etc.

Where I landed was a simple API that is non-blocking by default, since this is what game developers need, and has a few optional functions to wait for sockets to connect/get more incoming data/write their pending buffers, since we can put these to sleep at the OS level until events occur, instead polling+SDL_Delay in a loop, for the times when that might be appropriate.

Data to be sent queues internally when necessary, so if you want to write more than the socket can eat at the moment, we'll take care of it for you and let it drain through over time.

I've also built in ways to say "pretend this is failing a little (or a lot)" which will throw away UDP packets as if the network lost them, stall stream writes and DNS lookups, maybe drop connections, etc, so you can see how your game fares when it isn't running on your wired connection to a gigabit fiber line, and instead smacks into the real world of flakey wifi connections, etc.

Backwards compat with SDL2_net is not at all on the table. That being said, I assume SDL2_net will compile against SDL3 (or can be made to work relatively quickly) because there isn't much about it that requires SDL at all. But if we're doing this, I'm leaving it in the dust, and not building an SDL2_net-compat library.

This is at the "it works on my Linux machine" state. I have a few Windows #ifdefs filled in, but we would need to compile and test this elsewhere.

If we want to get more ambitious in the future, a zero-dependency library that offers NAT punching, encryption, WebSockets and WebRTC on top of these primitives would make this an absolutely must-have piece of tech, but that would be a massive undertaking and I'm mostly concerned with offering a humble library that's a modern improvement over SDL2_net at the moment.

[icculus]

Okay, this is in!

The Wiki has been split into SDL2_net and SDL3_net.

The previous code still lives in an "SDL2" branch.