What is the maximum size for read and write operations?

[ioquatix]

Thanks for your work on io_uring. I have been working on an integration with Ruby.

Looking at standard operations like read and write we see the following interface:

ssize_t read(int fildes, void *buf, size_t nbyte);
// error in errno

The interface in io_uring is this:

struct io_uring_sqe {
  // ...
  // The amount to read.
  __u32  len;
  // ...
};

struct io_uring_cqe {
  __s32  res;
  // if negative, contains error, if positive contains amount read.
};

It seems to me that the use of __s32 res implies a maximum of 2 GiB. However the limit on the read and write system calls would be up to the maximum value of size_t which is significantly higher.

As a point of comparison/reference, struct kevent uses intptr_t:

     struct kevent {
	     uintptr_t ident;	     /*	identifier for this event */
	     short     filter;	     /*	filter for event */
	     u_short   flags;	     /*	action flags for kqueue	*/
	     u_int     fflags;	     /*	filter flag value */
	     intptr_t  data;	     /*	filter data value */
	     void      *udata;	     /*	opaque user data identifier */
     };

The use case is a bit different but it stands to reason they could represent size_t in intptr_t.

1. What are the implications regarding the maximum amount of data that can be safely read (& because of similarity, written) in one operation?
2. Is this a problem in practice?
3. In a generic code base like Ruby how should we handle this? Should we put a cap on the maximum amount that can be read/written in one operation?

[axboe]

Linux imposes INT_MAX as the maximum transfer size, which is why io_uring uses the types that it does. read(2) and write(2) has the same limit, even if the interface itself takes a size_t. For Ruby, I guess capping it at 2G would be sensible and ensure that it matches what io_uring is able to do per request?

[axboe]

It's been like that for ~30 years, so I don't think a change is likely there...

[ioquatix]

Okay, that seems fair enough. The only other thing that I thought of is that apps that try to read, say 8GiB and only get back 2GiB, might think that the file is no bigger because of the partial read. Whether this is correct or not may depend on the app, but it seems like some apps might be broken in this regard if they assume a partial read means no more data is available (rather than doing a read and getting back 0 bytes).

[axboe]

But this is exactly how the Linux io interfaces already work, so they are either broken already or they already deal with this just fine?

[ioquatix]

It's entirely possible there is code that assumes a partial read from file indicates that it's got no further data. As the engineer between the "system" and the "user" we have to concern with poor understanding of how things work from the user PoV, and breaking compatibility is equally problematic from both system and user PoV. However in this case I think the risk is minimal.

[axboe]

I totally get your position, my point is just that this is the same in Linux whatever the interface you use. Hence io_uring doesn't present any differences in that particular area.

[axboe]

For Ruby, I guess capping it at 2G would be sensible and ensure that it matches what io_uring is able to do per request?

Or maybe, since it's the same regardless of what system call you use to transfer IO on Linux, you can just ignore it? If someone attempts a > 2GB IO request, it'll essentially just look like a short read/write to the application.

[ioquatix]

Yes, I think it's fair enough to do something like that.