How do you get the semantics of read without offset?

[ioquatix]

There are two main system calls for reading from a file:

     ssize_t
     pread(int d, void *buf, size_t nbyte, off_t offset);

     ssize_t
     read(int fildes, void *buf, size_t nbyte);

It seems that

void io_uring_prep_read(struct io_uring_sqe *sqe, int fd, void *buf, unsigned nbytes, off_t offset)

is closer to pread than it is to read. How does one get the read semantics? i.e. incrementing the read offset, and reading from the last offset? Do I need to call seek after every read?

[ioquatix]

I investigated some existing systems and it seems like supplying offset = -1 works. However, I can't find where this is documented. It would make me feel better if this was documented somewhere.

[axboe]

If you check the man page for io_uring_enter.2, and look at the IORING_OP_{READ,WRITE} section, it states:

If offs is set to -1, the offset will use (and advance) the file position, like the read(2) and write(2) system calls.

So you can safely rely on this behavior.

[axboe]

I didn't write nor do I control that web page, so really outside the scope of liburing or the io_uring project.

[ioquatix]

Oh okay.

So, I tried using -1 as the offset and I get errno 29, illegal seek.

IORING_OP_READ
IORING_OP_WRITE
Issue the equivalent of a pread(2) or pwrite(2) system call. fd is the file 
descriptor to be operated on, addr contains the buffer in question, len 
contains the length of the IO operation, and offs contains the read or write 
offset. If fd does not refer to a seekable file, off must be set to zero. If 
offs is set to -1, the offset will use (and advance) the file position, like 
the read(2) and write(2) system calls. These are non-vectored versions of the 
IORING_OP_READV and IORING_OP_WRITEV opcodes. See also read(2) and write(2) for 
the general description of the related system call. Available since 5.6.

I'm using a unix socket for testing. So, how do I emulate read/write in the general sense, since I wouldn't know whether to supply 0 or -1 as the offset to get the same behaviour as read/write without knowing whether it was seekable or not.

[axboe]

Sockets (and pipes, for example) aren't seekable.

[ioquatix]

Right, but how do you get read/write like semantics without knowing it's seekable or not? The Ruby interpreter doesn't know if a given file descriptor is seekable, so I don't know how to implement the same semantics as the read system call which seems to determine internally how seeking should work (or not).

[ioquatix]

Basically, how does one wrap read like semantics, given the read(fd, buffer, amount) into IORING_OP_READ without knowing whether the given fd is seekable or not?

ssize_t read(int fd, void *buffer, size_t amount) {
  off_t offset;

  // How to know this?
  if (seekable(fd)) offset = -1;
  else offset = 0;

  io_uring_prep_read(..., offset):
}

[axboe]

It behaves like the regular system calls in this regard. But a socket is a stream, and for streams io_uring passes in NULL ppos regardless of what you set the value to. So actually a bit puzzled, do you have a test case?

You could obviously check the file type, but seems wasteful to need to do a stat type call before a read if it isn't needed.

Honestly, not a huge fan of current file position, it doesn't make a lot of sense of an async interface. Consider what happens if you queue two reads with -1 offset - they end up reading from the same offset, as the position isn't updated until one of them completes. If at all possible, I'd highly recommend that Ruby use the explicit offsets rather than rely on the internal file position.

[ioquatix]

I'm using a unix domain socket. Does it make any difference?

With this, it works correctly. Without it, it fails.

[ioquatix]

Honestly, not a huge fan of current file position, it doesn't make a lot of sense of an async interface. Consider what happens if you queue two reads with -1 offset - they end up reading from the same offset, as the position isn't updated until one of them completes. If at all possible, I'd highly recommend that Ruby use the explicit offsets rather than rely on the internal file position.

I agree with generally but I don't have any way to change the existing interfaces. In actual fact, most users would be reading from different files, so there is value in an async interface, provided people don't try to read from the same file in a concurrent way. Of course you are right regarding the behaviour of two simultaneous reads - even without io_uring I could imagine this will be a race condition.

[axboe]

Let me take a look tomorrow why it fails, it's eluding me right now.

[ioquatix]

I'll try to create a small test case in C if that helps.

[ioquatix]

For reference, here is the code which is creating the unix socket pair I'm using for testing: https://github.com/ruby/ruby/blob/d92f09a5eea009fa28cd046e9d0eb698e3d94c5c/ext/socket/unixsocket.c#L550-L568

Maybe the protocol family or some other detail is being overlooked as "not a socket"?

[axboe]

This should probably fix it:

diff --git a/fs/io_uring.c b/fs/io_uring.c
index d5ab0e9a3f29..fb2a0cb4aaf8 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2891,9 +2891,13 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 		req->flags |= io_file_get_flags(file) << REQ_F_SUPPORT_NOWAIT_BIT;
 
 	kiocb->ki_pos = READ_ONCE(sqe->off);
-	if (kiocb->ki_pos == -1 && !(file->f_mode & FMODE_STREAM)) {
-		req->flags |= REQ_F_CUR_POS;
-		kiocb->ki_pos = file->f_pos;
+	if (kiocb->ki_pos == -1) {
+		if (!(file->f_mode & FMODE_STREAM)) {
+			req->flags |= REQ_F_CUR_POS;
+			kiocb->ki_pos = file->f_pos;
+		} else {
+			kiocb->ki_pos = 0;
+		}
 	}
 	kiocb->ki_flags = iocb_flags(file);
 	ret = kiocb_set_rw_flags(kiocb, READ_ONCE(sqe->rw_flags));

[ioquatix]

I'm guessing kiocb->ki_pos == -1 is what's causing the invalid seek?

[axboe]

https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.16&id=bfa4e083655ab62f3d8bcb82c9b977204113c8e1

Can you give me your email so I can update the commit in the reported-by?

And thanks for reporting this, I can't believe that nobody has ran into this before. I guess nobody really uses the offset == -1 feature in a generic fashion. Works fine for files, but obviously fails for streams IFF they don't use the older style read/write internal kernel interface. Things like pipes do use that, which probably explains why most people haven't seen this.

[ioquatix]

Please use [email protected] for the email address.

[axboe]

Right, for the newer ->read_iter() type interface in the kernel, we pass in the full kiocb structure. This means that they then see offset != 0, which obviously makes them unhappy for a stream.

[axboe]

I'll get this pushed out for 5.16-rc7, and it should percolate back to stable shortly thereafter and be generally available shortly.

[axboe]

https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.16&id=7b9762a5e8837b92a027d58d396a9d27f6440c36

Now updated with a bit more explanation, and your email. Thanks!!

[axboe]

And sent out for review as well, you are on the CC.

[ioquatix]

Thanks.