no_std support for ReadBytesExt/WriteBytesExt

[rjsberry]

This has already been discussed in part in #51, but discussion leant more towards integrating core-io into byteorder so I'm opening a new issue.

In their current form these traits extend the Read/Write traits from std::io. Unfortunately this leaves no_std developers stuck. no_std support for crates like bincode which are built on top of ReadBytesExt/WriteBytesExt is predicated on a no_std implementation in byteorder.

In my mind there are a number of different ways this could be achieved, but all are far from ideal.

1. Depend on a crate that aims to reduce std::io and provide Read/Write to no_std users.
2. Maintian Read/Write traits for no_std users in byteorder.
3. Expose ReadBytesExt/WriteBytesExt with no provided implementations without extending Read/Write for no_std users.
4. Wait for core::io.

I don't think 1 is a good idea as most of these crates are unmaintained/abandoned. I don't like 2 because these traits aren't exclusive to byteorder and don't belong inside the crate. 3 is okay as it would allow users that depend on ReadBytesExt/WriteBytesExt to write their own impls if they want to support no_std. 4 is great, but doesn't immediately solve the issue.

@BurntSushi I'm interested to hear your thoughts on whether you think this is a feasible upgrade to byteorder, and if you have any other ideas as to how this could be implemented while waiting for a potential core::io release.

[BurntSushi]

@rjsberry Thanks for bringing this issue up! I agree that the problem you're facing is something we should figure out how to solve, and I can definitely understand why it's a frustrating task.

Unfortunately, I don't really think this is byteorder's problem to solve. Saying that bincode's no_std support is predicated on byteorder's no_std support for things like ReadBytesExt/WriteBytesExt is kind of burying the lede. ReadBytesExt/WriteBytesExt are just convenience extension traits, and anyone using them could fairly easily stop using them if they wanted to. I think you can convince yourself of this by looking at the implementation of those extension traits. The actual problem is that if bincode does stop using those traits, what does it use instead? The most natural answer would be io::Read, but now you're back at square one. My ultimate point here is that byteorder's extension traits are ultimately a red herring, even though they are serving as the manifestation of your problem for the moment.

Expose ReadBytesExt/WriteBytesExt with no provided implementations without extending Read/Write for no_std users.

I don't think this works. Both of the extension traits specifically require io::Read implementations. Consequently, all downstream uses of these traits also require io::Read.

I don't think there is a simple answer here. I think the only path forward is for the no_std ecosystem to figure out how to represent I/O interfaces without the standard library. Alternatively, get bincode to provide APIs that don't use io::Read at all. However, now that I say that, I see that bincode already has such APIs for deserialization and serialization. So I'm left wondering why those aren't working for you. If it's because bincode internally uses byteorder's extension traits on things like &[u8], then the answer there is probably pretty straight forward: don't use the extension traits.

[rjsberry]

Thanks for the swift and thorough response! Really appreciate discussion on the topic.

Saying that bincode's no_std support is predicated on byteorder's no_std support [...]

Perhaps this wasn't the greatest way to frame my issue. This simply comes from an eagerness to get using both crates in embedded applications.

However, now that I say that, I see that bincode already has such APIs for deserialization and serialization. So I'm left wondering why those aren't working for you.

Indeed, under the hood bincode makes heavy use of Read/ReadBytesExt and Write/WriteBytesExt, so despite u8 slice APIs it cannot be used on constrained devices. A no_std compatible version of these extension traits would facilitate easy support for no_std in bincode which is why I originally used it to convey the problem. But, as you mentioned, it would be entirely possible to provide no_std support by not using the extension traits.

Bincode is simply an example, though. It would be great to see full support for all of byteorder for people working with no_std.

Unfortunately, I don't really think this is byteorder's problem to solve.

And as much as it hurts my case I tend to agree here. In your mind, is an option such as that raised in #51 out of the question (whether it's that specific crate, or another aiming to do the same)?

While I don't like this option as mentioned in my original comment, I don't see another way to expose the extension traits to no_std users while waiting on a refactor of std::io to be built on a core::io module such as in RFC 2262.

The issue could of course be dropped until it can be achieved without bringing in new dependencies.

[BurntSushi]

byteorder is a core library, and I don't really want it to grow experimental dependencies. I realize this is a chicken and egg problem, but ideally, any dependency that byteorder grows---especially a public dependency like core I/O traits---should go through a consensus driven design process, probably at the level of a Rust RFC. I looks like RFC 2262 is your best bet there.

I realize that bincode might just be an example, but if it is indeed the manifestation of your problem, then I think the path I would pursue is to rewire its internals to stop depending on std-only abstractions. If the author isn't amenable to it, then I'd probably fork it and just do it myself.

I really do not think the extension traits are that important. They are a tiny convenience in the grand scheme of things. The issue you're running into, I think, is that folks are depending on them without considering their no_std story.

But yeah, unfortunately this is a pretty hard problem to solve. Sorry. :-(

[rjsberry]

It sure is.

Despite their importance, they aren't supported in no_std! I'd definitely like to visit as RFC 2262 and/or others progress.

[BurntSushi]

I'm going to close this for now since I don't think there are any specific action items for byteorder in particular here.