Suggestion: provide higher-level read/write functions

[craigfe]

Conduit currently provides relatively few operations for interacting with flows, expecting the user to write them themselves. We have recv and send, but no functions like:

val send_string : string -> (unit, [> error ]) result io
val send_line   : string -> (unit, [> error ]) result io

val receive_line  :         flow -> (string or_end_of_flow, [> error ]) result io
val receive_up_to : char -> flow -> (string or_end_of_flow, [> error ]) result io

This is perhaps because buffered read/write operations are provided by mirage-channel as an abstraction over Conduit (with an API that encourages fewer buffer allocations). However, I think it'd be helpful for Conduit to provide these utility functions in order to lower the barrier to entry, even if they are not going to be optimally efficient for all use-cases.

As a motivation, the beginning of the Conduit tutorial currently assumes that the user has defined a getline function (c.f. #348) because this is a natural starting point for understanding & using Conduit; I think it'd be better to just provide this function outright, and let the user switch to a more efficient buffering strategy when (if?) they need one. I suspect that most users of Conduit would require the same boilerplate here anyway – how often does one want a completely unbuffered flow?

If we decide not to take this approach, I think it'd be good to state this as an explicit non-goal of Conduit in the documentation (e.g. "If you want high-level / buffered operations, look elsewhere."). @dinosaure seems to suggest this in #339 (comment), but it'd be nice to have a concrete stance on this 🙂

(CC @talex5, since you mentioned getline in your API feedback.)

[samoht]

I think it'd be nice if conduit could indeed provide these basics functions out-of-the box. Whatever would make conduit more useful it useful :-) Especially if it helps writing simpler tutorials.

But I think there will be a new design question to solve then: as we have a way to exposing the underlying concrete flow, that would also mean that we will have to expose the underlying buffers too ; or maybe it's ok to discard the buffers in that case, similar to Unix.descr_of_in_channel.

[dinosaure]

For my perspective, such functions need something more complex than just the Conduit.flow. About receive_line, you must have a ring-buffer (as mirage-channel does or as many of my glues do in many places). I don't have strong opinion about that and I agree that it will be easier for the end-user.

A possible solution is to systematically associate a ring-buffer to a Conduit.flow - and be able to implement these functions. I think, on this way, it will not change extraction operations - so if the user wants the real value, we can return it without the ring-buffer. To hide the ring-buffer (and be accessible internally), you probably should to extend this type (however, you should replicate this code of E0 which is common between the client and the service - and the service should not keep a ring-buffer ...):

ocaml-conduit/src/core/e0.ml

Line 101 in f2cc071

type v = Value : 'a * 'a Key.t -> v

An other possibility, as mirage-channel is the functor way - but I think it's not a good idea.