Reverse operation for strings_to_byte

[xguerin]

Hello there,

I am trying to use Markup with Cohttp_lwt to do some HTML rewriting. I followed the example to get it to parse, but I am running into some issues to write a Markup stream into a Cohttp body due to the lack of reciprocity of the function strings_to_byte.

Here is what I am trying to to:

  let body = body
             |> Body.to_stream
             |> Markup_lwt.lwt_stream
             |> Markup.strings_to_bytes
             |> Markup.parse_html
             |> Markup.signals
             |> Markup.write_html
             |> Markup_lwt.to_lwt_stream
             |> Body.of_stream

Unfortunately, the last operation is not compatible because to_lwt_stream returns a (char, 'a) stream while of_stream expects a (string, 'a) stream. I could not find a reciprocal version of strings_to_byte to hopefully do the job. Am I looking at the right place ?

Thanks,

[xguerin]

Actually, reading the code it looks like Htlml_writer and Xml_writer already return string streams, and write_html and write_xml transform that stream into a char stream.

EDIT

I locally added write_* functions that return a (string, 'a) stream and ended up with interesting EPIPE errors. As a workaround, I am passing through a string instead of a stream and it works.

[aantron]

Thanks, I'll take a look. This seems like it would be helped by addressing #10.

[aantron]

@xguerin, do you know what was causing the EPIPE signals?

[xguerin]

Yes, #10 would definitely be the answer (this is basically what I did locally). Regarding EPIPE, I am still investigating. But I think there may be an underlying problem. If I don't use streams but pure strings, at some points I would get truncated/incomplete data (the most obvious consequence of that is that the web browser hangs on loading the page). However, if I skip markup's parsing/writing phase, all is well (so it's not a Lwt/Cohttp issue).

[aantron]

Would you want to submit your patch as a PR? Otherwise let me know, I'll do it :)

I'm not sure still if you mean the second part is a bug in Markup.ml, nor sure myself if it would be. My guess would be no, because Markup.ml doesn't do its own I/O, but it could be triggering some kind of I/O pattern that it shouldn't trigger.

[xguerin]

Yes definitely. Re: the other issue, I'll look into it and let you know.

[xguerin]

More info on the choke/EPIPE: I am not able to reproduce it when reading from file and writing to file or reading from a socket (Cohttp_lwt_unix.Client) and writing to file. So it looks like there is something amok with the server socket on the return path. More to come.

[aantron]

Thanks for continuing to look into it.