Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support chunked transfer encoding #44

Open
Dominaezzz opened this issue Jun 21, 2024 · 7 comments
Open

Support chunked transfer encoding #44

Dominaezzz opened this issue Jun 21, 2024 · 7 comments

Comments

@Dominaezzz
Copy link

This allows either client or server send requests/responses without having to specify a Content-Length upfront.
This works by specifying a Transfer-Encoding of chunked, then the sender can send chunks of data and terminate the transfer by sending an empty chunk.
I'm particularly interested in chunked responses.

My specific use case for this is streaming JPEGs from an ESP32-S3.
When the ESP32-S3 is pulling JPEGs from a camera, it (usually) does not know the size of the image until the entire image is received into memory. Receiving the entire image into memory creates two problems; The first is latency, the image data should ideally be sent out ASAP after it's gotten from the camera. The second is image size, if the quality and width/height are large enough, the chip will be unable to fit the image in memory, which means it simply can't stream the image. Ideally the chip should be able to send out the image data ASAP to reduce the memory requirements.

I'm not sure what this should look like yet. Perhaps some kind of ChunkWriter I guess but I'm not sure how the terminating chunk should be handled yet.

(This library is fantastic btw! Thanks for building iand sharing it!)

@sammhicks
Copy link
Owner

I've written a draft implementation (see 8033a31), what do you think?

@Dominaezzz
Copy link
Author

It looks great! It's more/less what I imagined. Thanks for throwing an implementation together so quickly.

My only comment is that it'd be convenient if the ChunkWriter itself also implemented the Write trait, and that way I could use the write! macro with it.

@sammhicks
Copy link
Owner

I've implemented write_fmt for ChunkWriter, which means that you'll be able to use the write macro, even though technically ChunkWriter doesn't implement Write. Does that meet your request?

f66c158#diff-6186c6610b78b8b16d8526d10c5a2cfe8882b95b873e7a36a989de3e7b581599R34

@Dominaezzz
Copy link
Author

Yes that's perfect thanks!

@Dominaezzz
Copy link
Author

I've just used this in my application and it works like a charm. Last thing I'll ask for is a flush() method to ensure all previous chunks have been sent.

@sammhicks
Copy link
Owner

When you call ChunkWriter::finalize, the writer is flushed, other than that you shouldn't need to worry about flushing the buffers.
What scenario would you like to flush the buffers partway through writing a response?

@Dominaezzz
Copy link
Author

In my case I'm doing both chunked encoding and mixed-replace. So I have an infinite chunked steam, and I want to flush after every replacement (a complete jpeg).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants