Make a clear distrinction between streaming and package based connectors

[Licenser]

Describe the problem you are trying to solve

Some of our connectors are package based where one chunk of data == one event, udp, kafka, HTTP, WS are some examples.

Others are stream based where there is no alignment to event boundaries, TCP and file are examples. Those usually require a preprocessor and postprocessor to add framing.

We do not make it clear in the documentation which connector behaves which way and has which guarantees, that can lead to confusion.

We should be clearer about this.

[afzal442]

Hi @Licenser, I want to look into it!

[skoech]

Hello @afzal442! Welcome to Tremor!
We are happy you want to contribute. How would you like to go about this?

[afzal442]

Thanks @skoech! I wondered which doc's part for this issue needs to be fixed.

[skoech]

We would need to update the docs of all the connectors (onramps/offramps).

[skoech]

So, TCP is stream-based while the rest are all packet-based.

[afzal442]

Makes sense. I found those links as https://github.com/tremor-rs/tremor-www-docs/blob/main/docs/artefacts/offramps&onramps.md . Any next suggestions?

[skoech]

Yes, good. :)
Then maybe create your own fork to work on, if you haven't yet?

[afzal442]

Hy @skoech, your discord handle plz?

[skoech]

I'm shay#4695. I already reached out to you. Sorry I didn't mention it was me. 😀