[Feature idea] Introduce a FIFO queue with a max size to store data when the TCP socket write is unsuccessful

[ggrossetie]

The goal is to avoid "losing" data/logs when the TCP server is unavailable (for a short period of time). We should configure a max size to prevent unsafe unbounded storage (i.e. exhausting the memory).

To give you an idea of what I'm after, here's a naive implementation:

write (data, encoding, callback) {
  socket.write(data, encoding, (err) => {
    if (err) {
      // unable to write data, will try later when the server becomes available again
      queue.push({data, encoding})
    }
  })
  callback()
}

And in the reconnect/retry:

retry.on('ready', () => {
  connect((err) => {
    if (connected === false) {
      return retry.backoff(err)
    } else if (!queue.isEmpty()) {
      const items = queue.items()
      while (items.length > 0) {
        const { data, encoding } = items.shift()
        outputStream.write(data, encoding)
      }
    }
  })
})

[mcollina]

Thanks for reporting! Would you like to send a Pull Request to address this issue? Remember to add unit tests.

[ggrossetie]

Yes, I can send a pull request for this feature!

Do you have any preference regarding the FIFO queue implementation? Should we use an existing library? Should we use our own implementation to avoid adding a new dependency?

[mcollina]

I'll leave that to you and then review when a PR is open!