Log custom fields inside message router

[fetinin]

I would like to implement a way to inject a custom logger or logfields through Message.Context. Let me begin by addressing a problem we encountered.

The issue we faced is that logs inside the message handler are not very useful in practice. For example, when a handler returns an error this line produces something like:

9:49AM ERR Handler returned error error="some error" message_uuid= 
9:49AM ERR Handler returned error error="some other error" message_uuid=

Which topic or handler caused this error? What other logs are related to this error? I would like to include additional information such as Kafka partition, topic, offset and trace-id.
Currently, we address this issue through a custom middleware that logs handler errors as needed. However, this seems like a duck tape since errors are now logged twice - within our middleware and within the Watermill router. We had to disable logging within Watermill to prevent duplicate logs, which is not ideal.

We require a method to customize the handler logger based on the message it is handling. I propose introducing two functions within Watermill: LogFieldsFromCtx and LogFieldsToCtx. LogFieldsToCtx will be utilized in PubSub implementations like Watermill-Kafka to populate Message.Context with fields useful for logging, as demonstrated here. The message router logger would then get this fields by calling LogFieldsFromCtx(msg.Context) and incorporate them into all its logs.

@roblaszczak, @m110, I would appreciate your input. Is this a viable solution? Are there alternative approaches to consider?

[m110]

Hey @fetinin, I think custom middleware makes a lot of sense in a case like this. Are duplicate logs a deal breaker for you? They're, after all, a bit different. One is a low-level Watermill log, and the other is your custom log, where you can include as many details as you want.