Some Kafka headers are not displayed if type is not String

[gschmutz]

Hi

First, thanks a lot for this great tool!

When using the Dead-Letter topic support of Spring Kafka, then a message which is sent to the dead-letter topic includes the original-partition and the original-offset of the message as Kafka Headers. Partition is of type Integer and Offset of type Long and therefore they are not visible when showing the header details. The same is true for the original timestamp.

[tchiotludo]

Hi @gschmutz,

Thanks for the report, but this one is really complicated it seems.
For now, AKHQ don't have any mapping of type and you failed on the same issues than for the body, but for the headers.
Except manual configuration for the topic, I don't have any clues to handle that.
If you have an idea, it's welcome

[gschmutz]

@tchiotludo when creating the issue yesterday, I understood the difficulty for both key and value, as their types are provided by the Kafka producer client.

To be honest, until now (after checking the javadoc) I realized that the header value is a byte[] and not an Object. Which makes sense, so that it is Interoperable. That of course makes it impossible to handle the value without knowing how it has been serialized.

First I thought you could display a special value instead of empty but even that will not work because it was only the case because both offset and partition were single digits. I just found it confusing because first I thought the DLQ message did not contain values for these two headers.

Lessons learnt I guess is, that if using headers of type other than String makes it really hard for a consumer to handle in a generic way. Now I understand why all the visualization tools I have tried have the same problem with headers.

[tchiotludo]

👍 exactly!
Keep it open because AKHQ should not be filtered, and maybe displayed as special representation to understand unserialized case.

[gschmutz]

Assuming that the Strings are encoded iin UTF-8, I found this java code here to detect it: https://stackoverflow.com/questions/1193200/how-can-i-check-whether-a-byte-array-contains-a-unicode-string-in-java.

    static boolean looksLikeUTF8(byte[] utf8) throws UnsupportedEncodingException
    {
        Pattern p = Pattern.compile("\\A(\n" +
                "  [\\x09\\x0A\\x0D\\x20-\\x7E]             # ASCII\\n" +
                "| [\\xC2-\\xDF][\\x80-\\xBF]               # non-overlong 2-byte\n" +
                "|  \\xE0[\\xA0-\\xBF][\\x80-\\xBF]         # excluding overlongs\n" +
                "| [\\xE1-\\xEC\\xEE\\xEF][\\x80-\\xBF]{2}  # straight 3-byte\n" +
                "|  \\xED[\\x80-\\x9F][\\x80-\\xBF]         # excluding surrogates\n" +
                "|  \\xF0[\\x90-\\xBF][\\x80-\\xBF]{2}      # planes 1-3\n" +
                "| [\\xF1-\\xF3][\\x80-\\xBF]{3}            # planes 4-15\n" +
                "|  \\xF4[\\x80-\\x8F][\\x80-\\xBF]{2}      # plane 16\n" +
                ")*\\z", Pattern.COMMENTS);

        String phonyString = new String(utf8, "ISO-8859-1");
        return p.matcher(phonyString).matches();
    }

I have tested it and it works fine detecting Strings (True) and Long/Integer (False). I have also tested it as part of AKHQ and it looks good at least for my test.

Might want to do some more testing, but it looks promising.

I can also create a pull-request if you like, question is what kind of value would you want if it can not be converted to if it is not text?

[gschmutz]

BTW: I have also tested various other options such as StringUtils from commons-codec and Apache Tika but with less success.

[tchiotludo]

Everything that will allow us to better representation without hard configuration is better, so for sure, PR will be accepted ! 🚀

[gschmutz]

Have created the PR, header values of type Long, Short and Integer are now handled and displayed correctly.