Deserialising data

[mohit-zs]

Hi team, I want to understand the working of Unmarshal. I have two similar Go structs Cat and Dog but with different avro tags.

type Cat struct {
	IsCat bool `avro:"isCat" json:"isCat"`
}

type Dog struct {
	IsDog bool `avro:"isDog" json:"isDog"`
}

If I am not wrong then unmarshalling a Cat serialised message into a Dog obj should return me an error.

func main() {
	catSchema, _ := avro.Parse(
		`{
			  "type": "record",
			  "name": "cat",
			  "fields": [
				{
				  "name": "isCat",
				  "type": "boolean"
				}
			  ]
			}`)

	dogSchema, _ := avro.Parse(
		`{
			  "type": "record",
			  "name": "dog",
			  "fields": [
				{
				  "name": "isDog",
				  "type": "boolean"
				}
			  ]
			}`)

	catObj := Cat{IsCat: true}
	dogObj := Dog{}
	serialisedCat, _ := avro.Marshal(catSchema, catObj)
	errUnmarshal := avro.Unmarshal(dogSchema, serialisedCat, &dogObj)
	fmt.Println(errUnmarshal)
	fmt.Println(dogObj.IsDog)
}

Like here errUnmarshal should not be non nil, but I am not getting any error. Please help me understand this behaviour. Or more like how to identify the kafka messages if they are of Dog type or Cat type

[nrwiersma]

No, it will not error, as it is a common use case to only unmarshal part of a schema in to a struct. As you must provide the schema and object to unmarshal the data, it is assumed you have paired the 2 correctly.

Further it should be noted that with Avro there is no way from the data itself to know if the schema and data match, this must be tracked by the user. The most common way of doing this on Kafka is to use a Confluent Schema Registry and their wire format.

[nrwiersma]

Because you have no tags that match the fields, so it skips those fields, only reading the fields that have matching tags.

[mohit-zs]

So if I am correct, by skipping you mean if the avro tag is not matched while deserialising then that field should be left to default value. In the above example, the dogObj should have IsDog field set to false then, rt?
But it is being assigned the value true.

[nrwiersma]

In the case you have, this is expected, as you are using the correct schema for the object, so the field is matches. I had thought you had mixed the schema and objects, which you have not. What you are doing is perfectly valid.

[mohit-zs]

I still did not get it. Are trying to say that, while Unmarshal, assigning value to a different avro tag is expected if the data type is same?

[nrwiersma]

No. I misread the example, I had thought it was this:

errUnmarshal := avro.Unmarshal(catSchema, serialisedCat, &dogObj)

This would leave the bool value as false as the tags do not match. Instead it is this:

errUnmarshal := avro.Unmarshal(dogSchema, serialisedCat, &dogObj)

Here the tags in dogObj matches the schema dogSchema and so it is deserialised normally. There is no error/skip case here.