DB::Exception: Invalid version for SerializationLowCardinality key colum

[mshustov]

I have no rights to transfer issues, so I copied the content from ClickHouse/ClickHouse#63779:

I'm using 24.1, but looks like a problem on all builds

Create schema like so

SET flatten_nested=0;
CREATE TABLE IF NOT EXISTS default.test_table
(
    `Id` UInt8,
    `Device` LowCardinality(String),
    `Nestme` Nested(
    	        `Id` UInt32,
		`TestLC` LowCardinality(String),
		`Test` String
	)
)
ENGINE = MergeTree
ORDER BY Id
SETTINGS index_granularity = 8192;

I'm using the Go library to do batch inserts : https://github.com/ClickHouse/clickhouse-go

batch, err := DB.PrepareBatch(context.Background(), "INSERT INTO test_table")
if err != nil {
	log.Error(err.Error())
}
if err := batch.Append(uint8(1), "pc", []any{[]any{1, "test LC", "test"}, []any{2, "test LC", "test"}}); err != nil {
       log.Error(err.Error())
}
if err := batch.Send(); err != nil {
	log.Error(err.Error())
}

The lib reports error comes back with

code: 117, message: Invalid version for SerializationLowCardinality key column.

The debug logs on server show:

2024-05-14 13:29:09 2024.05.14 12:29:09.857733 [ 827 ] {964438d5-21af-4266-a341-05291d51bdba} <Error> TCPHandler: Code: 117. DB::Exception: Invalid version for SerializationLowCardinality key column. (INCORRECT_DATA), Stack trace (when copying this message, always include the lines below):
2024-05-14 13:29:09 
2024-05-14 13:29:09 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c804c9b in /usr/bin/clickhouse
2024-05-14 13:29:09 1. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000721d043 in /usr/bin/clickhouse
2024-05-14 13:29:09 2. DB::SerializationLowCardinality::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x00000000108d7710 in /usr/bin/clickhouse
2024-05-14 13:29:09 3. DB::SerializationArray::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x00000000108bc127 in /usr/bin/clickhouse
2024-05-14 13:29:09 4. DB::SerializationTuple::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x000000001090609c in /usr/bin/clickhouse
2024-05-14 13:29:09 5. DB::SerializationArray::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x00000000108bc127 in /usr/bin/clickhouse
2024-05-14 13:29:09 6. DB::SerializationTuple::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x000000001090609c in /usr/bin/clickhouse
2024-05-14 13:29:09 7. DB::SerializationArray::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&) const @ 0x00000000108bc127 in /usr/bin/clickhouse
2024-05-14 13:29:09 8. DB::NativeReader::read() @ 0x0000000012976313 in /usr/bin/clickhouse
2024-05-14 13:29:09 9. DB::TCPHandler::receivePacket() @ 0x0000000012947afd in /usr/bin/clickhouse
2024-05-14 13:29:09 10. DB::TCPHandler::readDataNext() @ 0x000000001294aa44 in /usr/bin/clickhouse
2024-05-14 13:29:09 11. DB::TCPHandler::runImpl() @ 0x000000001293f979 in /usr/bin/clickhouse
2024-05-14 13:29:09 12. DB::TCPHandler::run() @ 0x0000000012952e79 in /usr/bin/clickhouse
2024-05-14 13:29:09 13. Poco::Net::TCPServerConnection::start() @ 0x00000000153cb412 in /usr/bin/clickhouse
2024-05-14 13:29:09 14. Poco::Net::TCPServerDispatcher::run() @ 0x00000000153cc211 in /usr/bin/clickhouse
2024-05-14 13:29:09 15. Poco::PooledThread::run() @ 0x00000000154c4f27 in /usr/bin/clickhouse
2024-05-14 13:29:09 16. Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000154c355d in /usr/bin/clickhouse
2024-05-14 13:29:09 17. ? @ 0x00007f79013db609
2024-05-14 13:29:09 18. ? @ 0x00007f7901300353

If I change schema to this:

SET flatten_nested=0;
CREATE TABLE IF NOT EXISTS default.test_table
(
    `Id` UInt8,
    `Device` LowCardinality(String),
    `Nestme` Nested(
    	`Id` UInt32,
		`TestLC` String,
		`Test` String
	)
)
ENGINE = MergeTree
ORDER BY Id
SETTINGS index_granularity = 8192;

Then the insert works fine. So it would seem that only Nested LowCardinality(String) fields exhibit the problem.

Expected behavior

Batch insert into LowCardinality fields that are nested

[jkaflik]

Confirmed it's reproducible with following:

package issues

import (
	"context"
	"testing"

	"github.com/ClickHouse/clickhouse-go/v2"
	clickhouse_tests "github.com/ClickHouse/clickhouse-go/v2/tests"
	"github.com/stretchr/testify/require"
)

func Test1297(t *testing.T) {
	testEnv, err := clickhouse_tests.GetTestEnvironment("issues")
	require.NoError(t, err)
	conn, err := clickhouse_tests.TestClientWithDefaultOptions(testEnv, clickhouse.Settings{
		"flatten_nested": "0",
	})
	require.NoError(t, err)

	require.NoError(t, conn.Exec(context.Background(), `CREATE TABLE test_1297
(
    Id UInt8,
    Device LowCardinality(String),
    Nestme Nested(
    	Id UInt32,
		TestLC LowCardinality(String),
		Test String
	)
)
ENGINE = MergeTree
ORDER BY Id;`), "Create table failed")
	t.Cleanup(func() {
		conn.Exec(context.Background(), "DROP TABLE IF EXISTS test_1297")
	})

	batch, err := conn.PrepareBatch(context.Background(), "INSERT INTO test_1297")
	require.NoError(t, err, "PrepareBatch failed")

	require.NoError(t, batch.Append(uint8(1), "pc", []any{[]any{1, "test LC 1", "test"}, []any{2, "test LC 2", "test"}}), "Append failed")
	require.NoError(t, batch.Send())
}

Requires additional debugging.