Skip to content

Commit

Permalink
Add ADR explaining why '::' was chosen as nested separator
Browse files Browse the repository at this point in the history
  • Loading branch information
pivovarit committed Jul 11, 2024
1 parent 4590a4b commit 0a0645a
Showing 1 changed file with 36 additions and 0 deletions.
36 changes: 36 additions & 0 deletions adr/4_double_colon_as_nested_structure_separator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Use Double Colon (`::`) as a Separator for Nested Fields Representation in Clickhouse

## Context and Problem Statement

When ingesting JSON documents, representing nested objects in ClickHouse requires special handling - nested objects get flattened into columns.
To distinguish between nested fields, a separator is needed. The most natural choice would be a dot (`.`), but ClickHouse treats it as a special character, which can lead to parsing errors or unexpected behavior even when properly escaped.

### Considered Options

1. **Using Dot (`.`) as a Separator**
- Pros:
- A natural solution
- Familiar to users.
- Cons:
- Doesn't work
2. **Using Double Colon (`::`) as a Separator**
- Pros:
- Avoids conflict with ClickHouse's treatment of `.`.
- Clear and unambiguous representation of nested fields.
- Simplifies query generation and ensures correct parsing by ClickHouse.
- Rarely used in field names, reducing the risk of conflicts.
- Cons:
- Less common and may require initial adaptation by users.
- Requires extra effort to expose those to users as dots

### Decision Outcome and Drivers

**Chosen Option:** Using Double Colon (`::`) as a Separator because:
- **Avoids conflicts:** By using `::`, we avoid the special treatment of `.` in ClickHouse, ensuring that nested fields are parsed and interpreted correctly without additional processing.
- **No better alternatives exist:** Every separator that is not `.` will have more/less same cons as `::` and there's nothing we can do about it for now.

We rejected the option of using `.` because it didn't work.

## People

- @pivovarit

0 comments on commit 0a0645a

Please sign in to comment.