Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Alternative wire protocols support (RowBinaryWithNamesAndTypes) #255

Merged
merged 39 commits into from
Feb 28, 2020
Merged

Alternative wire protocols support (RowBinaryWithNamesAndTypes) #255

Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
301cf50
Remove unused keys
traceon Jan 20, 2020
f3529dd
Fix fromString()
traceon Jan 21, 2020
0ee121d
Fix int -> string attribute value extraction checks
traceon Jan 21, 2020
daa2452
Reaarrange parameters for better browsing
traceon Jan 22, 2020
4f4a563
Fix renamed parameter usage
traceon Jan 22, 2020
91bd8d0
Pass detected or anticipated format name to ResultSet c-tor
traceon Jan 22, 2020
a6c03ff
Fix icompare() typo
traceon Jan 22, 2020
8ec3763
Add more test cases
traceon Jan 23, 2020
b9fd612
Fix description for CentOS 7
traceon Feb 4, 2020
8a9b188
Deep refactoring of data conversion and result set processing
traceon Feb 14, 2020
8a0ad2f
GCC compilation fix/workaround: https://gcc.gnu.org/bugzilla/show_bug…
traceon Feb 15, 2020
24a1387
No more Python 2 in brew, using Python (3)
traceon Feb 15, 2020
9cfd946
Fix 'long' and 'unsigned long' conversion specialization conficts on …
traceon Feb 16, 2020
35cd88c
Merge branch 'switch-to-variant' into alternative-wire-protocol
traceon Feb 16, 2020
40f6405
Bump submodules
traceon Feb 16, 2020
8bcc22f
Fix STOP_MEASURING_TIME_AND_REPORT macro usage in new code
traceon Feb 16, 2020
8962392
Fix flag setting
traceon Feb 16, 2020
b811cdb
Fix macro name
traceon Feb 16, 2020
afe4e4c
Basic RowBinary deserialization
traceon Feb 16, 2020
5be48b2
UUID RowBinary deserialization
traceon Feb 16, 2020
8a1061c
Store precision and scale of decimal type specs in column info, and u…
traceon Feb 16, 2020
a714a0a
Use TEST_DSN_LIST instead of separate TEST_DSN and TEST_DSN_W ("-dsn-…
traceon Feb 16, 2020
85c4163
Fix cmake invocation
traceon Feb 16, 2020
448d55a
Shorten DSN name to avoid problems
traceon Feb 17, 2020
1c708be
Remove some cases to stay within common Decimal64 boundaries
traceon Feb 17, 2020
3355c9d
Make best efforts to provide meaningful display size as early as poss…
traceon Feb 17, 2020
626927b
Change ResultMutator interface
traceon Feb 18, 2020
b4f6880
Add default conversion from DataSourceType<DataSourceTypeId::Nothing>…
traceon Feb 18, 2020
1d483bf
Recycle strings in the pool based on capacity which should be greater…
traceon Feb 18, 2020
dc0e93d
Merge branch 'master' into alternative-wire-protocol
traceon Feb 18, 2020
4dd7f92
Fix cmake (build) invocation flag
traceon Feb 18, 2020
d8f523c
Add comments
traceon Feb 25, 2020
758f7f5
Update brew before installing packages
traceon Feb 25, 2020
5004eed
Add "URL query string" section
traceon Feb 25, 2020
eb5cf2d
Working around wierd behavior of brew
traceon Feb 25, 2020
b31a5c3
Try versionned openssl package name
traceon Feb 25, 2020
654d0d4
Set OPENSSL_ROOT_DIR explicitly
traceon Feb 25, 2020
c685014
Add note about deifference in timezone handling between formats
traceon Feb 25, 2020
58c068b
Add convert()'s that accept moving-into source values
traceon Feb 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,8 +375,9 @@ install: |-
DebugFile = ${RUN_DIR}/odbc-driver-manager-debug.log

[ODBC Data Sources]
ClickHouse DSN (ANSI) = ClickHouse ODBC Driver (ANSI)
ClickHouse DSN (Unicode) = ClickHouse ODBC Driver (Unicode)
ClickHouse DSN (ANSI) = ClickHouse ODBC Driver (ANSI)
ClickHouse DSN (Unicode) = ClickHouse ODBC Driver (Unicode)
ClickHouse DSN (ANSI, RBWNAT) = ClickHouse ODBC Driver (ANSI)

[ClickHouse DSN (ANSI)]
Driver = ClickHouse ODBC Driver (ANSI)
Expand All @@ -391,23 +392,34 @@ install: |-
Url = http://${CLICKHOUSE_SERVER_IP}
DriverLog = yes
DriverLogFile = ${RUN_DIR}/clickhouse-odbc-driver-w.log

[ClickHouse DSN (ANSI, RBWNAT)]
Driver = ClickHouse ODBC Driver (ANSI)
Description = Test DSN for ClickHouse ODBC Driver (ANSI) that uses RowBinaryWithNamesAndTypes as data source communication default format
Url = http://${CLICKHOUSE_SERVER_IP}/query?default_format=RowBinaryWithNamesAndTypes
DriverLog = yes
DriverLogFile = ${RUN_DIR}/clickhouse-odbc-driver.log
EOF
fi
fi

.configure: &configure |-
CMAKE_CONFIGURE_ARGS="'-DTEST_DSN=ClickHouse DSN (ANSI)' '-DTEST_DSN_W=ClickHouse DSN (Unicode)' $CMAKE_CONFIGURE_EXTRA_ARGS"
CMAKE_CONFIGURE_ARGS="-DTEST_DSN_LIST=\"ClickHouse DSN (ANSI);ClickHouse DSN (Unicode);ClickHouse DSN (ANSI, RBWNAT)\" $CMAKE_CONFIGURE_EXTRA_ARGS"
if [[ "$EXTERNAL_THIRD_PARTY" == "yes" ]]; then
CMAKE_CONFIGURE_ARGS="-DCH_ODBC_PREFER_BUNDLED_THIRD_PARTIES=OFF $CMAKE_CONFIGURE_ARGS"
fi
if [[ ! -z "$ODBC_PROVIDER" ]]; then
CMAKE_CONFIGURE_ARGS="-DODBC_PROVIDER=$ODBC_PROVIDER $CMAKE_CONFIGURE_ARGS"
fi
cd $BUILD_DIR
echo "#!/usr/bin/env bash" > cmake_configure.sh
echo "set -Eeuo pipefail" >> cmake_configure.sh
echo "cmake $CMAKE_CONFIGURE_ARGS $SOURCE_DIR" >> cmake_configure.sh
chmod u+x cmake_configure.sh
if [[ "$TRAVIS_OS_NAME" == "windows" ]]; then
$SOURCE_DIR/test/vcvars_enabled_bash.sh cmake $CMAKE_CONFIGURE_ARGS $SOURCE_DIR
$SOURCE_DIR/test/vcvars_enabled_bash.sh ./cmake_configure.sh
else
cmake $CMAKE_CONFIGURE_ARGS $SOURCE_DIR
./cmake_configure.sh
fi

.build: &build |-
Expand Down
16 changes: 7 additions & 9 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,18 @@ endif ()
# Make sure that all optimized builds have NDEBUG macro set.
foreach (config Release RelWithDebInfo MinSizeRel)
foreach (lang CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
set (var "${lang}_${config}")
string (TOUPPER "${config}" config_uc)
set (var "${lang}_${config_uc}")
set (${var} "${${var}} -DNDEBUG")
endforeach ()
endforeach ()

# Set BUILD_TYPE_* macro for each of build types.
foreach (config Debug Release RelWithDebInfo MinSizeRel)
foreach (lang CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
set (var "${lang}_${config}")
set (${var} "${${var}} -DBUILD_TYPE_${config}")
string (TOUPPER "${config}" config_uc)
set (var "${lang}_${config_uc}")
set (${var} "${${var}} -DBUILD_TYPE_${config_uc}")
endforeach ()
endforeach ()

Expand Down Expand Up @@ -149,12 +151,8 @@ if (CH_ODBC_ENABLE_TESTING)
enable_testing ()
include (GoogleTest)

if (NOT TEST_DSN)
set (TEST_DSN "${CH_ODBC_DEFAULT_DSN_ANSI}")
endif ()

if (NOT TEST_DSN_W)
set (TEST_DSN_W "${CH_ODBC_DEFAULT_DSN_UNICODE}")
if (NOT TEST_DSN_LIST)
set (TEST_DSN_LIST "${CH_ODBC_DEFAULT_DSN_ANSI};${CH_ODBC_DEFAULT_DSN_UNICODE}")
endif ()
endif ()

Expand Down
59 changes: 29 additions & 30 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,24 +128,23 @@ See the exact steps for each platform in the corresponding section below:

The list of configuration options recognized during the CMake generation step is as follows:

| Option | Default value | Description |
| :------------------------------------: | :------------------------------------------------------: | :--------------------------------------------------------------------------------------- |
| `CMAKE_BUILD_TYPE` | `RelWithDebInfo` | Build type, one of: `Debug`, `Release`, `RelWithDebInfo` |
| `CH_ODBC_ALLOW_UNSAFE_DISPATCH` | `ON` | Allow unchecked handle dispatching (may slightly increase performance in some scenarios) |
| `CH_ODBC_ENABLE_SSL` | `ON` | Enable TLS/SSL (required for utilizing `https://` interface, etc.) |
| `CH_ODBC_ENABLE_INSTALL` | `ON` | Enable install targets (required for packaging) |
| `CH_ODBC_ENABLE_TESTING` | inherits value of `BUILD_TESTING` | Enable test targets |
| `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | `ON` | Prefer bundled over system variants of third party libraries |
| `CH_ODBC_PREFER_BUNDLED_POCO` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of Poco library |
| `CH_ODBC_PREFER_BUNDLED_SSL` | inherits value of `CH_ODBC_PREFER_BUNDLED_POCO` | Prefer bundled over system variants of TLS/SSL library |
| `CH_ODBC_PREFER_BUNDLED_GOOGLETEST` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of Google Test library |
| `CH_ODBC_PREFER_BUNDLED_NANODBC` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of nanodbc library |
| `CH_ODBC_RUNTIME_LINK_STATIC` | `OFF` | Link with compiler and language runtime statically |
| `CH_ODBC_THIRD_PARTY_LINK_STATIC` | `ON` | Link with third party libraries statically |
| `CH_ODBC_DEFAULT_DSN_ANSI` | `ClickHouse DSN (ANSI)` | Default ANSI DSN name |
| `CH_ODBC_DEFAULT_DSN_UNICODE` | `ClickHouse DSN (Unicode)` | Default Unicode DSN name |
| `TEST_DSN` | inherits value of `CH_ODBC_DEFAULT_DSN_ANSI` | ANSI DSN name to use in tests |
| `TEST_DSN_W` | inherits value of `CH_ODBC_DEFAULT_DSN_UNICODE` | Unicode DSN name to use in tests |
| Option | Default value | Description |
| :------------------------------------: | :----------------------------------------------------------: | :--------------------------------------------------------------------------------------- |
| `CMAKE_BUILD_TYPE` | `RelWithDebInfo` | Build type, one of: `Debug`, `Release`, `RelWithDebInfo` |
| `CH_ODBC_ALLOW_UNSAFE_DISPATCH` | `ON` | Allow unchecked handle dispatching (may slightly increase performance in some scenarios) |
| `CH_ODBC_ENABLE_SSL` | `ON` | Enable TLS/SSL (required for utilizing `https://` interface, etc.) |
| `CH_ODBC_ENABLE_INSTALL` | `ON` | Enable install targets (required for packaging) |
| `CH_ODBC_ENABLE_TESTING` | inherits value of `BUILD_TESTING` | Enable test targets |
| `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | `ON` | Prefer bundled over system variants of third party libraries |
| `CH_ODBC_PREFER_BUNDLED_POCO` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of Poco library |
| `CH_ODBC_PREFER_BUNDLED_SSL` | inherits value of `CH_ODBC_PREFER_BUNDLED_POCO` | Prefer bundled over system variants of TLS/SSL library |
| `CH_ODBC_PREFER_BUNDLED_GOOGLETEST` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of Google Test library |
| `CH_ODBC_PREFER_BUNDLED_NANODBC` | inherits value of `CH_ODBC_PREFER_BUNDLED_THIRD_PARTIES` | Prefer bundled over system variants of nanodbc library |
| `CH_ODBC_RUNTIME_LINK_STATIC` | `OFF` | Link with compiler and language runtime statically |
| `CH_ODBC_THIRD_PARTY_LINK_STATIC` | `ON` | Link with third party libraries statically |
| `CH_ODBC_DEFAULT_DSN_ANSI` | `ClickHouse DSN (ANSI)` | Default ANSI DSN name |
| `CH_ODBC_DEFAULT_DSN_UNICODE` | `ClickHouse DSN (Unicode)` | Default Unicode DSN name |
| `TEST_DSN_LIST` | `${CH_ODBC_DEFAULT_DSN_ANSI};${CH_ODBC_DEFAULT_DSN_UNICODE}` | `;`-separated list of DSNs, each test will be executed with each of these DSNs |

Configuration options above can be specified in the first `cmake` command (generation step) in a form of `-Dopt=val`.

Expand Down Expand Up @@ -328,14 +327,14 @@ cmake -A x64 -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
Build the generated solution in-place:

```sh
cmake --build . -C RelWithDebInfo
cmake --build . -C RelWithDebInfo --target package
cmake --build . --config RelWithDebInfo
cmake --build . --config RelWithDebInfo --target package
```

...and, optionally, run tests (note, that for non-unit tests, preconfigured driver and DSN entries must exist, that point to the binaries generated in this build folder):

```sh
cmake --build . -C RelWithDebInfo --target test
cmake --build . --config RelWithDebInfo --target test
```

...or open the IDE and build `all`, `package`, and `test` targets manually from there:
Expand Down Expand Up @@ -390,14 +389,14 @@ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
Build the generated solution in-place:

```sh
cmake --build . -C RelWithDebInfo
cmake --build . -C RelWithDebInfo --target package
cmake --build . --config RelWithDebInfo
cmake --build . --config RelWithDebInfo --target package
```

...and, optionally, run tests (note, that for non-unit tests, preconfigured driver and DSN entries must exist, that point to the binaries generated in this build folder):

```sh
cmake --build . -C RelWithDebInfo --target test
cmake --build . --config RelWithDebInfo --target test
```

...or, if you configured the project with '-G Xcode' initially, open the IDE and build `all`, `package`, and `test` targets manually from there:
Expand Down Expand Up @@ -461,14 +460,14 @@ cmake3 -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
Build the generated solution in-place:

```sh
cmake3 --build . -C RelWithDebInfo
cmake3 --build . -C RelWithDebInfo --target package
cmake3 --build . --config RelWithDebInfo
cmake3 --build . --config RelWithDebInfo --target package
```

...and, optionally, run tests (note, that for non-unit tests, preconfigured driver and DSN entries must exist, that point to the binaries generated in this build folder):

```sh
cmake3 --build . -C RelWithDebInfo --target test
cmake3 --build . --config RelWithDebInfo --target test
```

### Building from sources: Debian/Ubuntu
Expand Down Expand Up @@ -514,12 +513,12 @@ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
Build the generated solution in-place:

```sh
cmake --build . -C RelWithDebInfo
cmake --build . -C RelWithDebInfo --target package
cmake --build . --config RelWithDebInfo
cmake --build . --config RelWithDebInfo --target package
```

...and, optionally, run tests (note, that for non-unit tests, preconfigured driver and DSN entries must exist, that point to the binaries generated in this build folder):

```sh
cmake --build . -C RelWithDebInfo --target test
cmake --build . --config RelWithDebInfo --target test
```
13 changes: 8 additions & 5 deletions driver/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,17 @@ unset (WORKAROUND_ENABLE_NO_UNICODE_CHARS_IN_COLUMN_NAMES_IN_TESTS)

# In order to enable testing, put every non-public symbol to a static library (which is then used by shared library and unit-test binary).
add_library (${libname}-impl STATIC
utils/read_helpers.cpp
utils/type_parser.cpp
utils/type_info.cpp

config/config.cpp

escaping/escape_sequences.cpp
escaping/lexer.cpp

format/ODBCDriver2.cpp
format/RowBinaryWithNamesAndTypes.cpp

api/impl/impl.cpp

attributes.cpp
Expand All @@ -66,7 +69,6 @@ add_library (${libname}-impl STATIC
object.cpp
result_set.cpp
statement.cpp
type_info.cpp


${CMAKE_CURRENT_BINARY_DIR}/platform/config_cmake${WIDECHAR_SUFFIX}.h
Expand All @@ -75,9 +77,8 @@ add_library (${libname}-impl STATIC
utils/utils.h
utils/unicode_conv.h
utils/iostream_debug_helpers.h
utils/read_helpers.h
utils/type_parser.h
utils/scope_guard.h
utils/type_info.h

config/config.h
config/ini_defines.h
Expand All @@ -89,6 +90,9 @@ add_library (${libname}-impl STATIC

api/impl/impl.h

format/ODBCDriver2.h
format/RowBinaryWithNamesAndTypes.h

attributes.h
connection.h
descriptor.h
Expand All @@ -99,7 +103,6 @@ add_library (${libname}-impl STATIC
object.h
result_set.h
statement.h
type_info.h
)

set (WIN_SOURCES)
Expand Down
Loading