feat: use a new official CH driver: clickhouse-connect

[EugeneTorap]

Use official clickhouse-connect driver instead of clickhouse-sqlalchemy

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Merging #22039 (b47f5fb) into master (c3f1873) will decrease coverage by 11.26%.
The diff coverage is n/a.

❗ Current head b47f5fb differs from pull request most recent head 40ac9f4. Consider uploading reports for the commit 40ac9f4 to get more accurate results

@@             Coverage Diff             @@
##           master   #22039       +/-   ##
===========================================
- Coverage   66.94%   55.67%   -11.27%     
===========================================
  Files        1831     1831               
  Lines       69833    69833               
  Branches     7570     7570               
===========================================
- Hits        46749    38880     -7869     
- Misses      21126    28995     +7869     
  Partials     1958     1958               

Flag	Coverage Δ
hive	52.56% <ø> (ø)
postgres	?
presto	52.45% <ø> (ø)
python	57.75% <ø> (-23.58%)	⬇️
sqlite	?
unit	50.83% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/utils/dashboard_import_export.py	0.00% <0.00%> (-100.00%)	⬇️
superset/tags/core.py	4.54% <0.00%> (-95.46%)	⬇️
superset/key_value/commands/update.py	0.00% <0.00%> (-90.91%)	⬇️
superset/key_value/commands/delete.py	0.00% <0.00%> (-87.88%)	⬇️
superset/key_value/commands/delete_expired.py	0.00% <0.00%> (-84.00%)	⬇️
superset/dashboards/commands/importers/v0.py	15.62% <0.00%> (-76.25%)	⬇️
superset/datasets/commands/create.py	30.61% <0.00%> (-69.39%)	⬇️
superset/datasets/commands/update.py	25.00% <0.00%> (-69.05%)	⬇️
superset/datasets/commands/importers/v0.py	24.03% <0.00%> (-69.00%)	⬇️
superset/reports/commands/execute.py	23.94% <0.00%> (-68.67%)	⬇️
... and 281 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[villebro]

Excited about this development, thanks for the PR 👍 One question about upgrading to 0.4.0 (it worked well locally for me). Also, could we migrate the legacy tests from tests/integration_tests/db_engine_specs/clickhouse_tests.py to tests/unit_tests/db_engine_specs/test_clickhouse.py and make sure all relevant code paths are covered? If you need help please ping me on Slack.

[EugeneTorap]

Waiting for the 0.4.1 version of clickhouse-connect

[EugeneTorap]

@villebro @genzgd How to fix the next error?

test_clickhouse.py:42: error: Unexpected keyword argument "message" for "NewConnectionError"

[genzgd]

I think tests are little tricky because clickhouse-connect provides its own EngineSpec class that is not part of the superset package or code base. As far as I know this is the only EngineSpec that works this way, and it uses the Python "entry point" mechanism to do that.

The result is the class to test is clickhouse_connect.cc_superset.engine.ClickHouseEngineSpec instead of superset.db_engine_specs.clickhouse -- superset.db_engine_specs.clickhouse is coupled with the clickhouse-sqlachemy library.

[genzgd]

ClickHouse Connect 0.4.1 has been released on PyPI.

[EugeneTorap]

ClickHouse Connect 0.4.1 has been released on PyPI.

Thank you so much!

[villebro]

LGTM and tested to work nicely. Looking forward to using this one in the future! 👍