use socketfile to improve perf

[alonme]

#397

[CLAassistant]

All committers have signed the CLA.

[alonme]

inspired by @alippai suggestion in #398.

i see a 30%~ performance boost on queries

[alonme]

The benchmark i used - running on the vertica-ce docker, and using the Vmart database.

I switched between the new read_bytes_using_file implementation to the current read_bytes in the source code, and autoreload reloads the module before running each line.

In this example i see a 50% performance improvement

In [1]: # Use auto-reload to switch between `read_bytes` and `read_bytes_using_file`
   ...: %load_ext autoreload
   ...: %autoreload 2

In [2]: import vertica_python
   ...:
   ...: conn_info = {
   ...:     "host": "127.0.0.1",
   ...:     "port": 5433,
   ...:     "user": "dbadmin",
   ...:     #'password': 'some_password',
   ...:     "database": "VMart",
   ...:     "session_label": "some_label",
   ...:     "unicode_error": "strict",
   ...:     "ssl": False,
   ...:     "autocommit": True,
   ...:     "use_prepared_statements": False,
   ...:     "connection_timeout": 5,
   ...: }
   ...:
   ...: query = """
   ...: SELECT sales_quantity, sales_dollar_amount, transaction_type
   ...: FROM online_sales.online_sales_fact
   ...: LIMIT 100000
   ...: """
   ...:
   ...: def run_query():
   ...:     with vertica_python.connect(**conn_info) as conn:
   ...:         cur = conn.cursor()
   ...:         cur.execute(query)
   ...:         cur.fetchall()
   ...:

In [3]: # using read_bytes_using_file

In [4]: %timeit run_query()
960 ms ± 6.09 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [5]: # Using read_bytes

In [6]: %timeit run_query()
1.95 s ± 7.33 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[alonme]

@sitingren - Hey, do you need any additional information regarding this change?
My last commits removed the previous implementation which i left in the code to make it easy to benchmark.

[ni-todo-spot]

Wow! Impressive work 🥇
Our company would definitely benefit from such improvement!

[alonme]

@sitingren - Thanks for the review,
I believe i fixed / answered all the issues.

[alonme]

Thanks @sitingren !

When can we expect a release?

[ni-todo-spot]

Thanks @sitingren !

When can we expect a release?

Very exciting news guys :)
We'd love to use this feature!

[sitingren]

This goes into release v1.0.5.

[alippai]

buf += data might not be needed at all. The read will return exactly the amount needed. As far as I remember this interface is different from the original read as it returns full results only

edit: both cpython and pypy return a io.BufferedReader which will handle this for you