Cursors and Performance

[dudehook]

We are running a PostgreSQL database with Infobright and very large data sets. We are running some rather involved queries against the data, which can return millions of rows, which we need to process as quickly as possible. Using the normal PostgreSQL JDBC driver is way too slow. I tried running our query through this Finagle 'driver' and the performance was even worse.

We get the best performance using the native libpg C library and cursors. It seems to me that we should be able to get pretty close to that in terms of performance from finagle-postgres... but I don't see the path towards that.

Is there "missing implementation" for cursors or streaming results from large queries? I would be willing to help in the implementation here if someone can point me in the right direction.

[dangerousben]

#128 should solve this?