Introduce SqliteObserver and use it to collect stats related to queries and db size #2707
Conversation
shrima-cf
requested review from
kentonv,
a-robinson,
justin-mp and
ramyabygari
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
1799527
to
153a9a2
Compare
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
153a9a2
to
aab4f76
Compare
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
aab4f76
to
ee5e13c
Compare
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
3 times, most recently
from
ce7963b
to
5dee08b
Compare
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
5dee08b
to
5e2e412
Compare
|
@justin-mp The new changes are the ones from here |
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
5e2e412
to
0ea3375
Compare
| @@ -1096,6 +1105,8 @@ void SqliteDatabase::Query::nextRow() { | |||
| db.currentRegulator = regulator; | |||
|
|
|||
| int err = sqlite3_step(statement); | |||
| rowsRead = getRowsRead(); | |||
There was a problem hiding this comment.
This seems inefficient as we're querying SQLite for rows read/written counts on every single row just in case the database is later resent before the query is destroyed -- which is very rare.
Instead, how about putting code into Query::beforeSqliteReset() that posts the metrics early (calls addQueryStats directly), and then have the destructor only post metrics if it hasn't been reset (i.e. maybeStatement is non-null).
There was a problem hiding this comment.
I did try that approach first. The problem with that comes in the way we do getRowsRead() and the way the Statement and Query are setup. There are a couple of corner cases that have to be handled - when the Query has its own statement vs passed in a statement, handling the beforeSqliteReset in both Query and Statement.
In the end, I decided to go this route as it is easy to reason about.
As far as inefficiency in querying Sqlite for rows read/written, looking at the implementation of getRowsRead/Written, it does a sqlite3_stmt_status(...) which does a pointer read, so its not as inefficient as we think it is
There was a problem hiding this comment.
I added a todo with more details
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
0ea3375
to
4b48d70
Compare
shrima-cf
force-pushed
the
shrima/STOR-3710-add-SqliteObserver
branch
from
4b48d70
to
222d244
Compare
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
https://jira.cfdata.org/browse/STOR-3710