Interpolate MySQL query parameters by default #5428
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tdeebswihart
force-pushed
the
tds/interpolate-mysql8-params
branch
from
5729b12
to
0d86141
Compare
|
So this is more complex than we initially though. When @mattrobenolt do you have measurements that show the benefit of enabling In the meantime I'm going to put a separate PR up to prevent folks from enabling |
tdeebswihart
added a commit
that referenced
this pull request
Feb 20, 2024
There are a number of reasons why this doesn't work right now so we're going to prevent it from happening while we discuss whether we should invest in fixing it. The current work to fix this is in #5428
|
|
tdeebswihart
force-pushed
the
tds/interpolate-mysql8-params
branch
from
0d86141
to
03a489a
Compare
tdeebswihart
changed the base branch from
main
to
tds/prevent-mysql8-vis-interpolateParams
tdeebswihart
force-pushed
the
tds/interpolate-mysql8-params
branch
from
72fa4c6
to
8d4428c
Compare
stephanos
approved these changes
Feb 26, 2024
rodrigozhou
approved these changes
Feb 28, 2024
tdeebswihart
added a commit
that referenced
this pull request
Feb 29, 2024
There are a number of reasons why this doesn't work right now so we're going to prevent it from happening while we discuss whether we should invest in fixing it. The current work to fix this is in #5428
tdeebswihart
force-pushed
the
tds/prevent-mysql8-vis-interpolateParams
branch
from
b9fa8cc
to
302d324
Compare
tdeebswihart
added a commit
that referenced
this pull request
Mar 6, 2024
There are a number of reasons why this doesn't work right now so we're going to prevent it from happening while we discuss whether we should invest in fixing it. The current work to fix this is in #5428
tdeebswihart
force-pushed
the
tds/prevent-mysql8-vis-interpolateParams
branch
from
302d324
to
768fd95
Compare
tdeebswihart
force-pushed
the
tds/interpolate-mysql8-params
branch
from
8d4428c
to
930c591
Compare
Base automatically changed from
tds/prevent-mysql8-vis-interpolateParams
to
main
March 8, 2024 20:03
tdeebswihart
added a commit
that referenced
this pull request
Mar 8, 2024
## What changed? Attempting to configure a MySQL 8 Visibility database with go-sql-driver/mysql's `interpolateParams` option will now fail. ## Why? There are a number of reasons why this doesn't work right now so we're going to prevent it from happening while we discuss how much we should invest in fixing it. The current work to fix this is in #5428 ## How did you test it? Added a new test ## Potential risks There should be none. Any user who has currently set up their database this way will already have encountered exciting errors like `Cannot create a JSON value from a string with CHARACTER SET 'binary'` and `Unable to parse ExecutionStatus value from DB`. ## Is hotfix candidate? That's up for discussion
You can't interpolate a []byte into a JSON field: go-sql-driver/mysql#819
It's useful to know which query failed when multiple are executed in a single operation
This cuts our queries down to 1 round-trip to the database per operation. When interpolateParams is false (the default) the mysql driver will call prepare before executing a query. Our MySQL 8 visibility code doesn't currently support this option however, so this is only configured for: - MySQL 5.7 Main dbs - MySQL 5.7 Visibility dbs - MySQL 8.0 Main dbs
tdeebswihart
force-pushed
the
tds/interpolate-mysql8-params
branch
from
930c591
to
3ea2045
Compare
stephanos
pushed a commit
to stephanos/temporal
that referenced
this pull request
Mar 21, 2024
…o#5441) ## What changed? Attempting to configure a MySQL 8 Visibility database with go-sql-driver/mysql's `interpolateParams` option will now fail. ## Why? There are a number of reasons why this doesn't work right now so we're going to prevent it from happening while we discuss how much we should invest in fixing it. The current work to fix this is in temporalio#5428 ## How did you test it? Added a new test ## Potential risks There should be none. Any user who has currently set up their database this way will already have encountered exciting errors like `Cannot create a JSON value from a string with CHARACTER SET 'binary'` and `Unable to parse ExecutionStatus value from DB`. ## Is hotfix candidate? That's up for discussion
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changed?
We now specify
interpolateParamsby default when using MySQL for our main.While we don't currently support specifying
interpolateParamsfor MySQL visibility databases, I changed the type returned when serializingVisibilitySearchAttributesas a first step towards doing so (MySQL 8 won't allow binary data within JSON columns, which makes sense).While we could also have explicitly prepared and reused our statements that requires far more work and it may not be the better approach anyways.
Why?
@mattrobenolt did a good job of explaining why in #5425 but I'll summarize it here anyways.
When
interpolateParamsis false (the default) the driver will prepare parameterized statements before executing them, meaning we need two round-trips to the database for each query. By settinginterpolateParamsto true the DB driver will handle interpolation and send the query just once to the database, halving the number of round trips necessary. This should improve the performance of all clusters using MySQL.How did you test it?
Existing tests.
Potential risks
No
Is hotfix candidate?
This is up for discussion