-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use min_size and max_size in postgres backends #129
Changes from 5 commits
36d54c8
aaa0a4b
8ee141b
0347c50
17c7d9a
54d8d33
c341a77
3fd7fba
c6ab59e
32d9776
6e11017
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,14 @@ | ||
""" | ||
Unit tests for the backend connection arguments. | ||
""" | ||
from contextlib import suppress | ||
|
||
import pytest | ||
|
||
from databases import Database | ||
from databases.backends.mysql import MySQLBackend | ||
from databases.backends.postgres import PostgresBackend | ||
from tests.test_databases import POSTGRES_URLS, async_adapter | ||
|
||
|
||
def test_postgres_pool_size(): | ||
|
@@ -30,6 +35,24 @@ def test_postgres_explicit_ssl(): | |
assert kwargs == {"ssl": True} | ||
|
||
|
||
urls_with_options = [ | ||
(f"{POSTGRES_URLS[0]}?min_size=1&max_size=20", suppress()), | ||
(f"{POSTGRES_URLS[0]}?min_size=0&max_size=0", pytest.raises(ValueError)), | ||
(f"{POSTGRES_URLS[0]}?min_size=10&max_size=0", pytest.raises(ValueError)), | ||
] | ||
|
||
|
||
@pytest.mark.parametrize("database_url, expectation", urls_with_options) | ||
@async_adapter | ||
async def test_postgres_pool_size_connect(database_url, expectation): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could do with structuring this a bit more simply. Not sure exactly how, but I'd rather have something a bit more plain. The empty There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I moved POSTGRES_URLS where it's used. |
||
with expectation: | ||
database = Database(database_url) | ||
await database.connect() | ||
assert database.is_connected | ||
await database.disconnect() | ||
assert not database.is_connected | ||
|
||
|
||
def test_mysql_pool_size(): | ||
backend = MySQLBackend("mysql://localhost/database?min_size=1&max_size=20") | ||
kwargs = backend._get_connection_kwargs() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably something like this instead:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tomchristie @euri10 sorry for jumping into, but it'll break other things. The matter is that the query is the only way to pass other params to
asyncpg
likesocket
orhost
that breaks the dsn parsing or caching params, etc.self._get_connection_kwargs()
will return only pool size details and the flag forssl
, but the rest will erased.I think we should catch all query params in
self._get_connection_kwargs()
or use an approach used in other backends -The second one seems as a better option as its aligned with the unified approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does that look better @gvbgduh ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it definitely does @euri10!
But what I missed is that params from query are coming through either.
_get_connection_kwargs()
accessesurl.options
, but looks for specific params, andDatabaseURL
doesn't look in query for missing things.It might be reasonable to tweak
DatabaseURL
to look into options/query if it can't find params in the main url, at least for {username, password, database, port, hostname}.If @tomchristie has no objections
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's true that
_get_connection_kwargs
currently removes anything that is not min_size, max_size or sslwith an dsn like
f"{POSTGRES_URLS[0]}?min_size=1&max_size=2&sslmode=require"
for instance the sslmode is lostThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually, it should pass it as of line https://github.com/encode/databases/blob/master/databases/backends/postgres.py#L58, but those options come from the
Database
instantiation - https://github.com/encode/databases/blob/master/databases/core.py#L32.So it'll broadcast provided kwargs as is to the DB driver.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't see the kwarg
sslmode
for the connection or for the pool inasyncpg
, so you should probably use:but in general you should be able to provide it as
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
got it, needs to be passed from Database
for reference sslmode is defined here in asyncpg and used in those tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, I see, thanks!