-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Passwords containing special characters are not decoded correctly #145
Comments
I've done lots more digging on this. I'm using databases with a PostgreSQL database (i.e. with the asyncpg backend. The reason this issue gets trickier is that databases passes the dsn directly to the asyncpg library which runs Now the issue is, There seems to be an issue relating to it in the asyncpg repository which I hope to read into further now. When using asyncpg directly, I would typically not use a dsn and pass individual elements which worked perfectly. Cheers |
I found that you can hack that behaviour the following way, it won't solve your pound symbol in password issue as it's a bug upstream, still you may avoid passing a full DSN
|
@euri10 Great trick, thanks heaps! Though I do believe that databases itself also has a little problem in that it doens't unquote parts of the URL (particularly the netloc) after using urlsplit in the |
@gvbgduh Thanks so much, yeah I think that the fix on asyncpg to do this was only just merged into master. So this should be safe for the next release or if you're installing from master. The latest release version (without the fix) doesn't decode URLs and therefore won't work. See MagicStack/asyncpg#436 for more info 😄 |
The only little thing that's a bit inelegant is when interrogating credentials via the databases library, youl'l still get URL encoded ouptut: e.g.
I'm not quite sure how much work the databases library should do here. But perhaps it should at least decode such components when they are obtained via a What do you think? Really appreciate your help! |
That seems correct to me, yeah. |
I've run into this as well, with other characters such as @ in the password. I thought this was a Starlette issue and opened up this issue: encode/starlette#662, but it looks like it's a little deeper. |
|
Looks like asyncpg has been updated: But even after upgrading asyncpg, this issue still shows up here. |
One thing that might be useful is support for urlencoding in Starlette.Configuration, somewhere. No idea what it'd look like yet. I know that doesn't solve the issue for databases, but that seems like the right level for this sort of fix. |
I put a proof of concept in encode/starlette#695. |
The reason for this issue is that
For using:
|
It's possible to "monkey patch" DatabaseURL to allow percent encoding and decoding of specific properties.
|
the cloud sql DSNs mentioned in MagicStack/asyncpg#419 really break the Example: # hostname is null:
>>> DatabaseURL('postgresql:///?host=/unix/socket').hostname
# replace() doesn't help:
>>> DatabaseURL('postgresql:///?host=/unix/socket').replace(hostname='/unix/socket').hostname
>>> This is extra-bad because:
Could you pass the DSN down to the backend unmodified? I'm not familiar with this project, so there may be good reasons not to do this, but it would make life easier for parsing edge cases like this one. |
Great discussion! I was curious that I didn't come across this issue before, since my passwords usually do contain special characters. Today, however, is my first brush with Databases. I use
def _parse_rfc1738_args(name):
pattern = re.compile(
r"""
(?P<name>[\w\+]+)://
(?:
(?P<username>[^:/]*)
(?::(?P<password>.*))?
@)?
(?:
(?:
\[(?P<ipv6host>[^/]+)\] |
(?P<ipv4host>[^/:]+)
)?
(?::(?P<port>[^/]*))?
)?
(?:/(?P<database>.*))?
""",
re.X,
)
... I tried a simple database URL in regex101 using |
I was running into the issue with special characters in the URL and came across this issue. This was helpful. Also it looks like To quote them, the following now works without monkeypatching: from urllib.parse import quote
user = quote(user) # e.g. user#001
password = quote(password) # e.g. P@ssw0rd#
url = f"{dialect}://{user}:{password}@{host}:{port}/{db}..." It would be nice to have this added to the documentation I think. |
Hey there guys, our database password contains a
#
character and databases doesn't seem to like it 😄I believe after
urlsplit
is called, you may also need to unquote each URL component?If you do not encode the password, urlsplit fails to split things properly:
This particular Python issue may be of relevance too.
Any help is greatly appreciated
Fotis
The text was updated successfully, but these errors were encountered: