Leaked connections caused by timeout on pool.acquire

[nevladov]

• asyncpg version: 0.18.2
• PostgreSQL version: 10
• Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
  the issue with a local PostgreSQL install?:
• Python version: 3.6.6
• Platform:
• Do you use pgbouncer?: yes
• Did you install asyncpg with pip?:
• If you built asyncpg locally, which version of Cython did you use?:
• Can the issue be reproduced under both asyncio and
  uvloop?:

Timeout on pool.acquire can be cause of leaked connections. The reason is that asyncio.wait_for cancels inner task on timeout here and it is the same problem as described in #464. But in this case using of asyncio.shield outside asyncpg cannot fix the problem.
The code below reproduces the problem (timeout which reproduces the problem depends on connection time):

import asyncpg
import traceback

async def create_pool():
    return await asyncpg.create_pool(
        host='localhost',
        port='6432',
        database='db1',
        user='user1',
        password='12345678',
        min_size=0,
        max_size=3,
    )

async def test_connect(pool):
    conn = None
    try:
        async with pool.acquire(timeout=0.11) as conn:  # the problem is here
            await conn.fetch(
                'SELECT pg_sleep(0.1);',
            )
    except Exception as e:
        print('Error occured')
        traceback.print_exc()

async def main():
    pool = await create_pool()
    for i in range(100):
        await test_connect(pool)
    await pool.close()
    await asyncio.sleep(600)

import asyncio
loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Traceback of error:

Traceback (most recent call last):
  File "<stdin>", line 4, in test_connect
  File "python3.6/site-packages/asyncpg/pool.py", line 760, in __aenter__
    self.connection = await self.pool._acquire(self.timeout)
  File "python3.6/site-packages/asyncpg/pool.py", line 605, in _acquire
    _acquire_impl(), timeout=timeout, loop=self._loop)
  File "python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

Even after pool is closed some connections are not closed while python process is running.

[stalkerg]

I don't know is it the same problem or not but my server time to time start freeze because all connection was stuck. I have some exceptions time to time from PG but it's not a timeout.

[1st1]

Should be fixed in master. Thanks for reporting!