Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak when creating new connections at each API Request. #258

Open
suvratjain1995 opened this issue Sep 21, 2023 · 0 comments
Open

Comments

@suvratjain1995
Copy link

Description

Making new connections at each request causes memory leak in python threading package threading.py at line no 908 and 913. Checked using tracemalloc for concurrent request.

Code to make connection and query

import tracemalloc
tracemalloc.start()
from fastapi import FastAPI, HTTPException, Query, APIRouter
import aiosqlite

app = FastAPI()

async def somefunction():
    db_path = "some.db"
    query = "select * from some_table limit 1"
    async with aiosqlite.connect(db_path) as db:
        cursor = await db.execute(query)
        row = await cursor.fetchone()
    if row is None:
        return None
    else:
        return row[0]
    
                

@app.post("/test"):
async def test():
    result = await somefunction()


@app.get("/get_memory_snapshot/")
async def get_memory_snapshot():
    try:
        snapshot = tracemalloc.take_snapshot()
        top_stats = snapshot.statistics("lineno")
        lines = []
        for stats in top_stats:
            lines.append(str(stats))
        return "\n".join(lines)
    except Exception as e:
        print("Error in getting memory allocation")

The issue was resolved when using global db connection.
async with aiosqlite.connect(db_path) as db: seems to be holding a threadlock even if the function context has ended.

run get_memory_snapshot after 1000 request to the the line consuming most memory

Details

  • OS: Mac Ventura
  • Python version: 3.8
  • aiosqlite version: aiosqlite==0.19.0
  • Can you repro on 'main' branch? yes
  • Can you repro in a clean virtualenv? yes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant