Superset memory leak

[chen-ABC]

A clear and concise description of what the bug is.
After querying the same chart for a long time, the available memory continues to decrease until the system runs out of memory

Expected results

Recycle memory after querying the chart

Actual results

The available memory keeps decreasing

Screenshots

If applicable, add screenshots to help explain your problem.
I started a scheduler job to query chart

How to reproduce the bug

1.force refresh of the chart in Dashboard
2. use memory profile

3. see log

After many times

use gc.collect() at end line, But the memory is not reclaimed

Environment

• superset version: 1.1.0
• python version: 3.7.9
• node.js version: 12.18.3

[zhaoyongjie]

Hi @chen-ABC. I also found this problem a long time ago when using incubator-superset in a production environment. My strategy is to restart the gunicorn process periodically. for instance:

gunicorn --max-requests 1024 ....
or
gunicorn --max-requests 1024 --max-requests-jitter 100 ....

refer to: https://docs.gunicorn.org/en/stable/settings.html#max-requests

[chen-ABC]

@zhaoyongjie Thanks, This is useful for HTTP requests. but it has no effect on apscheduler job(Internal calls)

[zhaoyongjie]

@zhaoyongjie Thanks, This is useful for HTTP requests. but it has no effect on apscheduler job(Internal calls)

The Superset only exposes an HTTP service, so I guess that it will work either way you call it(either external job request or common HTTP request).

[chen-ABC]

@zhaoyongjie Thanks, This is useful for HTTP requests. but it has no effect on apscheduler job(Internal calls)

The Superset only exposes an HTTP service, so I guess that it will work either way you call it(either external job request or common HTTP request).

thanks, That's right

[chen-ABC]

@zhaoyongjie
I think I found the cause of the memory leak.

 @utils.memoized(watch=("impersonate_user", "sqlalchemy_uri_decrypted", "extra"))
    def get_sqla_engine(

memoized use tuple key , The keys for the same query are different. Because there's an object in the key

I think we have the same problem in superset 1.3.0
example:

[villebro]

Thanks for investigating this @chen-ABC ! I can try to repro and help get this fixed and released soon if this is indeed the culprit!

[zhaoyongjie]

@zhaoyongjie I think I found the cause of the memory leak.

 @utils.memoized(watch=("impersonate_user", "sqlalchemy_uri_decrypted", "extra"))
    def get_sqla_engine(

memoized use tuple key , The keys for the same query are different. Because there's an object in the key

I think we have the same problem in superset 1.3.0 example:

Thanks a lot.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[jhult]

It looks like PR #17319 is still open and needs to be merged.

[zhaoyongjie]

@jhult I will follow this issue recently.

[mdeshmu]

I am also observing memory leak. We have auto dashboard refresh enabled. Memory keeps on increasing even though there aren't many active users.