Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CuCholeskySolver GPU memory leak #334

Closed
KSepetanc opened this issue Apr 24, 2024 · 3 comments
Closed

CuCholeskySolver GPU memory leak #334

KSepetanc opened this issue Apr 24, 2024 · 3 comments

Comments

@KSepetanc
Copy link

I found out that solving the ExaModels AC OPF documentation example in a loop results in possible memory leak. All of 6 GB of GPU's VRAM is taken in about 33 solves.

Below is part relevant part of the code and full code to reproduce can be downloaded: gpu memory leak.zip

m = ac_power_model("pglib_opf_case13659_pegase.m",backend=CUDABackend())

for i in 1:50
    println(i)
    madnlp(m,linear_solver=MadNLPGPU.CuCholeskySolver)
end

Configuration:
Windows,
CPU: AMD R9 5950x
GPU GTX 1060 6GB

@sshin23
Copy link
Member

sshin23 commented Apr 24, 2024

GPU memory usage is not necessarily optimized on MadNLP. Curious if this is CuCholeskySolver's issue or general MadNLP issue. Have you tried CUDSSSolver (the default solver for CUDA)?

@KSepetanc
Copy link
Author

KSepetanc commented Apr 24, 2024

I have tried, but it fails with an error reported in 333.

I run dual boot of Windows and Linux and on Linux there is much less severe memory issue, but still existent.

@KSepetanc
Copy link
Author

With package update as suggested in 333 issue, I've managed to run OPF with CUDSSSolver.
I the memory issue is still present. Memory usage goes slowly up with repeated runs of MadNLP until failure due to full VRAM.

@MadNLP MadNLP locked and limited conversation to collaborators Apr 24, 2024
@sshin23 sshin23 converted this issue into discussion #335 Apr 24, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants