Reduce generate_dump mem usage for cores #3052
Conversation
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.
|
@yxieca can you please take a look |
|
@dgsudharsan can you suggest a reviewer for this change? Thanks! |
|
@vivekrnv Please review |
There was a problem hiding this comment.
So the assumption is saving into tarball would be efficient than saving to flash path. But we are not compressing the file while appending it to tarball so i'm not very sure how efficient this is than the current approach.
Can you provide date to signify this?
|
Hi @vivekrnv save_crash_files() calls save_file() for each core file, which will copy the core files into the tar_path. If the 4th argument of save_file() is true it will add it to the tarball, and delete the core file. If the 4th argument of save_file() is false, it will not add it to the tarfile, and will not delete the core the file. At the end of the generate_dump script save_to_tar() is called which will add all the files in tar_path to the tarball. If we had 4 core files, before this change we would end up with: After this change we would end up with: Resulting in 4 less core files consuming flash space. Of course this is temporary as tar_path will get cleaned up after this has run. But during this time the savings is significant in tests with lots of core dump files. |
|
@vivekrnv @dgsudharsan could you help to comment if this is good to merge. |
|
ADO:26118419 |
|
@davidm-arista there is a cherry pick failure, could you help to create a separate PR to 202305? |
|
Sorry. Maybe it is a bug in automation. |
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.
|
Cherry-pick PR to 202305: #3074 |
mssonicbld
added
Included in 202305 Branch
and removed
Created PR to 202305 Branch
labels
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.
|
Cherry-pick PR to 202311: #3140 |
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.
mssonicbld
added
Included in 202311 Branch
and removed
Created PR to 202311 Branch
labels
Add the core files to the tarball while they are been processed, this ensures that only one core file at a time will be consuming flash space inside the tarpath and the tarball.