Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageDecoder+LibGfx: collate decoded bitmaps before sending over IPC #1435

Merged
merged 1 commit into from
Oct 2, 2024

Conversation

zack466
Copy link
Contributor

@zack466 zack466 commented Sep 18, 2024

There is an issue (#425) where gifs with many frames cannot be loaded, as each bitmap is sent over IPC using a separate file descriptor, and there is limit on the maximum number of descriptors per IPC message. Thus, trying to load gifs with more than 64 frames (the current limit) causes the image decoder process to die.

This commit introduces the BitmapSequence class, which is a thin wrapper around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and provides an IPC encode/decode routine that collates all bitmap data into a single buffer so that only a single file descriptor is required per IPC transfer, even if multiple frames are being sent. This seems to fix the issue with large gifs (at least on my machine).

Right now, I'm using memcpy and pointer math to move data between AnonymousBuffers, which is probably not great for memory safety. If there's a better way of doing this that's more in line with project standards, please let me know.

@ladybird-bot
Copy link
Collaborator

Hello!

One or more of the commit messages in this PR do not match the Ladybird code submission policy, please check the lint_commits CI job for more details on which commits were flagged and why.
Please do not close this PR and open another, instead modify your commit message(s) with git commit --amend and force push those changes to update this PR.

@nico
Copy link
Contributor

nico commented Sep 18, 2024

Didn't #688 fix #425 already?

@zack466
Copy link
Contributor Author

zack466 commented Sep 18, 2024

Doesn't seem so? At least I was still experiencing the issue in #425 locally (as in the image decoder process would die when loading gifs with many frames).

@zack466
Copy link
Contributor Author

zack466 commented Sep 18, 2024

Also, even with the changes in #688, the issue seems to be that the image decoder is trying to send the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> over IPC, which boils down to sending one Gfx::Bitmap per frame, and each Gfx::Bitmap gets serialized as its own file with a file descriptor, which is why so many file descriptors get opened for large gifs.

Copy link
Member

@ADKaster ADKaster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this! It is indeed a very real problem that we can't handle large gifs, and create too many files.

I left some comments on the implementation. I think it could be more modular and factored. But overall the approach is sound, to create a metadata header followed by the actual data.

I'm not sure if it makes the most sense to structure it as

[ sequence of metadata ]
[ big blob ]

rather than

[ sequence of 
   [ metadata ]
   [ blob for this image ]
]

But the result is the same, as long as we take care to sanity check all the size values before reading from the binary blobs.

Userland/Libraries/LibGfx/BitmapSequence.cpp Outdated Show resolved Hide resolved
Userland/Libraries/LibGfx/BitmapSequence.cpp Outdated Show resolved Hide resolved
Userland/Libraries/LibGfx/BitmapSequence.cpp Outdated Show resolved Hide resolved
Userland/Libraries/LibGfx/BitmapSequence.cpp Outdated Show resolved Hide resolved
Userland/Libraries/LibGfx/BitmapSequence.h Outdated Show resolved Hide resolved
@zack466
Copy link
Contributor Author

zack466 commented Sep 24, 2024

Thanks for the feedback!

I'm not sure if it makes the most sense to structure it as


[ sequence of metadata ]

[ big blob ]

rather than


[ sequence of 

   [ metadata ]

   [ blob for this image ]

]

As far as I can tell, it necessarily has to be the first way, since each blob is encoded as a separate file (with a separate file descriptor). That's why I made it so that the separate blobs get merged into a single blob in order to send over IPC. Also, the existing code is already doing it the second way, which is where the problem stems from.

There is an issue where gifs with many frames cannot be loaded, as each
bitmap is sent over IPC using a separate file descriptor, and there is
limit on the maximum number of descriptors per IPC message. Thus, trying
to load gifs with more than 64 frames (the current limit) causes the
image decoder process to die.

This commit introduces the BitmapSequence class, which is a thin wrapper
around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and
provides an IPC encode/decode routine that collates all bitmap data into
a single buffer so that only a single file descriptor is required per
IPC transfer, even if multiple frames are being sent.
Copy link
Member

@ADKaster ADKaster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks! I confirmed that the patch lets us load the gif from #425 as well. There's a few minor nits that I'll update in a follow-up. For future PRs, consider creating a branch per change set so that you aren't locked in to only have one PR open from your master branch at once :).

@ADKaster ADKaster merged commit e0bd42b into LadybirdBrowser:master Oct 2, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants