Support Screen capture

[dkubrakov]

No description provided.

[dkubrakov]

Working example flutter-webrtc:
https://github.com/dkubrakov/flutter-webrtc/tree/feat/screen-capture

[cloudwebrtc]

hey @dkubrakov, that‘s awesome

Working example flutter-webrtc:
https://github.com/dkubrakov/flutter-webrtc/tree/feat/screen-capture

I tested this branch and it works, we need to return the thumbnail in Source, the underlying processing should be after getting the source list, applying to get a frame RTCVideoFrame for each source, and then get to the thumbnail.

I'm implementing DesktopCapture for macOS on another branch, https://github.com/webrtc-sdk/webrtc/blob/feat/support-screen-capture-for-mac/sdk/objc/components/capturer/RTCDesktopCapturer.h

I think we can synchronize and keep consistency at the dart level, and complete this feature.
Any idea can be discussed

[dkubrakov]

At the moment, that's all.
After a while I will be able to return to this task (return the thumbnail in Source).
As an interface, I was targeting flutter-webrtc/flutter-webrtc#799.

https://github.com/dkubrakov/webrtc-interface/blob/feat/desktop-cpaturer/lib/src/desktop_capturer.dart
https://github.com/dkubrakov/flutter-webrtc/tree/feat/screen-capture
https://github.com/dkubrakov/libwebrtc/tree/feat/screen-capture

[dkubrakov]

Updated the release of screen capture, taking into account the upgrade to m97.
What prevents (still needs to be done) to add this feature?

• There is a small difference in libvwebrtc (class RTCDesktopCapturer), how critical it is...

https://github.com/dkubrakov/libwebrtc/tree/feat/screen-capture
https://github.com/dkubrakov/flutter-webrtc/tree/feat/screen-capture
https://github.com/dkubrakov/webrtc-interface/tree/feat/desktop-cpaturer

[cloudwebrtc]

@dkubrakov great work, I think we can merge after syncing both PRs, here I have some new changes.

1, I moved webrtc_interface/lib/src/desktop_capturer.dart to flutter_webrtc/lib/src/desktop_capturer.dart, because dart_webrtc uses getDisplayMedia to pop up screen/window selection dialog via browser api. so desktopCapture is only for desktop.

here is the PR for macOS.
https://github.com/flutter-webrtc/flutter-webrtc/pull/940/files

So we need to merge these two files.

• desktopcapturer_impl.dart@dkubrakov
• desktop_capturer_impl.dart@cloudwebrtc

2, I implemented full screen/window capture in macOS and supports thumbnail export, so I think you can refer to this code to generate thumbnails for windows part. (use libjpeg)

https://github.com/webrtc-sdk/webrtc/pull/24/files

I refer to electron and chromium to implement the DesktopMediaList, and DesktopMediaList::MediaSource interface. Use a generic ObjCDesktopCapturer class to capture Screen or Window.

objc_desktop_capture.h
objc_desktop_capture.mm
objc_desktop_media_list.h
objc_desktop_media_list.mm

BTW, If you need more timely discussion, you can add my Telegram ID. https://t.me/cloudwebrtc

[dkubrakov]

Hello, @cloudwebrtc!

Fixed 1'st remark, also renamed to desktop_capturer_impl.dart.

https://github.com/dkubrakov/flutter-webrtc/blob/feat/screen-capture/lib/src/native/desktop_capturer_impl.dart

I did the thumbnail generation at the plugin level. This is much less expensive than implementing the DesktopMediaList, DesktopMediaList::MediaSource interface.

https://github.com/dkubrakov/flutter-webrtc/blob/feat/screen-capture/common/cpp/src/flutter_screen_capture.cc

Can we use it like this?

[cloudwebrtc]

hey @dkubrakov, looks great, can you create a new PR for flutter-webrtc? I think we will be able to merge soon

I did the thumbnail generation at the plugin level. This is much less expensive than implementing the DesktopMediaList, DesktopMediaList::MediaSource interface.

At present, it is indeed possible to make consistent calls between windows and mac.

My intention to create thumbnails with libjpeg is for the upcoming `flutter Linux version, so we can make the Linux version achieve exactly the same screen-sharing functionality with almost no code changes, the Windows API is used in your PR, so this code will also need to be rewritten on the Linux platform.

I have committed the relevant common code to this branch
https://github.com/webrtc-sdk/libwebrtc/compare/feat/desktop-capture

would you mind merging this code into your branch to continue working on your PR? I guess this can reduce some workload.

[cloudwebrtc]

Or you can create a flutter-webrtc PR first, I will merge it after the test is ok They also include this PR, then I will replace the libwebrtc part of the code with my PR, it will be a perfect feature for us to work together.

[dkubrakov]

of course, it is here )

flutter-webrtc/flutter-webrtc#984