Remove background from Unity Player

[Ahmadre]

At first: thank you sooo much for all the work and super good documentation ❤️ ❤️ ❤️ :)

Second: Let's be honest, if you're a flutter developer and want to just render an object for your main-flutter application, you'll come to the problem, where you got the ugly background in your unity-player.

Example:

I am googling and researching over weeks to get my background transparent, so I can use my shown car without the background.

Settings I've tried:

1. Activate: Render over Native UI (also known as: Preserve framebuffer alpha)
2. Set my main Camera to: Clear Flags: Solid Color and the related color to: RGBA: {0,0,0,0} (so black-alpha!)

I also just tried to set Clear Flags to Depth only, but that leads to the ugly brown background.

So: what is the sense of Unity's Export Flutter function, when we cannot render our objects without a unity background?

I would really appreciate any help :)

[TarekMedhat]

I don't know to make a transparent background in unity but I'm wondering if as you said,"you just want to render an object for your main-flutter application".
Why don't you just resize your UnityView widget instead of taking the fullscreen to just match the area you want to show your object in, with a background color same as your flutter page background color.
Something like that: the small black box showing your UnityView bounders.

Would that be possible?

[Ahmadre]

@TarekMedhat I already did that, but I got a complex application, where I got a Gradient background and I want to use the 3D-Model in many pages and/or Hero-Widgets. So it doesn't solve the cause problem :/. But maybe I am just a unity newbie and am missing something here?

[thomas-stockx]

Hi @Ahmadre, Unity is not really the best tool for the job for you.
There's no such thing as "removing the background", as it's very much part of a Unity environment.

It seems like you're looking for an 3D object viewer instead of a full embedded 3D game engine.
Something like this, this, this, or this perhaps?

[Ahmadre]

Hi @Ahmadre, Unity is not really the best tool for the job for you.
There's no such thing as "removing the background", as it's very much part of a Unity environment.

It seems like you're looking for an 3D object viewer instead of a full embedded 3D game engine.
Something like this, this, this, or this perhaps?

@thomas-stockx I already tried every package you've linked ^^.

My Model can't be rendered in these packages, because of the complex materials & shaders.
If someone ever exported a full 3D-Model from unity to view it with the linked packages from @thomas-stockx then I really would appreciate any hint for that.

Edit: None of the above packages let me transform my model partly (like opening the drivers door), but with Unity I am able to send command to my model

[juicycleff]

@Ahmadre Actually the project that lead me to write this library had a similar request and here is how I solved this problem. I had to pass the color of my current screen as a prop to the unity game engine and changing the entire unity background dynamically and it worked like charm. No one could tell the difference between unity and flutter. Hope this helps

[juicycleff]

If this solves your issue please close it :)

[Ahmadre]

@juicycleff sounds great! Do you have any code example or reference where i can look that up? What color do I have to change and what settings do I have to set to dynamically change the color? You can also dm me if you like: [email protected] or on twitter: @4hm4dr3

Thx :)

[Ahmadre]

Ok, solved by dynamically setting the color:

public Camera cam;

    void Start()
    {
        cam = GetComponent<Camera>();
        cam.clearFlags = CameraClearFlags.SolidColor;
    }

    void Update()
    {
    }

    public void SetBackgroundColor(string rgbString)
    {
        string[] part = rgbString.Split(',');
        Color color = new Color(float.Parse(part[0]), float.Parse(part[1]), float.Parse(part[2]));
        cam.backgroundColor = color;
    }

[nmfisher]

This is possible:

(AvatarWidget contains the UnityWidget).

Updated instructions (Android/iOS)

On 6e55f08:

1. Use Unity 2021.1.0b11 (2020.3.0f1 should also work) [0]
2. Ensure "Render over Native UI" is checked and your Camera Clear flags are set to Solid Color with alpha 0 BLACK (white will not work)

(Android only)

3 Add android:theme="@style/UnityThemeSelector.Translucent" to the element in your android/unityLibrary/src/main/AndroidManifest.xml for your application:

<application android:theme="@style/UnityThemeSelector.Translucent">

4. Clone the flutter-unity-view-widget repository and modify android\src\main\kotlin\com\xraph\plugin\flutter_unity_widget\UnityPlayerUtils.kt:

to add this:

import android.view.Surface;

here
and this:

val f = UnityPlayer::class.java.getDeclaredField("mGlView")
f.isAccessible = true
val v = f.get(unityPlayer) as SurfaceView
unityPlayer!!.removeView(v);
v.setZOrderOnTop(false);
unityPlayer!!.addView(v);

here.

Modified file here.
5. Modify your pubspec.yaml to use the cloned flutter_unity_widget rather than the published package/GitHub repository.

IMPORTANT - this is a temporary fix only. The flutter_unity_widget plugin itself should implement this workaround, or you should implement it by subclassing your own OverrideUnityActivity and moving the logic there. However, given #286, running from a separate activity doesn't actually seem to work. I'll wait for #286 to be resolved before providing a more permanent solution.

[0] unfortunately there's a separate bug with loading native libraries in these versions of Unity, so until that's fixed, you'll need to follow the workaround @ #329 (comment)

Also haven't tried on iOS yet, I think there's a separate workaround needed there too.

This worked as-is for me on iOS.

EDIT 1:

Actually I just realized that even though the transparency works on Android, the Unity view is rendered above the entire Flutter layer (so widgets logically higher in the Z-index are rendered behind the Unity layer). This isn't a problem on iOS.

So far it's either render with no transparency (and have the layers correct), or render with transparency (and have the layers incorrect). Investigating further.

EDIT 2:

(Partial) success! After 12 hours of butting my head against the wall, this is possible on Android too:

However I only managed to hack this in by patching the Unity bytecode. Now I need to figure out a more elegant solution.

EDIT 3:

Ok, fixed. We need to grab the SurfaceView created by UnityPlayer and call setZOrderOnTop(false). Updated instructions above.

EDIT 4:

Some further weirdness on Android - when the UnityWidget is in the foreground, any changes to the background (colour/animation/opacity/etc) won't be visible until the UnityWidget is removed and re-inserted.
This is what it should look like (from iOS where everything works fine):

IMG_3392.mp4

and this is what it looks like on Android:

device-2021-03-20-000213.mp4

Given it's fine on iOS, I'm assuming it's not a Flutter compositor issue and it's a quirk of working with Android SurfaceView. I know others have mentioned rendering to a TextureView instead, though that comes with performance issues.

EDIT 5:

OK, so I've managed to fix this on Android too - but at a cost:

device-2021-03-20-152057.mp4

So this is now working, but with a considerable drop in framerate. I did this by forcing the UnityPlayer to render to a TextureView rather than a SurfaceView:

            if (unityPlayer != null && !reInitialize) {
                callback?.onReady()
                return
            }

            try {
                Handler(Looper.getMainLooper()).post {
                    if (!reInitialize) {
                        activity.window.setFormat(PixelFormat.RGBA_8888)
                        unityPlayer = UnityPlayer(activity, ule)
                        val view = TextureView(activity);
                        view.setOpaque(false)
                        view.setSurfaceTextureListener(object : TextureView.SurfaceTextureListener{
                            override fun onSurfaceTextureAvailable(surface:SurfaceTexture,  width:Int, height:Int) {
                                unityPlayer!!.displayChanged(0, Surface(surface))
                            }

                            override fun onSurfaceTextureDestroyed(surface:SurfaceTexture ) : Boolean {
                                return true
                             }

                            override fun onSurfaceTextureSizeChanged(surface:SurfaceTexture , width:Int, height:Int) {
                            }

                            override fun onSurfaceTextureUpdated(surface:SurfaceTexture) {
                            }
                        });
                        unityPlayer!!.addViewToPlayer(view, true)
...

I'm not really familiar with Android rendering internals, but I'm guessing that the original problem is something to do with the Flutter compositor not playing nicely with the additional Window created with a SurfaceView (which I believe uses the system compositor), meaning that the layer is only ever rendered with the first background (i.e. "Flutter") frame from when the SurfaceView was attached. A TextureView does not create an additional Window, meaning it is composited directly by Flutter (but with the noticeable drop in performance).

This seems to be an inherent limitation of TextureView, so I'm not going to play around any further. For now, I'm going to stick with the workaround - use the SurfaceView method (above), don't use an animated backgrounds, and if I need to change the background, remove the UnityWidget from the Flutter render tree first, change the background, then reinsert. I will open an issue with the Flutter engine team, though, to see if there's any way of properly rendering the background (Flutter) layers with a SurfaceView.

[kerwanp]

The solution @nmfisher provided looks obsolete.

On Unity 2020.3.32f1 even with Render over native UI checked, my background is still not transparent.

Do you have any update ?

[lxp-git]

The solution @nmfisher provided looks obsolete.

On Unity 2020.3.32f1 even with Render over native UI checked, my background is still not transparent.

Do you have any update ?

😭We cant set background to transparent since this lib set the container background color to white by this line

flutter-unity-view-widget/android/src/main/kotlin/com/xraph/plugin/flutter_unity_widget/FlutterUnityWidgetController.kt

Line 60 in 7995993

view.setBackgroundColor(Color.WHITE)

[timbotimbo]

I just tested @nmfisher ’s transparency instructions on a more recent setup, and it seems to be a little easier now.

Demo

In this video the CircularProgressIndicator is behind Unity, and the speed slider is above Unity.
This is the desired behaviour as far as I understand.

There is a black flash while Unity is launching, but I assume this is a Flutter bug. (I get the same in google maps).

Screen_Recording_20221030-234819.mp4

Versions

• Unity 2021.3.6f1
• Flutter 3.3.5
• flutter_unity_widget 2022.2.0
• Devices: Android 7 & 12, iPadOS 16.1

Instructions

(iOS only needs step 1 & 2)

1. Unity Player settings: Enable 'Render over Native UI'.
2. Unity scene: Set camera clear flags to 'Solid Color', black with alpha 0.

(Android only from here on)

3. Make sure the android style has transparency.
   I simply added transparency to the current example to remove a black background. There is probably a cleaner way to do this.

example/android/app/src/main/res/values/styles.xml

        <!-- 
        ...
	This Theme is only used starting with V2 of Flutter's Android embedding. -->
 <style name="NormalTheme" parent="@android:style/Theme.Black.NoTitleBar">
        <!-- <item name="android:windowBackground">@android:color/white</item> -->
        <item name="android:windowBackground">@android:color/transparent</item>
        <item name="android:windowIsTranslucent">true</item>
 </style>

4. Edit the flutter_unity_widget plugin: Change the white colour to transparent in FlutterUnityWidgetController.kt

view.setBackgroundColor(Color.TRANSPARENT)   // view.setBackgroundColor(Color.WHITE)

5. Back to your flutter project.
   If you use useAndroidViewSurface: false in the UnityWidget, this is all that is needed.
   You should now be able to replicate the video above.

6. If you need Hybrid composition (useAndroidViewSurface: true), you will probably need to edit the z-order in UnityPlayerUtils.kt. nmfisher managed to do this in the comment above.
   I did not get this one fixed yet.

Other

I did not have to adjust anything in unityLibrary/src/main/AndroidManifest.xml.
Unity seems to apply a transparent theme automatically. Which makes adding <application android:theme="@style/UnityThemeSelector.Translucent”> redundant.

[fiercepark]

@timbotimbo
It works on version 2021.3.15f1.
**note: I also tried this solution on unity editor 2022.x and it did not work.

[timbotimbo]

@fiercepark

**note: I also tried this solution on unity editor 2022.x and it did not work.

I just tried exporting with 2022.1.23f1, and got it to work following the steps above. Don't know about possible bugs in earlier versions, but it looks like it is still comaptible.

[fiercepark]

@timbotimbo

As with the other issue we covered, I am reproducing the problem on the unity-2022.2.0b16[silicon beta] of the m1 chip device.

[BranislavMateas]

Hey!

Wasn't able to make it work at first with @timbotimbo instructions at first.

It is important to look at your styles.xml file. As it seems, in newer Flutter versions, when creating new project, there is not just one styles.xml. One is located in the 'values' directory and other one in 'values-night'.

To make this work correctly I had to delete the whole 'values-night' folder.

Not only that but also watch out for the NormalTheme. If you deleted the 'values-night' folder before, now in your 'values' directory - inside style.xml file should now see:
<style name="NormalTheme" parent="@android:style/Theme.Light.NoTitleBar">
<item name="android:windowBackground">?android:colorBackground</item>
</style>

Rename the parent="@android:style/Theme.Light.NoTitleBar" to parent="@android:style/Theme.Black.NoTitleBar".

In my case I also renamed the LaunchTheme from Light to Black as well.

Happy coding! (and transparency 😁)

[timbotimbo]

Based on a recent test:

If you want to do this with URP, you also need to disable both post processing and HDR on the camera.

[emileswain]

Hey!

Wasn't able to make it work at first with @timbotimbo instructions at first.

It is important to look at your styles.xml file. As it seems, in newer Flutter versions, when creating new project, there is not just one styles.xml. One is located in the 'values' directory and other one in 'values-night'.

To make this work correctly I had to delete the whole 'values-night' folder.

Not only that but also watch out for the NormalTheme. If you deleted the 'values-night' folder before, now in your 'values' directory - inside style.xml file should now see: <style name="NormalTheme" parent="@android:style/Theme.Light.NoTitleBar"> <item name="android:windowBackground">?android:colorBackground</item> </style>

Rename the parent="@android:style/Theme.Light.NoTitleBar" to parent="@android:style/Theme.Black.NoTitleBar".

In my case I also renamed the LaunchTheme from Light to Black as well.

Happy coding! (and transparency 😁)

I found that if I just applied the following to example/android/app/src/main/res/values/styles.xml that i would still get a black background.

<item name="android:windowBackground">@android:color/transparent</item>
<item name="android:windowIsTranslucent">true</item>

Applying the above to the example/android/app/src/main/res/values-night/styles.xml as well fixed the issue. I didn't delete the values-night folder.

Note that i'm exporting from Unity 2022.3.22f1 with the cameras background set to black transparent. The Unity project was created using the URP demo project with the Flutter Unity package added.