[rpi4] 5.10.52-v7l+ Cma allocation issues after kernel upgrade (“[drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA”)

[Drag0nFly]

After upgrading to kernel 5.10.52-v7l+ (w/ no other changes performed) the system no longer functions. It is able to boot, display output/splashscreen on HDMI and then subsequently the screen goes black (but still with an active signal)

Looking at the kernel log it is being flooded with:

[Sun Aug  1 15:03:02 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Sun Aug  1 15:03:02 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Sun Aug  1 15:03:02 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Sun Aug  1 15:03:02 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Sun Aug  1 15:03:02 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Sun Aug  1 15:03:02 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Sun Aug  1 15:03:02 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Sun Aug  1 15:03:02 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Sun Aug  1 15:03:02 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)

It appears that the CmaFree is now set to zero kB available, causing all kinds of gfx issues:

grep Cma /proc/meminfo
CmaTotal:           8192 kB
CmaFree:               0 kB
pi@playbox4 ~>
pi@playbox4 ~> uname -a
Linux playbox4 5.10.52-v7l+ #1440 SMP Tue Jul 27 09:55:21 BST 2021 armv7l GNU/Linux
grep Cma /proc/meminfo
CmaTotal:           8192 kB
CmaFree:               0 kB

Downgrading to the previously-used kernel 5.10.17-v7l+ (again, without any other changes), restores functionality. Emulationstation starts as usual, with no errors from vc4-drm logged, and CmaFree set to 212880kB.

uname -a
Linux playbox4 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
pi@playbox4 ~>
pi@playbox4 ~> grep Cma /proc/meminfo
CmaTotal:         262144 kB
CmaFree:          212880 kB

[popcornmix]

Do you have a gpu_mem setting? Does removing it help?
Are you using kms driver?
Does changing

dtoverlay=vc4-kms-v3d

to

dtoverlay=vc4-kms-v3d,cma-256

help?

[Drag0nFly]

gpu_mem is used, as it was necessary for performance reasons in certain emulators and also (at least at some point) for Kodi for 4K playback.

The fake KMS (fkms) driver is used for dispmanx support.

Appended 'cma-256' to the dtoverlay parameter. The problem still persists.

/proc/meminfo now reports 8192kB total, with only 1320kB available

grep Cma /proc/meminfo
CmaTotal:           8192 kB
CmaFree:            1320 kB

grep cma /boot/config.txt
dtoverlay=vc4-fkms-v3d,cma=256
dtoverlay=vc4-fkms-v3d,cma=256

vcgencmd get_mem gpu
gpu=512M

I find no documentation for the "cma" setting to dtoverlay. Does it perhaps need a unit parameter appended to it (e.g "m") ?

Excerpt from dmesg–

sudo dmesg -T|tail
[Mon Aug  2 15:38:25 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Aug  2 15:38:25 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Mon Aug  2 15:38:25 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Aug  2 15:38:25 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Mon Aug  2 15:38:25 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Aug  2 15:38:25 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Mon Aug  2 15:38:25 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Aug  2 15:38:25 2021] vc4-drm gpu: [drm]                           dumb:      4kb BOs (1)
[Mon Aug  2 15:38:26 2021] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Aug  2 15:38:26 2021] vc4-drm gpu: [drm]

Again, reverting to the previous kernel restores functionality (with the same change to dtoverlay). Note the log entry for cma.

uname -a
Linux playbox4 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
pi@playbox4 ~>
pi@playbox4 ~>
pi@playbox4 ~> grep cma /boot/config.txt
dtoverlay=vc4-fkms-v3d,cma=256
dtoverlay=vc4-fkms-v3d,cma=256

grep Cma /proc/meminfo
CmaTotal:         262144 kB
CmaFree:          229132 kB

sudo dmesg -T|grep cma
[Mon Aug  2 15:44:27 2021] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[Mon Aug  2 15:44:27 2021] Memory: 7359244K/7798784K available (10240K kernel code, 1358K rwdata, 3160K rodata, 2048K init, 890K bss, 177396K reserved, 262144K cma-reserved, 7274496K highmem)
[Mon Aug  2 15:44:31 2021] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[Mon Aug  2 15:44:31 2021] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[Mon Aug  2 15:44:31 2021] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.

vcgencmd get_mem gpu
gpu=512M

As a side-note: is it possible to now convert the /boot part to proper ext4 (in order to avoid 'Adding diversion of ') which takes forever to process (since kernels have to be shuffled around).

[popcornmix]

gpu_mem is used, as it was necessary for performance reasons in certain emulators and also (at least at some point) for Kodi for 4K playback.

I can't imagine any way emulators will benefit from gpu memory on pi4 (the opengl is on arm side and uses arm memory).
4K hevc playback will use cma not gpu memory.

LibreELEC 10 (kodi matrix) ships with gpu_mem set to default (76M) and handles 4K playback fine.
The previous kodi (leia) may need a little more gpu memory for h264, but I'd expect 128M to be more than enough.

[Drag0nFly]

This was for DreamCast / Redream. Several sources mentioned increasing the gpu_mem value–https://www.reddit.com/r/RetroPie/comments/cjcrnk/pi4_overclock_vs_stock_dreamcast/

For Kodi (just a quick search); https://forum.libreelec.tv/thread/23739-no-4k-playback-via-sony-str-dn1050-on-4k-beamer/

Whether that is still accurate I do not know. In any case, I have 8Gb available on the Pi, so it also made sense to reserve some of it for GPU usage.

[popcornmix]

Note, both cma and gpu_mem must come from bottom 1GB of sdram. Having 8GB doesn't help that.

The LE thread is valid for LE 9/kodi 18 (Leia) which still uses the 5.4 kernel (where hevc allocates from gpu_mem).
LE 10/Kodi 19 (Matrix) uses the 5.10 kernel (as you are) and uses the default gpu_mem.

And don't trust every random reddit user's settings. I'm sure that gpu_mem setting is hurting more than helping.

[Drag0nFly]

To clarify, those sources were not the basis for using this setting. I simply provided those as I don't have the other sources immediately available. Many were from YouTube, and I did do testing at the time which revealed better performance on DreamCast.

Since cma and gpu_mem are correctly allocated with the previous kernel, should't the focus be on why the newer kernel impacts core functionality due to this setting?

I can't be the only one who has increased gpu_mem from 76M.

[popcornmix]

The reason is simple. The default cma has been increased, because that gives some actual benefits (like being able to decode 4K hevc video, or getting smoother desktop performance).
And as said, gpu_mem+cma+firmware+kernel must live in bottom 1GB of sdram.

We are looking to add a workaround, so you don't get the higher cma default if you have an unnecessarily large gpu_mem, but that is not the best solution to the problem.

[JamesH65]

How about giving the gpu_mem value what is left after you take away all the important stuff (up to a certain limit)

i.e. gpu_mem = min(total - kernel - cma, min(gpu_mem, 256))

[Drag0nFly]

I see, thanks for the clarification. Perhaps the Wiki page (https://www.raspberrypi.org/documentation/configuration/config-txt/memory.md) should also be updated to better specify this, as it still appears that 512 is a supported gpu_mem size–

“The memory allocated to the GPU is used for display, 3D, Codec and camera purposes as well as some basic firmware housekeeping. The maximums specified below assume you are using all these features. If you are not, then smaller values of gpu_mem can be used.”

The recommended maximum values are as follows:

total RAM	gpu_mem recommended maximum
256MB	128
512MB	384
1GB or greater	512, 256 on the Pi4

I have not tested with 256.

I also notice further below, which is in line with what you write above:

“On the Raspberry Pi 4 the 3D component of the GPU has its own memory management unit (MMU), and does not use memory from the gpu_mem allocation. Instead memory is allocated dynamically within Linux. This allows a smaller value to be specified for gpu_mem on the Pi 4, compared to previous models.”

This change has been present on my Pi4 for basically its entire lifetime, as Redream was one of the earliest emulators I tested.

[JamesH65]

The docs repo is locked at the moment, but will be updated once whatever is decided here is fixed.

[dankcushions]

This was for DreamCast / Redream. Several sources mentioned increasing the gpu_mem value–https://www.reddit.com/r/RetroPie/comments/cjcrnk/pi4_overclock_vs_stock_dreamcast/

To clarify, those sources were not the basis for using this setting. I simply provided those as I don't have the other sources immediately available. Many were from YouTube,

i would extremely disregard reddit and youtube for retropie/emulation information, and stick to the official forums/docs. eg: https://retropie.org.uk/docs/Memory-Split/#raspberry-pi-4

(i am not sure about kodi's relationship with gpu_split, other than what we sourced here)

[popcornmix]

I think retropie is out of date with this comment:

The FKMS driver still utilises the split for certain media tasks, e.g., hardware video decoding for 4K (HEVC) video.

if they have switched to 5.10 kernel. That is no longer true and hevc buffers are allocated from CMA.

They are completely correct with:

there is no performance advantage from specifying values larger than is necessary, and in fact it can harm performance.