GStreamer update breaks video playback on Arch Linux

[kflak]

Description

After the update to GStreamer 1.20.0-1 I am no longer able to play back video on any of my Arch Linux systems. I suspect that this will soon render any video work on Linux impossible as soon as the other distros catch up 😬

Expected Behavior

To be able to play back video as described in this tutorial

Current Behavior

Getting a white window and the error:

(Processing core video:105730): GStreamer-Base-CRITICAL **: 16:34:27.150: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaap
ipostproc0:sink

Steps to Reproduce

1. Run this code from the Processing IDE or from Nvim with the vim-processing plugin:

import processing.video.*;

Movie movie;

void setup(){
  size(320, 240);
  movie = new Movie(this, "sanskarpromo.mp4");
  movie.play();
}

void movieEvent(Movie movie){
  movie.read();
}

void draw(){
  image(movie, 0, 0);
}

Your Environment

• Processing version: 1280

• Operating System and OS version: Linux t480s 5.15.2130.realtime1-1-rt-lts Java 11, OpenJDK, ANTLR 4, and Travis benfry/processing4#1 SMP PREEMPT_RT Thu, 10 Feb 2022 15:14:13 +0000 x86_64 GNU/Linux

• Other information: Gstreamer v. 1.20.0-1.

• See also this issue I opened for openFrameworks: gstreamer 1.20.0 breaks video on Linux openframeworks/openFrameworks#6871. Same backend, same problem... Had to roll one of my systems back to early February and gstreamer 1.18 to solve the issue.

Possible Causes / Solutions

[benfry]

Another reason we (usually) include the binaries with the library, because we want the video library to work out of the box without users needing to install or understand dependencies. I wish they hadn't been removed on Linux, hopefully we can get that sorted for a future release.

And of course, we need to deal with whatever changes are necessary for 1.20, but since nobody is actively maintaining this library, it would be better to have the old libraries in there so that things continue working in the meantime.

[kflak]

Thanks for the quick answer! I have flagged this to the aur maintainers, hope they will be able to offer a compatible gstreamer package. Would help out my openFrameworks situation as well 🙂

[letorbi]

Hello,

I am the maintainer of the Processing 4 packages in the Arch User Repository (AUR). I've tested the code example, but was not able to reproduce the error. I've checked with Processing 4.0b5 (1280) and 4.0b6 (1281), both installed via the AUR package.

The only thing, that was different from the example, was the video-file, so I assume the video causes the GStreamer issue.

@kflak Could you provide a video file that causes the error? And could you test your example with the official package? I would be great to know whether the error happens with the official version as well or not.

@benfry I totally see, why you bundle certain software with the official release, but these bundles also provide a big overload to the package. Right now only JDK and ffmpeg have been removed, which reduces the package size from 218 MB to 51 MB. I only remove bundled software, if its major and minor version numbers are equal to the ones provided by the Arch Linux package. Shouldn't this be enough to avoid conflicts?

Regards,
Torben

[kflak]

Hi @letorbi, I haven't had the chance to test this on the system with gstreamer 0.18, but on the one with 0.20 I get the same error when using the official package:

Processing video library using GStreamer 1.20.0

(Processing core video:57306): GStreamer-Base-CRITICAL **: 15:08:40.280: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

This happens to both .mp4 and .mov files, that play back flawlessly in my other video apps (mpv, mainly), but not in openFrameworks and Processing. Will look into the possibility of sharing them. Do you have any drive I could upload any of these to?

[letorbi]

@kflak Thanks for testing with the official package. You could use a service like WeTransfer to share the video with us.

@benfry If the problem also occurs with the official package, it does not seem to be related to my AUR package. Or am I missing something? I just want to make sure, that my package is not the root of this problem...

[kflak]

OK, put this up on wetransfer together with my minimal test program: https://we.tl/t-mBQmPbEUvJ The link expires in a week. Hope to have some time later this week to test this on my other machine that runs gstreamer 0.18. Unfortunately there are waaaay too many things in my system depending on gstreamer to do a clean downgrade. Any tips on how I could build gstreamer separately and have processing call this binary instead of the systemwide one would be most appreciated. Then I could test and see if this is really a gstreamer issue or something more exotic.

EDIT: Tried building gstreamer myself, but didn't succeed. Have no idea if this could be related to my problems... The failed build outputs this error:

FAILED: subprojects/gst-libav/ext/libav/libgstlibav.so
cc  -o subprojects/gst-libav/ext/libav/libgstlibav.so subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstav.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavprotocol.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavcodecmap.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavutils.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavaudenc.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavvidenc.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavauddec.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavviddec.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavcfg.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavdemux.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavmux.c.o subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavdeinterlace.c.o -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,--start-group -Wl,-soname,libgstlibav.so -Wl,--exclude-libs=ALL '-Wl,-rpath,$ORIGIN/../../../gstreamer/gst:$ORIGIN/../../../gstreamer/libs/gst/base:$ORIGIN/../../../gst-plugins-base/gst-libs/gst/video:$ORIGIN/../../../orc/orc:$ORIGIN/../../../gst-plugins-base/gst-libs/gst/audio:$ORIGIN/../../../gst-plugins-base/gst-libs/gst/tag:$ORIGIN/../../../gst-plugins-base/gst-libs/gst/pbutils' -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gstreamer/gst -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gstreamer/libs/gst/base -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gst-plugins-base/gst-libs/gst/video -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/orc/orc -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gst-plugins-base/gst-libs/gst/audio -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gst-plugins-base/gst-libs/gst/tag -Wl,-rpath-link,/home/kf/build/gst-build/builddir/subprojects/gst-plugins-base/gst-libs/gst/pbutils subprojects/gstreamer/gst/libgstreamer-1.0.so.0.1902.0 subprojects/gstreamer/libs/gst/base/libgstbase-1.0.so.0.1902.0 subprojects/gst-plugins-base/gst-libs/gst/video/libgstvideo-1.0.so.0.1902.0 subprojects/orc/orc/liborc-0.4.so.0.32.0 subprojects/gst-plugins-base/gst-libs/gst/audio/libgstaudio-1.0.so.0.1902.0 subprojects/gst-plugins-base/gst-libs/gst/tag/libgsttag-1.0.so.0.1902.0 subprojects/gst-plugins-base/gst-libs/gst/pbutils/libgstpbutils-1.0.so.0.1902.0 /usr/lib/libavfilter.so /usr/lib/libavformat.so /usr/lib/libavcodec.so /usr/lib/libavutil.so /usr/lib/libglib-2.0.so /usr/lib/libgobject-2.0.so -Wl,--export-dynamic /usr/lib/libgmodule-2.0.so -pthread -lm /usr/lib/libz.so -Wl,--end-group
/usr/bin/ld: subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavaudenc.c.o: in function `gst_ffmpegaudenc_set_format':
/home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavaudenc.c:246: undefined reference to `avcodec_get_context_defaults3'
/usr/bin/ld: /home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavaudenc.c:292: undefined reference to `avcodec_get_context_defaults3'
/usr/bin/ld: /home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavaudenc.c:336: undefined reference to `avcodec_get_context_defaults3'
/usr/bin/ld: /home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavaudenc.c:317: undefined reference to `avcodec_get_context_defaults3'
/usr/bin/ld: subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavaudenc.c.o: in function `gst_ffmpegaudenc_start':
/home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavaudenc.c:197: undefined reference to `avcodec_get_context_defaults3'
/usr/bin/ld: subprojects/gst-libav/ext/libav/libgstlibav.so.p/gstavvidenc.c.o:/home/kf/build/gst-build/builddir/../subprojects/gst-libav/ext/libav/gstavvidenc.c:252: more undefined references to `avcodec_get_context_defaults3' follow
collect2: error: ld returned 1 exit status
[4324/4769] Compiling C object subproje...erver-1.0.so.0.1902.0.p/rtsp-stream.c.o
ninja: build stopped: subcommand failed.

[letorbi]

I was able to run the test program you've provided without any problems on Wayland and X11. I've checked with the official release as well as with the AUR package.

Maybe the problem is a result of your specific system configuration (installed packages, graphics driver, ...just guessing).

Right now, I would say that the problem is not related to the processing AUR package or Processing in general. Since the problem occurred just after a Gestreamer update, it might be an idea to ask the Gstreamer guys for help. Maybe they know about some regressions and/or can fix the bug...

[kflak]

Thank you so much for checking it out, @letorbi! The weird thing is that it happened on both of my laptops, one with an intel iGPU, one with a hybrid AMD/Nvidia gpu. I have an old computer lying around that I might just do a clean reinstall of arch to see if I get the same behavior, or if it is really just something connected to my particular system. I did test it out on wayland (sway) as well and got the same error as on Xorg (bspwm), so it seems to be independent of graphics server. I will try opening an issue at gstreamer and hope they can assist me further.

Closing this due to not being reproducible.

[letorbi]

@kflak I hope you can find the issue. Good luck to you. Just an idea: I was testing on GNOME, so it might be a desktop environment problem.

Regarding the GPU: I've tested on a NVIDIA 1080 with the proprietary driver.

[codeanticode]

Please try this new release of the video library:

https://github.com/processing/processing-video/releases/tag/r9-2.1.0

It bundles an arm64 build of GStreamer 1.16, which should override the system-wide GStreamer that's causing trouble.

[kflak]

Works beautifully! Thank you so much!
Just curious: are you able to reproduce the bug I have with 1.20 on Linux? The next couple of weeks I will have more time to dig into the whole issue around gstreamer on Arch, and it would be helpful to know if it's just the idiosyncracies of my two laptops that are causing trouble, or if there is something else going on.

[kflak]

@kflak I hope you can find the issue. Good luck to you. Just an idea: I was testing on GNOME, so it might be a desktop environment problem.

Regarding the GPU: I've tested on a NVIDIA 1080 with the proprietary driver.

I am using bspwm, but had the same issue on XFCE4 as well. Tried on both a laptop with internal Intel GPU as well as one with and AMD GPU, using proprietary drivers. Haven't tested it yet with NVIDIA, as I am using that laptop for production right now, and keep it rolled back.

[cherio]

A reliably reproducible test case is listed here: openframeworks/openFrameworks#6871 (comment)