Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PipeWire support #554

Open
Be-ing opened this issue Mar 18, 2021 · 29 comments · May be fixed by #692 or #887
Open

PipeWire support #554

Be-ing opened this issue Mar 18, 2021 · 29 comments · May be fixed by #692 or #887

Comments

@Be-ing
Copy link
Contributor

Be-ing commented Mar 18, 2021

PipeWire will soon be replacing both JACK and PulseAudio on Linux starting with Fedora 34 which will be released in a few weeks. Arch currently has packages that make it easy to switch to PipeWire. I am unclear about Ubuntu's plans and just asked them about that.

PipeWire reimplements the JACK and PulseAudio APIs. Currently the PipeWire developer is not recommending to use the new PipeWire API if applications already work well with PipeWire via JACK or PulseAudio. I am not familiar enough with cpal to know, but there might be some benefits to using the PipeWire API instead of using PipeWire via the JACK API.

PipeWire upstream already provides Rust bindings.

I have attempted to test cpal's JACK backend with PipeWire but have not gotten it to work. The pw-uninstalled.sh script sets the LD_LIBRARY_PATH environment variable so PipeWire's reimplementation of libjack is loaded before JACK's original library. I know Rust links statically to other Rust libraries but I am unclear how Rust links to dynamic C libraries. Maybe the LD_LIBRARY_PATH trick does not work with Rust?

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ cargo run --release --example beep --features jack
    Finished release [optimized] target(s) in 0.03s
     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 170, max: 1466015503 }, sample_format: F32 }
memory allocation of 5404319552844632832 bytes failed
fish: “cargo run --release --example b…” terminated by signal SIGABRT (Abort)

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ ~/sw/pipewire/pw-uninstalled.sh
Using default build directory: /home/be/sw/pipewire/build
Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ cargo run --release --example beep --features jack
    Finished release [optimized] target(s) in 0.03s
     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 170, max: 1466015503 }, sample_format: F32 }
memory allocation of 5404319552844632832 bytes failed
fish: “cargo run --release --example b…” terminated by signal SIGABRT (Abort)

Related discussions regarding PipeWire and PortAudio:
https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/130
PortAudio/portaudio#425

@Be-ing Be-ing mentioned this issue Mar 18, 2021
6 tasks
@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

The PipeWire developer is very responsive and working hard on PipeWire every day. If anyone is interested in working on this for cpal, I recommend getting in touch with him through PipeWire's issue tracker and/or #pipewire on Freenode.

@ishitatsuyuki
Copy link
Collaborator

I'm running Arch + pipewire{,-jack,-pulse} and the jack example works here. What distro do you use?

     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 3, max: 4194304 }, sample_format: F32 }

@ishitatsuyuki
Copy link
Collaborator

Oh, actually I made a mistake; the above one is probably using PipeWire's ALSA integration.

When testing JACK you should run with cargo run --release --example beep --features jack -- --jack.

@ishitatsuyuki
Copy link
Collaborator

When I run with pipewire-jack it doesn't crash but doesn't make sound either. Strange.

@ishitatsuyuki
Copy link
Collaborator

Seems that we have the same regex problem as PortAudio. Fix incoming.

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

Hopefully it doesn't take 3 days to figure out how to find and replace a string in Rust like it did for me in C 😆

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

I recommend reading the PipeWire wiki. There is a lot of good information there.

@ishitatsuyuki
Copy link
Collaborator

FYI it's

Some("system:playback_.*"),
and
Some("system:capture_.*"),
. But I'm planning to change to a different logic based on mpv's.

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

Hardcoding using the system ports in JACK is definitely not right and won't work with PipeWire at all. I am wondering why that was done. Was it just a quick hack to get something working or does this reflect a bigger problem in the cpal API?

PortAudio's JACK implementation is quite quirky. Normally JACK applications create all their ports which any application communicating with the server can connect to arbitrarily. But PortAudio's abstractions flip that relationship upside down so PortAudio creates a port when the application asks to start a stream; all of the PortAudio application's ports are not available for other applications to connect to until the PortAudio application opens the stream. Does cpal's API have the same problem?

@ishitatsuyuki
Copy link
Collaborator

ishitatsuyuki commented Mar 18, 2021

As usual, our development effort is limited and the JACK backend is somewhat unfinished. We do care about making the abstractions not leaky, though.

We register a port when you create a Stream... I hope it's fine? You can then start or stop the Stream through the methods, and stopping isn't tied to closing the port.

let port_try = client.register_port(&format!("in_{}", i), jack::AudioIn::default());

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

As long as the application can create JACK/PipeWire ports for other applications or the session manager to connect to without the cpal application necessarily using all of them, I think it can work. There may need to be a mechanism for signaling the cpal application when another application or hardware device has connected to/disconnected from the cpal application's port, but I'm not familiar enough with any of these APIs to be sure about that. That may be related to hotplug support (#373).

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

I have identified at least one good reason for a new PipeWire backend rather than relying on PipeWIre's reimplementation of the JACK API. From the PipeWire wiki:

PipeWire can adapt the latency dynamically, which is important for power usage on a laptop. When low latency is required, the system can switch automatically and seamlessly to smaller buffer sizes.

This means that a CPAL application using PipeWire could request a small buffer size for low latency. When the application goes idle, it could call Stream::pause and yield control of the buffer size back to PipeWire. This would allow keeping a low latency application open in the background, pausing it, then watching a YouTube video in Firefox without having to close any applications or mess with reconfiguring anything. If I force a JACK application using the PipeWire JACK reimplementation to use a small buffer size with the PIPEWIRE_LATENCY environment variable, YouTube videos in Firefox have glitchy audio while the other application remains running.

JACK has an API somewhat like this with int jack_deactivate (jack_client_t * client), but this does not fit with CPAL's abstractions. To use this, CPAL would need to track the state of each Device's streams and call jack_deactivate when all the CPAL Streams for a Device are paused.

Related discussion on the PipeWire issue tracker: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/722

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

@ishitatsuyuki if you or anyone else works on a new PipeWire backend for CPAL, I would be happy to help test it. I am afraid writing it myself would be above my skill level though.

@Ralith
Copy link
Contributor

Ralith commented Mar 18, 2021

It kinda sounds like you have the most domain expertise here. Why not have a go?

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 18, 2021

I think that would be a bit ambitious for a first Rust project.

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 19, 2021

FWIW, here is SDL's PipeWire backend: libsdl-org/SDL#4094

@Be-ing
Copy link
Contributor Author

Be-ing commented Apr 19, 2021

This would be great for Whisperfish!

@kamiyaa
Copy link

kamiyaa commented Jul 19, 2021

Any updates on this?

@ishitatsuyuki
Copy link
Collaborator

As far as I have tested, it seems the JACK backend (+pw-jack) works fine for low-latency use cases on PipeWire. If you have any specific thing missing that requires directly integrating with PipeWire, then please let me know with a comment.

@ethindp
Copy link

ethindp commented Nov 9, 2021

I've been looking at developing a PW backend, but one major problem I see is the main loop.

A PipeWire application requires a main loop object to be created. An application calls the run method, which never returns (it can be terminated but you do it through sources). This would be a major problem with CPALand anything consuming it since it would never allow anything to run beyond the PW main loop. I'm guessingother audio backendsdo this, but I can't seem to find any way of stepping through the loop without using sources (e.g. timers). Thoughts about getting around this?

@ishitatsuyuki
Copy link
Collaborator

I don't really know, but can you run that loop on another thread? It's common to create additional helper threads in cpal backends.

@ethindp
Copy link

ethindp commented Nov 9, 2021

@ishitatsuyuki Yep, that's exactly what the PipeWire documentationstates:

The pipewire library is not really thread-safe, so pipewire objects do not implement Send or Sync.
However, you can spawn a MainLoop in another thread and do bidirectional communication using two channels.
To send messages to the main thread, we can easily use a std::sync::mpsc. Because we are stuck in the main loop in the pipewire thread and can’t just block on receiving a message, we use a pipewire::channel instead.

I'll look at doing that -- I didn't think of that lol.

@Be-ing
Copy link
Contributor Author

Be-ing commented Nov 30, 2021

With RustAudio/rust-jack#154 I confirm that the cpal examples work with PipeWire's implementation of JACK.

The enumerate example is only listing the ports cpal crates. I don't know if that's an issue with the enumerate example or cpal:

cpal on  HEAD (59ee8ee) [!] is 📦 v0.13.4 via 🦀 v1.56.1 
❯ cargo run --release --example enumerate --features jack
    Finished release [optimized] target(s) in 0.05s
     Running `target/release/examples/enumerate`
Supported hosts:
  [Jack, Alsa]
Available hosts:
  [Jack, Alsa]
JACK
  Default Input Device:
    Some("cpal_client_in")
  Default Output Device:
    Some("cpal_client_out")
  Devices: 
  1. "cpal_client_in"
    Default input stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported input stream configs:
      1.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    Default output stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported output stream configs:
      1.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
  2. "cpal_client_out"
    Default input stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported input stream configs:
      2.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    Default output stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported output stream configs:
      2.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
ALSA
  Default Input Device:
    Some("default")
  Default Output Device:
    Some("default")
  Devices:
...

@Be-ing
Copy link
Contributor Author

Be-ing commented Dec 13, 2021

#624 fixes the build using Pipewire for the jack feature.

@m00nwtchr
Copy link

m00nwtchr commented Mar 9, 2022

I'm started working on a PipeWire host here: https://github.com/m00nwtchr/cpal/tree/pipewire-host

Any help/thoughts would be appreciated.

I think I'm fairly far along, but I can't find any way aside from the PipeWire Stream api to actually write/read data, and I'm not sure if my node creation code is correct.

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 9, 2022

Cool! Could you make a draft pull request?

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 9, 2022

I can't find any way aside from the PipeWire Stream api to actually write/read data

The pw_filter API does not have Rust bindings yet. This is in progress: https://gitlab.freedesktop.org/pipewire/pipewire-rs/-/merge_requests/112

@m00nwtchr
Copy link

m00nwtchr commented Mar 9, 2022

Is pw_filter needed for this, though?

https://docs.pipewire.org/page_streams.html

For more complicated nodes such as filters or ports with multiple inputs and/or outputs you will need to use the pw_filter or make a pw_node yourself and export it with pw_core_export.

Also, opened the draft PR: #651

@Be-ing
Copy link
Contributor Author

Be-ing commented Mar 9, 2022

If you're creating pw_nodes manually then I don't think you strictly need pw_filter, but my understanding is that using pw_filter would be easier to implement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants