-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
audio: continuous DMA from AudioIn, real-time audio filtering #2676
Comments
Mozilla WebAudio API covers a large portion of what you would like to see working in CircuitPython A CircuitPython port of this API would be able to bridge the gap between existing code in the web and microcontrollers.While looking into porting some of the code that uses the Biquad Filter (potentially useful for the Equaliser for WinAmp), The next port of call was the newly updated Arduino Core (for which the SiLabs CP210x Serial Drivers had to be updated) which was able to upload sketches in bootloader mode with the microUSB on the mezzanine (with their firmware upload tool and some custom written batch files since the Arduino "burn bootloader" button appeared to be broken). Spresense Upload Syntax: I wrote a simple starter sketch replicating a retro TV Beep that counts down before shows used to go on air. I think the Spresense board with some tweaks could be a really good DSP Accelerator for other CircuitPython boards, More details about what I found trying to get the headphone jack working are included as comments in the Arduino Sketch attached below in Arduino Sketch and Audio Samples included: SpresenseTVBeep.zip |
@TheMindVirus I'd recommend adding an issue for supporting the existing audio apis on spresense too. |
I support this proposal; it would be really useful to be able to use e.g. the M4 CPU power to do FFT, digital filters etc. in a background process from CircuitPython without needing to go back to lower-level programming environments. For example, people are doing Software-defined Radio (SDR) stuff on ATmega328Ps (see here). So being able to at least do filtering of audio frequencies (low-pass, band-pass, high-pass), amplitude modification, amplitude modulation, impuls shaping would be great. From an architectural perspective and without having looked at available libraries, I think it would be sufficient to
Does that sound realistic? And could we get this from "some distant future feature" to "near-term priorities"? |
@jepler wondering what is your latest thinking on this? Recently I opened #9225 with an eye towards DSP sorts of things but as I become more familiar with the codebase I am realizing that really the thing is to do is to integrate input from built-in, I2S, and even SPI ADCs into the |
I've prototyped on rp2040 a set of minimal, non-breaking changes which enable processing of live sampled audio. With these changes, the following code will sample audio from import analogbufio
import array
import audiocore
import audiopwmio
import board
buffer = array.array("H", [0x0000] * 8)
adc = analogbufio.BufferedIn(board.A0, sample_rate=10000)
adc.readinto(buffer, loop=True)
pwm = audiopwmio.PWMAudioOut(board.D12)
sample = audiocore.RawSample(buffer, sample_rate=10000, single_buffer=False)
pwm.play(sample) Changes required:
With this in place it should be straightforward to add features like multi-channel sampling using the ADC round-robin registers, and to implement DSP via functions which take in a buffer and return a processed buffer. Wondering where this might fit on the development roadmap, let me know if you'd like me to tidy it up and submit a PR. |
@timchinowsky This is very nice! Would it also work with audiobusio? |
It should work anywhere that uses the audio buffer protocol - basically it tricks a |
Does this idea pave the way to doing any processing on the audio data before it's output, such as applying a digital filter? |
@jepler yes, by composing the |
I'm happy to do the reviews for this whenever you are ready. Folks would love to see this I'm sure. |
Right now in rp2040 audio DMA in |
thats a cute I2S codec, i ordered some samples of the TAC5242 |
I've got ~5 populated breakout boards for the TAC5212 that are up for grabs, if anyone's interested in messing with it. |
Would it be enough for us to coordinate internally so that the ADC and PWM output dma's use the same pacing timer when set to the same rate? I don't really like the special value. Instead, I'd just pass the object you want to tie to. |
That would be better, it's just more code. Using a magic number is a ~two-line change. |
Re: the TAC5212, these things are great, up to 768 kHz sampling at up to 32 bits, and you can daisy-chain the devices to get up to 8 channels over one pin. To do them justice, planning to use |
Note that this isn't portable. An I2SIn would be better because it could be implemented on other platforms. RP2 can use PIO internally.
More code is fine. Magic numbers are too much magic imo. |
Yeah, I was thinking to do both, like right now there's both https://github.com/adafruit/Adafruit_CircuitPython_PIOASM/blob/main/examples/pioasm_i2sout.py and the same pio code in https://github.com/adafruit/circuitpython/blob/main/ports/raspberrypi/common-hal/audiobusio/I2SOut.c. |
Although I think what we're really going to want is an |
Progress with the TAC5xxx codecs using |
Added draft |
We'd like to be able to do real time audio filtering with CircuitPython. Example: Read from mic, perform FIR filtering with ulab, output on speaker.
We would also like to process all data from the mic without interruption. Example: Waterfall FFT demo
One option to enable this is to allow Python classes to be samples, to allow Python code to retrieve sample data, and to make microphones into audio samples.
Either M4 or nrf52840 would get the initial (PDM)AudioIn DMA work, as this is different per MCU. nrf52840 because clue is the current hotness, M4 because its audio out is better for the filtering case.
A sketch of how this might look as a class outline is at https://gist.github.com/2701e024cdea2c2f60527d674a7b28bc
However, it might also turn out that this is infeasible due to GC, constraints on what we can do from background tasks etc., and instead we need to add a fixed-function pipeline of some sort. In that scenario, a set of FIR taps could be associated with e.g., an AudioMixer object instead.
I'm initially associating this with the 5.x.0 features milestone because @ladyada mentioned this as a possible use case for ulab. It may be too aggressive a timeline to have this done with the ulab release, so the milestone can be postponed as appropriate.
The text was updated successfully, but these errors were encountered: