Add MelSpectrogram layer

[awsaf49]

This PR will add the MelSpecrtoram layer as an audio_preprocessing layer as mentioned in keras-team/tf-keras#55. I have added the backbone of this layer. This layer will convert raw audio signals to Mel spectrograms. This layer is compatible with both GPU & TPU.

Need to add some tests to ensure everything is okay.

Todo

• unbatched audio test
• batched audio test
• zero values audio test
• serialize callable ref

cc: @fchollet , @mattdangerw

[awsaf49]

It turns out linting is failing due to conflict between flake8 and black. Black is making the formatted code line too wide which is flagged by flake8 on other hand if code is reformated to adjust for too wide line then black flags it
This is resolved setting --line-length 80 in black

[fchollet]

Thanks for the PR! This is good work. There's a lot of API design work to be done here to make the API as intuitive as possible.

[awsaf49]

@fchollet Thanks for your valuable feedback. While I understand your concern about the potential confusion caused by non-intuitive names, I would like to point out that the current argument names are widely used and recognized in the community, as demonstrated by popular libraries such as librosa,torchaudio, and nnaudio. For example,

for librosa

librosa.feature.melspectrogram(y=None, sr=22050, n_fft=2048, n_mels=128, hop_length=512, win_length=None, 
                               window='hann', power=2.0, fmin=0.0, fmax=None)

for torchaudio

torchaudio.transforms.MelSpectrogram(sample_rate=16000, n_fft=400, win_length=None, hop_length=None, 
                                     f_min=0.0, f_max= None, n_mels = 128, window_fn = <fn>, power = 2.0)

for nnaudio,

nnAudio.Spectrogram.MelSpectrogram(sr=22050, n_fft=2048, n_mels=128, hop_length=512, 
                                   window='hann', power=2.0,fmin=0.0, fmax=None)

While I am open to alternative names, I believe that changing them could create confusion for users who are already familiar with these naming conventions. Therefore, it seems like a trade-off between renowned names and more intuitive names. Any thoughts what should we do?

[fchollet]

Therefore, it seems like a trade-off between renowned names and more intuitive names. Any thoughts what should we do?

We should use more intuitive names.

• There are more people who will use these APIs in the future than they are people using these APIs today. We're doing them a service by adopting better naming conventions.
• If the names are intuitive, then they will be intuitive / easy to understand for people already familiar with the current APIs.
• Keras APIs must be consistent with the Keras API. It would be annoying and surprising if something called "sampling_rate" in several other places of the API was called "sample_rate" here.

[awsaf49]

@fchollet I've updated the names according to your suggestions and replaced the rest with better intuitive names. Let me know if they meet the requirements.

By the way just noticed, this PR keras-team/keras-hub/pull/847 in Keras-NLP is using sample_rate instead of sampling_rate & stride instead of fft_stride.

[fchollet]

Thanks for the update -- the API is looking good (just one comment)! Please add unit tests.

[awsaf49]

@fchollet could you please check?

[awsaf49]

@gbaned any update?

[awsaf49]

@gbaned could you approve the workflow for unit-test ??

[awsaf49]

@mihirparadkar Hi, I just noticed keras-team-review-pending label has been removed. Any update on this??

[fchollet]

LGTM

[awsaf49]

Currently the PR is on hold due to the following error,

/keras/distribute/ctl_correctness_test.runfiles/org_keras/keras/metrics/confusion_metrics.py", line 22, in <module>
    from keras import activations
  File "/home/kbuilder/.cache/bazel/_bazel_kbuilder/31d6f47147b75c35404d734345be7323/execroot/org_keras/bazel-out/k8-opt/bin/keras/distribute/ctl_correctness_test.runfiles/org_keras/keras/activations.py", line 22, in <module>
    import keras.layers.activation as activation_layers
ImportError: cannot import name 'layers' from partially initialized module 'keras' (most likely due to a circular import) (/home/kbuilder/.cache/bazel/_bazel_kbuilder/31d6f47147b75c35404d734345be7323/execroot/org_keras/bazel-out/k8-opt/bin/keras/distribute/ctl_correctness_test.runfiles/org_keras/keras/__init__.py)

I can't seem to find a fix for it as it relates to keras circular import but the code is written with a similar structure as the image_processing layer which works just fine. Any suggestions would be really helpful.
@fchollet @mattdangerw

[sachinprasadhs]

Hello, Thank you for submitting a pull request.

We're currently in the process of migrating the new Keras 3 code base from keras-team/keras-core to keras-team/keras.
Consequently, merging this PR is not possible at the moment. After the migration is successfully completed, feel free to reopen this PR at keras-team/keras if you believe it remains relevant to the Keras 3 code base. If instead this PR fixes a bug or security issue in legacy tf.keras, you can instead reopen the PR at keras-team/tf-keras, which hosts the TensorFlow-only, legacy version of Keras.