Network Architecture and Design.

[aman-17]

Feel free to put your suggestions to improve the model's performance. For instance: architectural changes, dataset changes, and change in inference.

[warichet]

Hello,
I'm Sébastien Warichet working for Alcatel Lucent Enterprise.
First of all congratulation for your work.
We are looking to bring your technology into our desk phone.

To do that we have started to translate your code into C++, which is lighter for our integration.

We have also try to analyse your encoder result with our dataset.
Our data set is very small and the result could be not relevant.
But on the attached image we can the embeddings of the dataset.
One abscisse are the embedding vector generated by the encoder.
Each color correspond to a specific word (décroche, raccroche diminue, augmente et volume) and each line correspond to a specific encoded vector.

In the picture we can see that, a large part of embedding give similar result for all sample, regardless of the word or the speaker.
This could indicate that perhaps the output vector (and possibly even the encoder itself) is too large and coud be reduced.

We are willing to participate in your development, especially on the encoder design.

Thanks a lot
Sébastien

[TheSeriousProgrammer]

This could be attributed to the fact that the model was trained on English words, resulting in random embeddings for the words from other languages. Can you test it with English words? What are your target languages?

[warichet]

Hello,
could you tell me which dataset you have used to train/test the siamese network?
Is it a word by word dataset ?
Best regards
Sébastien

[TheSeriousProgrammer]

It was trained on a synthetic dataset from azure's neural tts engine. But recently ml perf released a dataset for wakeword detection for multiple languages cinsider checking that out

[warichet]

Hello,
i have try with an English dataset (about 22.000 samples)
Here the result

I will try to train a model with this dataset to compare results.
About the mel transform model, is it trained by yourself or did you get it ?
Thanks

[TheSeriousProgrammer]

No that's not a model, it's just a function converted to tflite for getting the same and quicky results of mel spectrograms no matter the platform

I was not able to find easier implementations of mel spectrogram in android and was worried if I would get the same results if I implemented so converted it to tflite

[warichet]

Hello,
i have started to train very very simple models based on VGG-M.
And when i dump my vectors, all the parts of vectors seems useful.

In this images, we dump the vectors (size 16) and each colors represent a different word.

It's just for testing and the final network should be less simple/more complex.

But the question is in your layers did you add enough bach_normalization?
This might explain why you have a part of the vector that doesn't have enough variance.

Best regards
Sébastien

Layer (type) Output Shape Param
conv2d_39 (Conv2D) (None, 30, 45, 32) 1472
batch_normalization_39 (None, 30, 45, 32) 128
max_pooling2d_39 (None, 15, 22, 32) 0
..................
2 other layers with conv + batch_normalization + max_pooling
some dropout to limits overfitting
..........................
flatten_13 (Flatten) (None, 256) 0
dense_14 (Dense) (None, 16) 4112

[TheSeriousProgrammer]

We used l2 normalization of the output vector of batch normalization. Would that cause the issue?