[FEATURE] Essentia

[username116]

Hi,

This is not a request, it's just to let you know.

I've been able to use Essentia for a few days now, and I can find the same kind of results as with AcousticBrainz. I can calculate several genre and mood tags. It seems to me that the Tensorflow models are replacing the old Gaia models (I haven't made a comparison though).

I can use Essentia in 3 different ways:

• on Windows with Python and Tensorflow
  I followed this step by step (with a few changes). It takes time. (I don't know if this procedure will always remain valid.)
  I can give my .whl file for windows x64 (the Python wheel). I don't know if it works, it would allow to go directly to step E. It would be much easier.

  Screenshot

  I've copied the messages here.
  Here the results are percentages, but it's just a formatting, initially the results are a number between 0 and 1 with many decimals, like this.

  

  Example of a python script to calculate a mood

  In the working directory of your choice:
  In a tracks subfolder, I've placed my .flac audio file (adapt the path according to the name of your audio file).
  In a models subfolder, I've placed the .pb and .json files of the tensorflow models.
  (otherwise, you need to adapt the paths)
  Link to the 'happy' model for this example: .pb file, .json file.

  Copy this into a .py file, e.g. my_script.py:

  import json
  
  from essentia.standard import MonoLoader, TensorflowPredictMusiCNN
  
  audio_file = 'tracks/07. Dancing barefoot.flac'
  audio = MonoLoader(sampleRate=16000, filename=audio_file)()
  
  with open('models/mood_happy-musicnn-msd-2.json', 'r') as json_file:
      metadata = json.load(json_file)
  
  activations = TensorflowPredictMusiCNN(graphFilename='models/mood_happy-musicnn-msd-2.pb')(audio)
  
  for label, probability in zip(metadata['classes'], activations.mean(axis=0)):
      print(f'{label}: {100 * probability:.1f}%')

  (I don't claim it's a good script. Chances are it's a bad one.)

  In a command prompt, navigate to this working directory with the cd command, and launch the script with my_script.py in Windows, or python3 my_script.py in Ubuntu.

  A result:

  happy: 78.5%
  non_happy: 44.8%
  

  The other model files can be found here, or here with descriptions, or here.
  In the essentia repository, you'll find examples of python scripts and tutorials with Jupyter notebooks (so you can run code directly in the tutorial). Or simply start here: documentation. Or there are examples at the end of the article.

• installation in WSL (Windows Subsystem for Linux) with Python and Tensorflow (I'm on Windows 10)

  Details

  (maybe there's a Windows feature to enable first elsewhere)
  Install WSL in PowerShell as administrator:

  wsl --install
  

  Possibly in PowerShell, if you get an error message:

  wsl --update
  

  If Ubuntu is not installed automatically (you know it's installed when it asks you to choose a username and password):

  wsl --install -d Ubuntu
  

  I had some difficulties with Debian to install Essentia, so I recommend Ubuntu (it's the default distribution).

  In an Ubuntu terminal, install Python:

  sudo apt install python3-pip
  

  Install essentia with this command:

  pip install essentia-tensorflow
  

  Screenshot

  The same script in an Ubuntu terminal:

  

• with essentia.js and converted Tensorflow models

  Details

  I've modified this demo page a little to use other genre and mood models. My version is here (repository). You need to open the console and look at the *. What I've done is a rough draft, then I found the other methods, so I didn't go much further.

  I'm not sure it works well in Firefox, apparently there are differences in how Firefox and Chrome work, so it's best to try it in a Chrome-based browser (Vivaldi in my case).

  I manage to run this page locally with some kind of server (the Live Preview extension in VS Codium), otherwise there are errors because of the scripts that are downloaded. I haven't looked any further.

  You need converted models for essentia.js (.bin + .json). Most of them are already on the models repository. Otherwise I've managed to do some conversions following this tutorial, but it doesn't work with all models.

[username116]

I've written these wiki pages to show what I've done with Essentia, and the script I use.

It's written with an automatic translator. The code is written by a beginner. And there are foreign language variable names and messages in the code. Sorry about that.

You can reuse and modify it if you like.