Some Questions about MACE for pure water and in general

[nihil39]

Hi,

I'm trying to train a model for pure water made with MACE. My dataset consists of 3000 configuration at 300K taken at different densities, from 0.972 to 1.04 $\frac{g}{cm^3}$ coming from DFT calculations.

I have some questions, sorry if they are trivial:

I am trying to use the "right" hyperparameters. In order to do so: I am starting reading this and comparing those parameters with the ones written here that should be the ones used for the MACE MP-0 L1 model.

1. What are the advised parameters for pure water? In particular the Energy and the forces weight.

2. What do the following hyperparameters mean?

--interaction_first="RealAgnosticResidualInteractionBlock" \
    --interaction="RealAgnosticResidualInteractionBlock" \
    --num_interactions=2 \
    --correlation=3 \

3. Is this the max order of the spherical harmonics -max_ell=3 \ used in the model? Why it is the same for the L0, L1 and L2 MACE MP0 models?

What does "last readout" mean? What is the width of radial MLP?

--MLP_irreps MLP_IRREPS
   hidden irreps of the MLP in last readout
 
--radial_MLP RADIAL_MLP
  width of the radial MLP

5. Last and maybe least important. How can I properly set a wandb interface? Where should I put my wandb token? In --wandb_entity? What are the possible loggable hyperparameters, how can I select them?

Thank you in advance.

[gabor1]

MAke sure you use the latest version in the main branch. The code is evolving fast. I suggest you use "1" and "1" for energies and forces weights. use --model ="MACE" and don't specify any interaction blocks. Both the parameter lists that you linked use average E0s (reference energies for atoms), we don't recommend this for general use, and indeed the latest version of the mace-mp-0 model doesn't do this. Compute isolated atom energies for H and O, and either put those into your dataset, or give the isolated atom energies as E0s using command line options.

-max_ell is the spherical harmonics order for the local environment, use 3. L is the spherical harmonic order of the messages in the message passing step, 0 is invariant, 1 and 2 are higher order (equivariant). Higher L will give you more accurate, but larger and slower models.

"last readout" means the readout function of the last layer. We typically put a shallow MLP on it. the radial MLP is the parametrisation of the radial basis used to expand the local environment. These are all defined and discussed in the MACE papers.

I'll let others who use it regularly comment on wandb

[nihil39]

Thank you,

I am trying to understand how MACE works and for me it's a honor to be able to get answer from you authors. I don't know if this is the place to ask for explanation on MACE in general, even theoretical questions about the architecture.

Ok, I'm putting isolated atoms energies in the training script, without using average values, even if I don't think there are any Isolated atoms in my training set, but ok. I 'd like also to use Scaleshift MACE instead of MACE model because there should be any dissociation of pure water at those temperatures and densities but I will experiment.

What is the difference between max_L and max_ell ? If I understand correctly max_L is the global order of equivariance while max_ell=3 is the local order of equivariance? The fact that max_ell is 3 is not related to the dimensions of the system (3D system), right? max_L = 1 means scalars and 3d vectors if I understand correctly.

What are exactly the channels? If I want to have an L = 1 model I should set max_L=1 and number of channels 128 meaning that I will have 128 scalars and 128 odd vectors, if I understand correctly. But why 128? Is this just a standard value coming from heuristics?

the relation between channels and order of tensor is not so clear, it seems that there is no correlation between them.

[nihil39]

I don't know if mark your reply as answer or not: it is an answer but then I asked something else. :-)

[ilyes319]

I'll turn that into a discussion. Issues are for bugs and feature requests.
As a general comment, I suggest you use the help function of the arg parser, as it will provide some answers to these questions.

• For the loss weights, I would try forces: 10, energy: 1 for the go to. I always start with that.
• You should provide these hypers but for reference --correlation=3 says that MACE will construct 4 body messages at each layer. --num_interactions=2 means that it will have 2 layers. The type of interaction blocks is advanced and you should not specify them.
• For wandb, I suggest you read the wandb documentation. You should first login in your wandb account on your head node or computer (see https://docs.wandb.ai/ref/cli/wandb-login). Then you should select --wandb in the input script and and select the right project with --wandb_project. To select hypers to log, use --wandb_log_hypers. To see the list of what can use, use the help function of the arg parser.

[nihil39]

Thanks,

Sorry, I didn't notice the discussion tab :-) .

I will try to login first from the cluster and then try to set a wandb run but from the help function of the arg parser I only get

--wandb_log_hypers WANDB_LOG_HYPERS
  The hyperparameters to log in Weights and Biases

not the list of the actual loggable Hyperparameters

Are all the data about loss function and other saved in the model output file? I mean are those data available without using wandb? I think so, but just to be sure where to find them.

[nihil39]

I managed to login with my wandb api keys, but before the training starts, it complains about the the lack of wandb.init() command, I don't know if I have to put it in the shell script I use in order to launch the training

[ilyes319]

Look here for the list of things you can log:

mace/mace/tools/arg_parser.py

Line 631 in 22a2e3e

default=[

Can you post what error you get.

[nihil39]

This is the error, I don't know if it's a network error but it would be strange

wandb: W&B API key is configured. Use `wandb login --relogin` to force relogin
wandb: - Waiting for wandb.init()...

\ Waiting for wandb.init()...^Mwandb: | Waiting for wandb.init()...^Mwandb: / Waiting for wandb.init()...^Mwandb: - Waiting for wandb.init()...^Mwandb: \ Waiting for w
andb.init()...^Mwandb: | Waiting for wandb.init()...^Mwandb: / Waiting for wandb.init()...^Mwandb: - Waiting for wandb.init()...^M2024-08-30 16:47:53.801 WARNING: Retry
ing (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 
0x1485ae85c700>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /api/4504800232407040/envelope/

2024-08-30 16:47:53.803 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3
.connection.HTTPSConnection object at 0x1485ae85c340>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /api/4504800232407040/envelope/
wandb: ERROR Run initialization has timed out after 90.0 sec.