PySR model tuning

[LP2006]

Hello,

I am new to PySR and currently working on the following problem.

I am trying to derive the following empirical formula using PySR as part of my thesis work. The formula, upon simplification, has a complexity of around 50 (53) and contains many terms (5 input variables and combinations of them).
I already made many changes to the model including setting deterministic to True, maxsize, niterations, nested_constraints to derive the exact expression but have yet to achieve it.

The expression on expanding looks like this and the output expression from PySR should be similar to this.

I have gas (CO2, CO, NO, O2 and HC ) measurements as input data and Lambda values for output.

Could anyone help me in understanding on how to provide constraints such as constraints, nested_constraints so that they match the above expression and help the model narrow the search to the exact expression.
I have already referred to the PySR regressor reference page to understand how the above mentioned constraints work.

Any response would be highly appreciated.

[MilesCranmer]

Hi @LP2006,

Thanks for the post. Take a look at the tuning tips page here: https://astroautomata.com/PySR/tuning/ for some info on how you could optimize the parameters for your expression. An expression with a complexity of 50 might be difficult to get exactly, but you could try.

Could you post your current PySRRegressor settings?

Cheers,
Miles

[LP2006]

Hello,

Thank you very much for the response.
Here is the current model

model = PySRRegressor(
    model_selection="best",                                                    
    populations = 10,                                                          
    maxsize = 60,                                                           
   population_size = 60,                                                   
    niterations=100,                                                         
    binary_operators=["+", "*",'/','^'],                                    
    unary_operators = ['square'],                                                
    warm_start = False,                                                        
    tournament_selection_n = 10,
    constraints={
    'add': (3, 3),
    'mult': (3, 3),
    'pow': (-1, 1),
    'div': (5, 5)
},                                               
    use_frequency_in_tournament = True,
    warmup_maxsize_by = 0.2,
    use_frequency = False,
 nested_constraints = {
    '+': {'+': 0, '*': 2, '^': 0, '/': 0},
    '*': {'+': 2, '*': 2, '^': 0, '/': 0},
    '^': {'+': 0, '*': 0, '^': 0, '/': 0},
    '/': {'+': 2, '*': 3, '^': 0, '/': 0}
},
  ]
     denoise = True,
    select_k_features=5,                                                     
    loss="loss(x, y) = (x - y)^2", 
    verbosity = 1e5,
    progress = True,
    output_torch_format = True,
    random_state = 17,
    procs = 0,
    multithreading = False,
    deterministic = True,                                          
   )

Please let me know if the above settings are good to work with or if any improvements could be made.

Thanks.

[MilesCranmer]

Comments:

1. "^" is not needed, so remove it.
2. Add the operator "-".
3. Remove denoising (denoise=False).
4. Turn off the select_k_features=5 setting, because you have more than 5 features you need.
5. Your constraints are too strong for your equation. See the docs on the constraints here. In your example:

constraints={
    'add': (3, 3),
    'mult': (3, 3),
    'pow': (-1, 1),
    'div': (5, 5)
},                                  

this says that "add" can only have maximum of 3 complexity in its left and right arguments. Which rules out the equation you wish to find.

I would instead do:

constraints={
    'square': 1
},                                  

which only imposes a constraint on square that it can have only a variable as its input (which seems reasonable), and leaves the others alone.

6. Your nested constraints are also too strong. See the docs for more info. I would do:

 nested_constraints = {
    '/': {'/': 2}
},

which means that it can only nest up to three / operator, but not more. The CO/CO2 term in your desired equation is nested within two /, so at least this constraint is needed. You could even do '/': {'/': 3} to give it more "room" to explore.

7. Why use deterministic mode? It will be slower. I would turn that off as well, so you then can use CPU parallelism.

So in total, I would make the changes:

model = PySRRegressor(
    maxsize=60,
    population_size=60,
    niterations=1_000_000_000,  # Just make this very large and exit when satisfied
    binary_operators=["+", "-", "*", "/"],   # Adjusted
    unary_operators=["square"],   # Adjusted
    constraints={"square": 1},   # Adjusted
    nested_constraints={"/": {"/": 2}},  # Adjusted
    loss="loss(x, y) = (x - y)^2",
)

I removed some of the other parameters because they are already set as defaults. No need to change them unless you know you need to. You could also play around with the parsimony parameter.

Hope this helps!

Miles

[LP2006]

Hello Miles,

Thank you for the detailed explanation.

I tried the model with your suggestions.

Is there a way to constrain the presence of input variables w.r.t numerator and denominator positions?
This is because the desired expression has NO only in the numerator and HC in the denominator.

The expressions that the model has generated so far do not have this constraint.

Thank you once again for your valuable input.

[MilesCranmer]

Unfortunately that kind of constraint is not possible. However I want to add this in the future - see MilesCranmer/SymbolicRegression.jl#193.