Manipulating identified expressions using a custom loss function

[TadeuNP]

Is it possible to (numerically) integrate an expression or use a symbolic library (like Sympy) from a custom loss function?

I thought of using "node_to_symbolic" from SymbolicRegression.jl to obtain something that I could later convert to a callable function, but all I managed to do with it was simple evaluation:

@syms x::LiteralReal y::LiteralReal

evaluated_expr =  (substitute(expr, Dict(x  => 1,  y => 2)))

And this only worked when calling Julia code directly, not from a custom loss function.

Ideally, the tree itself would be a callable function, but it doesn't seem to be the case. Any suggestions?

[MilesCranmer]

Have you looked at the loss_function parameter (equivalently, full_objective in PySR)? It lets you write an arbitrary function for the loss, including any sort of non-differentiable external calls you want. Numerical integration is totally possible.

[TadeuNP]

Hi Cranmer,

Yes, I'm trying to implement it using that. The problem resides in calling some predefined numerical integration routine, since I don't know how to define a callable out of the tree object. But... I managed to implement a simple Euler integrator using eval_tree_array (included below if anyone ever needs it), so that may not be as important now.

If you don't mind a bit of a derailing, is it possible to access iteration number and other properties at every loss function evaluation? So I can define a custom loss function that will act different every/after n iterations or evaluate less complex solutions with a slightly different formula.

Thanks!

#This should work inside a custom objective function:
t0 = 0f0
tf = 1.2f0
dt = 0.01f0
x0 = 1.f0
ts = range(t0, stop=tf+dt, step=dt)
xs = zeros(Float32, length(ts))

# Set initial condition
xs[1] = x0

# Simple example function
f = (x, t) -> x[1] * x[1] * 0.29

# Perform numerical integration using Euler's method
for i = 1:length(ts)-1
      t = ts[i]
      f = eval_tree_array(tree, transpose(xs[i:i]) , options)[1][1]
      xs[i+1] = xs[i:i][1] + f*dt
end  

[MilesCranmer]

So I can define a custom loss function that will act different every/after n iterations or evaluate less complex solutions with a slightly different formula.

This is not allowed because the loss values are cached, so they may be out-of-date in one when comparing two individuals. Instead what you should do is restart the search (pass in the saved_state) with a different loss function, which will trigger all existing loss values to be cleared in the saved state.

options = Options(return_state=true)
hof, state = EquationSearch(...)

# change loss function and create new Options

hof, state = EquationSearch(..., saved_state=(hof, state))

[TadeuNP]

That makes sense! Thanks!