Constraint that excludes constants that are not (part of) coefficients

[GoldenGoldy]

I'm trying to figure out how to implement a constraint that excludes any constants that are not a coefficient, and not part of any coefficient. Basic example:
y = aX1 bX2 + cX3 + d
a, b and c are ok, but d should be excluded, so an equation of this form should be penalized for containing the "separate constant" d.

I searched through previous discussions about custom losses, which allow to do a lot of cool things but I haven't found a solution for the above yet.
I also checked https://discourse.julialang.org/t/tree-collection-looking-for-api-feedback-in-dynamicexpressions/98109 to see what components are available in a custom loss to potentially make this work, but I couldn't yet figure out how to do it.

At first, I thought node.constant would be helpful, but as I understand this would also be True for a, b and c in the example above. So how to differentiate between a b c, and d in this example?
Is it necessary to cycle through all nodes for which node.constant is True, and then somehow check if they are only "linked" to the rest of the equation via + or -, or is there an easier way?
Apart from not knowing how to do this exactly, I also wonder how big the performance impact of something like this might be.

[GoldenGoldy]

Just giving this a nudge, which I hope is ok.
Suggestions anyone?
Any help would be much appreciated.

[MilesCranmer]

Sorry for the late reply @GoldenGoldy. This constraint sounds a bit tricky but it is certainly doable. Have a look at https://symbolicml.org/DynamicExpressions.jl/dev/examples/base_operations/ to learn how to write custom constraints. For your problem I suppose I would have some check like:

num_constants_under_multiply = sum(tree) do node
    is_mul = node.degree == 2 && node.op == 1  # Assuming * was passed as first operator
    if is_mul
        left = node.l
        right = node.r
        left_is_constant = left.degree == 0 && left.constant
        right_is_constant = right.degree == 0 && right.constant
        
        return left_is_constant + right_is_constant
    else
        return 0
    end
end

This will count how many times <const> * (anything) or (anything) * <const> appears in the equation tree. And then you could just count the number of constants with

num_constants = count(tree) do node
    node.degree == 0 && node.constant
end

And I guess if num_constants == num_constants_under_multiply, you could use that as a constraint?

And then you would include this in a custom loss and add some penalty term for a violation.

Hope this helps!

[GoldenGoldy]

@MilesCranmer Thanks for the detailed reply!
I tested it and it works so far! I now realized I might have to change it further to also allow for division and perhaps some other situations, but I think I should manage to make such changes based on this working example and the resource you linked.
Thanks!

[MilesCranmer]

Cool. I guess the one other thing it doesn't take into account is that you could still have things like <const> * <const>. Presumably you also want to skip those. For that situation I think I would tweak the calculation to be:

num_constants_under_multiply = count(tree) do node
    is_mul = node.degree == 2 && node.op == 1  # Assuming * was passed as first operator
    if is_mul && node.l.degree == 0 && node.l.constant
        return any(subnode -> subnode.degree == 0 && !subnode.constant, node.r)
    elseif is_mul && node.r.degree == 0 && node.r.constant
        return any(subnode -> subnode.degree == 0 && !subnode.constant, node.l)
    else
        return false
    end
end

Basically the any check is going to see if any nodes in the other side of the * operator are non-constant via depending on a variable node, which needs to satisfy node.degree == 0 && !node.constant.

[GoldenGoldy]

Awesome, thanks for anticipating that because indeed const * const terms already were popping up and I was planning to deal with those still.
I can confirm that with the updated example from your last reply, such terms are also gone.
And I changed it a bit further now to also allow for division:

num_constants_under_mult_or_div = count(tree) do node
	is_mul_or_div = node.degree == 2 && (node.op == 3 || node.op == 4)  # Assuming * was passed as third operator and / as fourth operator (or vice versa).
	if is_mul_or_div && node.l.degree == 0 && node.l.constant
		return any(subnode -> subnode.degree == 0 && !subnode.constant, node.r)
	elseif is_mul_or_div && node.r.degree == 0 && node.r.constant
		return any(subnode -> subnode.degree == 0 && !subnode.constant, node.l)
	else
		return false
	end
end

I will test a bit more to see if it now exactly works as intended, but it looks very promising, thanks!