Implement tree rotation operator

[MilesCranmer]

This implements a tree rotation as one of the mutation operators:

Though the code implementation here also covers unary operators – both as a root and as a pivot (and as both).

As an example, the expression $$1.5 \cdot \exp(x_1) + \frac{2.5}{x_3}$$ can randomly mutate into one of: $$\frac{(1.5 \cdot \exp(x_1)) + 2.5}{x_3},$$ $$1.5 \cdot \left(\exp(x_1) + \frac{2.5}{x_3}\right),$$ or $$\exp(1.5 \cdot x_1) + \frac{2.5}{x_3},$$.

Thanks to @larsentom for the idea.

This current implementation is a draft. From watching the learning dynamics it definitely seems to help diversify the expressions (!) but we likely want to tune the default mutation weight.

• It would also be nice to have NodeSampler be able to return parent nodes. Otherwise we either need to skip root nodes or do what I do currently which is do a second tree search for the parent - which seems inefficient. But maybe it is the best strategy, I'm not sure.

[MilesCranmer]

Going to modify this so it works for unary operators too Done!

[MilesCranmer]

cc @foxtran – seems this new operator complements your swap operator well

[foxtran]

Nice!

[github-actions]

Benchmark Results

	master	da12afd...	master/da12afdc1f091f...
search/multithreading	20.7 ± 0.48 s	20.1 ± 0.62 s	1.03
search/serial	30.1 ± 0.059 s	30.1 ± 0.066 s	1
utils/best_of_sample	0.962 ± 0.31 μs	0.952 ± 0.29 μs	1.01
utils/check_constraints_x10	11.7 ± 3.1 μs	11.8 ± 3.1 μs	0.995
utils/compute_complexity_x10/Float64	2.1 ± 0.11 μs	2.12 ± 0.11 μs	0.995
utils/compute_complexity_x10/Int64	2.06 ± 0.11 μs	2.13 ± 0.11 μs	0.967
utils/compute_complexity_x10/nothing	1.39 ± 0.1 μs	1.42 ± 0.11 μs	0.979
utils/insert_random_op_x10	4.24 ± 0.44 μs	4.29 ± 0.44 μs	0.988
utils/optimize_constants_x10	30.1 ± 7 ms	30.1 ± 6.8 ms	0.997
utils/randomly_rotate_tree_x10		5.23 ± 0.59 μs
time_to_load	0.981 ± 0.0021 s	0.984 ± 0.0029 s	0.997

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

[coveralls]

Pull Request Test Coverage Report for Build 11205280259

Details

• 61 of 61 (100.0%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall first build on rotate-operator at 96.075%

---

Totals
Change from base Build 11205145452:	96.1%
Covered Lines:	2717
Relevant Lines:	2828

---

💛 - Coveralls