BUG #394 fix prox and regularization

[himkwtn]

#394

• fix the calculation in prox and regularization function
• clear up some confusion regarding weight lambda and threshold

In the A unified sparse optimization framework to learn parsimonious physics-informed models from data paper, in each iteration, prox is called with lambda * nu, so if we set lambda = (threshold^2)/(2*nu) then for l0, coefficients smaller than the threshold will be eliminated.

This pull request only fixes the issue for prox and regularization calculation but it is still being called incorrectly in SR3 class (except for l0). Future PR will be needed to accept weight lambda correctly.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.23%. Comparing base (0d47d7a) to head (01ce00d).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #544      +/-   ##
==========================================
+ Coverage   94.22%   94.23%   +0.01%     
==========================================
  Files          37       37              
  Lines        4105     4045      -60     
==========================================
- Hits         3868     3812      -56     
+ Misses        237      233       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Jacob-Stevens-Haas]

Thanks for this work, Watcharin!

Within utils.base

1. rename lam to regularizer_weight or reg_weight_lambda
2. Document shapes of inputs, add validation
3. Move private functions inside of get_... functions
4. delete CAD

test_utils

• minimal number of elements for data and weights (2 instead of 3)
• Add a parameter for reg_weight_lambda to combine weighted and unweighted tests
• Add assertion test for shapes of weight and shapes of data
• remove 2D tests

outside utils.base and test_utils

• restore objective() in stable_linear_sr3
• restore shape of thresholds in test_sr3_quadratic_library
• restore shape of thresholds in test_constrained_sr3_quadratic_library
• restore shape of thresholds in test_constrained_inequality_constraints

[himkwtn]

I double-checked StableLinearSR3 objective function and it is the same as SR3.

It also seems that the examples in the notebook use regularization_weight of the same shape as data input but the ones in the unit test use the transposed version. So there are two conflicting examples. Which one should we choose to be correct?

[Jacob-Stevens-Haas]

It also seems that the examples in the notebook use regularization_weight of the same shape as data input but the ones in the unit test use the transposed version. So there are two conflicting examples. Which one should we choose to be correct?

Can you provide a MWE? How does the code work if it's not the right shape?

It seems like the example notebooks call SR3, which then transposes before calling these functions. That said, for these helper functions, the data should be in the same shape as the weights.

[Jacob-Stevens-Haas]

Good work Watcharin! Most of what remains is cleaning this up and cutting out stuff that can be made redundant.

[himkwtn]

@Jacob-Stevens-Haas
For xi of size (m,n), in example.py the thresholds variable is of size (m,n) but in test_optimizer.py it is (n,m).
I changed example.py to be (n,m). Does this work for you?

[Jacob-Stevens-Haas]

Good work, Watcharin! We're in the home stretch. Mostly just deletion and edits

[himkwtn]

@Jacob-Stevens-Haas I ran python examples/publish_notebook.py examples/1_feature_overview and it broke the notebook. Could you help with this issue?

[Jacob-Stevens-Haas]

Well done, Watcharin! Looks like you fixed the notebook issue.