Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve non-finite parameters rejection when training models #1209

Merged
merged 1 commit into from
Mar 12, 2024

Conversation

gabemery
Copy link
Collaborator

Update of the build_models method to ensure more effectively that non-finite values in features are removed.

Main change : additional call to utils.filter_events before the classifier training to account for the new 'signed_time_gradient'. Other new parameters that may be affected include the 'signed skewness', reconstructed energy and reconstructed disp.

I also moved the 'sin_az_tel' computation before the first call to utils.filter_events. I have never seen a case when 'az_tel' is not finite but this would handle it.

Dedicated tests could be added in a LHfit related PR since this is where a failure case was observed.

closes #1206

@gabemery gabemery self-assigned this Jan 29, 2024
@gabemery gabemery requested a review from maxnoe February 15, 2024 11:15
@SeiyaNozaki
Copy link
Collaborator

Hi @gabemery The code looks fine. Maybe it is possible to remove particle_classification_features for the first event filtering since it will be applied later?

@gabemery
Copy link
Collaborator Author

gabemery commented Mar 4, 2024

@SeiyaNozaki I would keep it there. The idea being that the best would be to filter all problematic events from the start, if possible (i.e. if events have bad features for particle classification it is better to also remove them from the energy and direction training). Then we filter again for the features which are produced later just in case.

@SeiyaNozaki
Copy link
Collaborator

SeiyaNozaki commented Mar 4, 2024

Okay, thanks for the explanation! Then, it looks already fine!

@rlopezcoto rlopezcoto merged commit 1d98657 into main Mar 12, 2024
7 checks passed
@rlopezcoto rlopezcoto deleted the fix_RF_feature_filtering branch March 12, 2024 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Training failure when using features computed after utils.filter_events
4 participants