Mask values are all 0, loss never change

[TalBarami]

Hello,
I have tried to create a mask for an example of my data.
The data itself has been created from the skeleton joint coordinates of a human body. Its shape is of size TxN, where T is the length of the time series and N is the feature (Either x/y/z locations of a specific joint j).
For this reason, the size of each sample is relatively big, approximately 200x300.
My model predicts whether the individual presented in the scene performs some type of action.
I have tried to tune the hyperparameters as explained here, but it didn't change. I have managed to play with the loss values so if the mask is large enough the loss will be higher, but it seems like the task isn't optimized - as the losses that follow remain relatively the same.

pert = GaussianBlur(device, sigma_max=80)
mask = Mask(pert, device)
mask.fit(flat, lambda p: posec3d(update_pert(p, X)), target=torch.tensor([0, 1], device=device),
             loss_function=cross_entropy, keep_ratio=0.1, size_reg_factor_init=0.01)
mask.plot_mask()

Any thoughts?

Thanks in advance,
Tal