[QST]In batch sampler

[Tselmeg-C]

❓ Questions & Help

Details

This InBatchSampler pram is only applicable to TwoTowerModel but not for the DLRM model right? For DLRM I need to feed both positive and negative samples for training and validate, is that correct?

[CarloNicolini]

Currently it took me many head-scratches to understand the InBatchSampler.
First of all, it seems that at least with 23.08, the InBatchSamplerV2 is utilized instead of mm.InBatchSampler as the default sampler (in case TwoTowersModelV2 has negative_samplers set to None).
Second, the InBatchSamplerV2 simply yields the batch itself with no differences between positives and negatives, at least on the target point of view.

I would like someone more expert than me help me understand the main logic behind the negative sampling strategies, and how to apply popularity bias correction to the InBatchSamplerV2 without incurring into a handful of errors.