Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the MAP metric. #8931

Merged
merged 5 commits into from
Mar 22, 2023
Merged

Rework the MAP metric. #8931

merged 5 commits into from
Mar 22, 2023

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Mar 16, 2023

  • The new implementation is more strict as only binary labels are accepted. The previous implementation converts values greater than 1 to 1.
  • Deterministic GPU. (no atomic add).
  • Fix top-k handling.
  • Precise definition of MAP. (There are other variants on how to handle top-k).
  • Refactor GPU ranking tests.

Extracted from #8822 .

- The new implementation is more restricted as only binary label is accepted. The previous
implementation converts value greater than 1 as 1.
- Deterministic GPU. (no atomic add).
- Precise definition of MAP.
pytestmark = tm.timeout(30)


def comp_training_with_rank_objective(
Copy link
Member Author

@trivialfis trivialfis Mar 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is the same as the old one, just converted to pytest style using pytest.mark.parametrize, along with cache for the mq2008 dataset to prevent downloading it repeatedly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants