Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pyspark] hotfix for GPU setup verification issue #9495

Merged
merged 2 commits into from
Aug 17, 2023

Conversation

wbo4958
Copy link
Contributor

@wbo4958 wbo4958 commented Aug 17, 2023

If the user sets "spark.task.resource.gpu.amount" to be a float value like 1.0 or 0.25, pyspark will throw an exception with the below msg,

  File "/opt/conda/default/lib/python3.10/site-packages/xgboost/spark/core.py", line 657, in _fit
    self._validate_params()
  File "/opt/conda/default/lib/python3.10/site-packages/xgboost/spark/estimator.py", line 230, in _validate_params
    super()._validate_params()
  File "/opt/conda/default/lib/python3.10/site-packages/xgboost/spark/core.py", line 378, in _validate_params
    if not gpu_per_task or int(gpu_per_task) < 1:
ValueError: invalid literal for int() with base 10: '0.25'

This PR is to fix this bug and kind of GPU configuration validating rework.

@wbo4958
Copy link
Contributor Author

wbo4958 commented Aug 17, 2023

@WeichenXu123 @trivialfis please help to review it.

python-package/xgboost/spark/core.py Outdated Show resolved Hide resolved
python-package/xgboost/spark/core.py Outdated Show resolved Hide resolved
python-package/xgboost/spark/core.py Outdated Show resolved Hide resolved
python-package/xgboost/spark/core.py Outdated Show resolved Hide resolved
@trivialfis trivialfis merged commit 68be454 into dmlc:master Aug 17, 2023
21 checks passed
@wbo4958 wbo4958 deleted the validate-bug branch April 23, 2024 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants