Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Zero box causes invalid configuration argument in CUDA #4122

Open
njzjz opened this issue Sep 12, 2024 · 0 comments · May be fixed by #4180
Open

[BUG] Zero box causes invalid configuration argument in CUDA #4122

njzjz opened this issue Sep 12, 2024 · 0 comments · May be fixed by #4180
Assignees
@njzjz
Labels
bug
Milestone
v3.0.0

Comments

@njzjz
Copy link
Member

njzjz commented Sep 12, 2024

Bug summary

A zero box input to the TF model will cause the invalid configuration argument error in CUDA. Although there is expected to be an error, the error message is confusing and one may not realize that the box is wrong.

DeePMD-kit Version

96ed5df

Backend and its version

TF 2.15.0

How did you download the software?

Built from source

Input Files, Running Commands, Error Log, etc.

Detected at node 'load/ProdEnvMatAMix' defined at (most recent call last):
Node: 'load/ProdEnvMatAMix'
Detected at node 'load/ProdEnvMatAMix' defined at (most recent call last):
Node: 'load/ProdEnvMatAMix'
2 root error(s) found.
  (0) INTERNAL: Operation received an exception: DeePMD-kit Error: CUDA Runtime library throws an error: invalid configuration argument, in file /home/jz748/codes/deepmd-kit/source/lib/src/gpu/coord.cu: 275, in file /home/jz748/codes/deepmd-kit/source/op/tf/custom_op.cc:18
         [[{{node load/ProdEnvMatAMix}}]]
         [[load/o_descriptor/_49]]
  (1) INTERNAL: Operation received an exception: DeePMD-kit Error: CUDA Runtime library throws an error: invalid configuration argument, in file /home/jz748/codes/deepmd-kit/source/lib/src/gpu/coord.cu: 275, in file /home/jz748/codes/deepmd-kit/source/op/tf/custom_op.cc:18
         [[{{node load/ProdEnvMatAMix}}]]
0 successful operations.
0 derived errors ignored

deepmd-kit/source/lib/src/gpu/coord.cu

Lines 272 to 276 in 0f07afa

const int nblock_loc_cellnum = (loc_cellnum + TPB - 1) / TPB;
_fill_loc_cellnum_map<<<nblock_loc_cellnum, TPB>>>(
temp_idx_order, loc_cellnum_map, idx_cellmap_noshift, nloc, loc_cellnum);
DPErrcheck(gpuGetLastError());
DPErrcheck(gpuDeviceSynchronize());

Debug:

line 275: nblock_loc_cellnum=0, TPB=256, loc_cellnum=0

Steps to Reproduce

Input a zero box.

Further Information, Files, and Links

No response
@njzjz njzjz added the bug label Sep 12, 2024
@njzjz njzjz added this to the v3.0.0 milestone Sep 26, 2024
njzjz added a commit to njzjz/deepmd-kit that referenced this issue Oct 2, 2024
@njzjz
fix(tf): throw errors when loc_cellnum is 0
9f40295 
Fix deepmodeling#4122.

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz linked a pull request Oct 2, 2024 that will close this issue
Open
@njzjz njzjz linked a pull request Oct 2, 2024 that will close this issue
fix(tf): throw errors when loc_cellnum is 0 #4180
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@njzjz njzjz

Labels
bug
Projects
Multiple backend support for DeePMD-kit
Status: Todo
Milestone
v3.0.0
Development

Successfully merging a pull request may close this issue.

fix(tf): throw errors when loc_cellnum is 0 njzjz/deepmd-kit
1 participant
@njzjz