Remove libcudacxx symlinks #1075

wmaxey · 2023-11-09T07:42:26Z

Description

closes #143

It deletes the symlinks and restructures the test layout for libcudacxx.
It deletes many libcxx symbol testing artifacts that are currently unused (we can bring them back if we need to).

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

…lders

libcudacxx/include/cuda/std/detail/libcxx/include/__iterator/move_iterator.h

libcudacxx/include/cuda/std/detail/libcxx/include/array

ahendriksen

What a huge PR! I see that a lot of changes were in code I recently committed. Sorry for that! I left some questions inline and I also ran CI on H100.

I ran into some failures were not part of the diff, so adding them here:

5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/barrier/cp_async_bulk_tensor_generic.h:234:19: error: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Werror=sign-compare]                            
5:   234 |     for (int i = 0; i < rank-1; ++i) {                                                                                                                                                                                                                               
5:       |                 ~~^~~~~~~~~~~~                                                                                                                                                                                                                                       
5: cc1plus: all warnings being treated as errors

Quick fix would be to change to change to: for (int i = 0; i < int(rank-1); ++i)

5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp: In instantiation of ‘void test_conversion_from_resource_ref() [with PropA = property_with_value<short int>; PropB
= property_with_value<int>]’:                                                                                                                                     
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:157:94:   required from here
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:79:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    79 |     const auto fake_orig = *reinterpret_cast<Fake_alloc_base*>(&ref_input);                                                            
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                       
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:79:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:80:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    80 |     const auto fake_conv = *reinterpret_cast<Fake_alloc_base*>(&ref);                                                                            
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                              
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:80:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:98:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    98 |     const auto fake_orig = *reinterpret_cast<Fake_alloc_base*>(&ref_input);                                                               
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                              
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:98:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:99:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    99 |     const auto fake_conv = *reinterpret_cast<Fake_alloc_base*>(&ref);                                                 
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                  
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:99:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp: In instantiation of ‘void test_conversion_from_resource_ref() [with PropA = property_with_value<short int>; PropB
= property_without_value<int>]’:                                                                                                           
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:157:194:   required from here
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:79:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    79 |     const auto fake_orig = *reinterpret_cast<Fake_alloc_base*>(&ref_input);                                                                     
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                          
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:79:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:80:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    80 |     const auto fake_conv = *reinterpret_cast<Fake_alloc_base*>(&ref);                                                                      
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                           
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:80:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:98:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    98 |     const auto fake_orig = *reinterpret_cast<Fake_alloc_base*>(&ref_input);                                                                
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                             
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:98:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:99:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    99 |     const auto fake_conv = *reinterpret_cast<Fake_alloc_base*>(&ref);                                                               
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                      
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:99:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp: In instantiation of ‘void test_conversion_from_resource_ref() [with PropA = property_without_value<short int>; Pro
pB = property_without_value<int>]’:                                                                                                    
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:157:297:   required from here
5: /home/scratch.ahendriksen_gpu/projects/cccl/libcudacxx/test/libcudacxx/cuda/memory_resource/memory_resource.resource_ref/resource_ref.conversion.pass.cpp:79:27: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
5:    79 |     const auto fake_orig = *reinterpret_cast<Fake_alloc_base*>(&ref_input);                                                                           
5:       |                          ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

No idea what this error is about.

Timeouts:

5: UNRESOLVED: libcu++ :: heterogeneous/barrier_abi_v2.pass.cpp (212 of 1795)                                                                                                                                                                                                   
5: UNRESOLVED: libcu++ :: heterogeneous/barrier.pass.cpp (207 of 1795)

ahendriksen · 2023-11-10T08:55:44Z

libcudacxx/include/cuda/barrier

-          "r"(__size),
-          "r"(static_cast<_CUDA_VSTD::uint32_t>(__cvta_generic_to_shared(::cuda::device::barrier_native_handle(__bar))))
-        : "memory");
+  NV_IF_TARGET(


Question: why is the NV_IF_TARGET necessary here? I would have expected

#if (!defined(__CUDA_MINIMUM_ARCH__)) || (defined(__CUDA_MINIMUM_ARCH__) && 900 <= __CUDA_MINIMUM_ARCH__)

to have made sure that the function is not available on older architectures?

It's still 'available' on host. So we need to prevent host from seeing it in case it is included in .cpp files.

ahendriksen · 2023-11-10T09:02:20Z

libcudacxx/include/cuda/std/detail/libcxx/include/__cuda/ptx/ptx_helper_functions.h

-  // Consider adding debug asserts here.
-  return static_cast<_CUDA_VSTD::uint64_t>(__cvta_generic_to_global(__ptr));
+  NV_DISPATCH_TARGET(
+    NV_IS_DEVICE, (


Question: do we need NV_DISPATCH_TARGET in device functions as well?

I am also curious about that one

Yep, headers should be includable in host only TUs.

miscco · 2023-11-10T10:01:52Z

@wmaxey I have pushed fixes for libcudacxx. Please retest them

wmaxey · 2023-11-13T22:28:17Z

@wmaxey I have pushed fixes for libcudacxx. Please retest them

@miscco I had to remove fixes from this branch, we'll open them in another to keep it small.

On that note, I did forget to pull the changes, I really hope you have them as I never had them hit my object store.

github-actions · 2023-11-14T00:46:20Z

Backport failed for branch/2.3.x, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin branch/2.3.x
git worktree add -d .worktree/backport-1075-to-branch/2.3.x origin/branch/2.3.x
cd .worktree/backport-1075-to-branch/2.3.x
git checkout -b backport-1075-to-branch/2.3.x
ancref=$(git merge-base 9bbec0cc43170059360bc9483085412ddf8168a7 4d1d613126279ec31a0a8ec2fc9920f649138036)
git cherry-pick -x $ancref..4d1d613126279ec31a0a8ec2fc9920f649138036

* [backport] Cherry-pick PR #1075 to branch/2.3.x manually * [backport] Move PTX tests that missed the symlink PR #1098

wmaxey added 13 commits November 8, 2023 15:50

Delete libcudacxx/test|utils symlinks

109624d

Rename libcudacxx/.upstream-tests to /test and delete a few unused fo…

00bbac2

…lders

Delete the mac test stuff from utils.

77449a9

Move the support headers out of the libcudacxx test folder

3d1ae8f

Move the include smoke tests out of libcudacxx

d5468b0

Delete the pretty printers tests

02fdaa0

Move the cmakelists for lit

c159cfa

Update the config to recognize the new layout

6b2afb1

Delete unused python scripts in our lit config.

5defef1

Delete unused docker/old infra.

bddedd7

Split cmakelists test folders.

46eef92

Remove extra include handled by lit config.

17dfc60

Enable building smoke tests.

6a82748

wmaxey requested review from a team as code owners November 9, 2023 07:42

wmaxey requested review from robertmaynard, ericniebler, jarmak-nv, miscco and griwes and removed request for a team November 9, 2023 07:42

miscco reviewed Nov 9, 2023

View reviewed changes

libcudacxx/include/cuda/std/detail/libcxx/include/__iterator/move_iterator.h Outdated Show resolved Hide resolved

miscco reviewed Nov 9, 2023

View reviewed changes

libcudacxx/include/cuda/std/detail/libcxx/include/array Outdated Show resolved Hide resolved

wmaxey requested a review from miscco November 10, 2023 00:46

ahendriksen reviewed Nov 10, 2023

View reviewed changes

jrhemstad added the backport branch/2.3.x For backporting to the 2.3.x release branch label Nov 13, 2023

wmaxey added 3 commits November 13, 2023 14:02

Rename host_only to smoke.

51264e2

Make lit output artifacts into the right directory.

8a318aa

Fix path to nvrtcc binary.

4d1d613

wmaxey force-pushed the fea/remove_libcudacxx_symlinks branch from f6e5462 to 4d1d613 Compare November 13, 2023 22:26

jrhemstad approved these changes Nov 13, 2023

View reviewed changes

wmaxey merged commit 0447ad7 into main Nov 14, 2023
517 checks passed

wmaxey deleted the fea/remove_libcudacxx_symlinks branch November 14, 2023 01:06

wmaxey added a commit that referenced this pull request Nov 14, 2023

[backport] Cherry-pick PR #1075 to branch/2.3.x manually

5a84459

jrhemstad pushed a commit that referenced this pull request Nov 14, 2023

Backport PR #1075 (#1100)

37ba2d4

* [backport] Cherry-pick PR #1075 to branch/2.3.x manually * [backport] Move PTX tests that missed the symlink PR #1098

jrhemstad removed the backport branch/2.3.x For backporting to the 2.3.x release branch label Nov 14, 2023

This was referenced Dec 6, 2023

Repository restructuring to remove symlinks #937

Closed

Rename .upstream-tests #935

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove libcudacxx symlinks #1075

Remove libcudacxx symlinks #1075

wmaxey commented Nov 9, 2023 •

edited

Loading

ahendriksen left a comment

ahendriksen Nov 10, 2023

wmaxey Nov 10, 2023

ahendriksen Nov 10, 2023

miscco Nov 10, 2023

wmaxey Nov 10, 2023

miscco commented Nov 10, 2023

wmaxey commented Nov 13, 2023 •

edited

Loading

github-actions bot commented Nov 14, 2023

Remove libcudacxx symlinks #1075

Remove libcudacxx symlinks #1075

Conversation

wmaxey commented Nov 9, 2023 • edited Loading

Description

Checklist

ahendriksen left a comment

Choose a reason for hiding this comment

ahendriksen Nov 10, 2023

Choose a reason for hiding this comment

wmaxey Nov 10, 2023

Choose a reason for hiding this comment

ahendriksen Nov 10, 2023

Choose a reason for hiding this comment

miscco Nov 10, 2023

Choose a reason for hiding this comment

wmaxey Nov 10, 2023

Choose a reason for hiding this comment

miscco commented Nov 10, 2023

wmaxey commented Nov 13, 2023 • edited Loading

github-actions bot commented Nov 14, 2023

wmaxey commented Nov 9, 2023 •

edited

Loading

wmaxey commented Nov 13, 2023 •

edited

Loading