[cryptopp] build failure using clang-cl (Windows)

[vovus]

Operating system

Windows Server 2019

Compiler

CLang-CL (17.0.1) standalone and VS Community 2022 (17.9.2) with MSBuild support for LLVM

Steps to reproduce the behavior

vcpkg-tool version: 2023-02-16-12e657924d99511514c0287ca5ce46882d3657c7
vcpkg-scripts version: fbf25ddd1 2024-03-05 

regardless I'm referring here standalone CLang installation, but the same behavior observed using C++ Clang Compiler for Windows (17.0.3) added through VS Installer

1. copy into working vcpkg installation (latest official master branch) the following clang-cl triplets from https://github.com/walbourn/vcpkg/tree/clangcltriplets:

triplets/community/x64-clangcl-static.cmake 
scripts/toolchains/clangcl.cmake

2. open PowerShell terminal

3. (optional - if using standalone CLang installation) define env variable for LLVM custom location:

PS C:\git\vcpkg> $env:LLVMInstallDir = 'C:\Program Files\LLVM'

4. run vcpkg with clang-cl to build cryptopp:

PS C:\git\vcpkg> .\vcpkg.exe install cryptopp --triplet x64-clangcl-static

observe stage-1 failure (as in stage-1 output in the failure logs panel below)

5. get it "fixed" by disabling ASM through portfile.cmake (see patches used in Additional context below):

if(VCPKG_PLATFORM_TOOLSET MATCHES "ClangCL")
	set(CRYPTOPP_DISABLE_ASM "ON")
endif()

6. re-run vcpkg again: 
PS C:\git\vcpkg> .\vcpkg.exe install cryptopp --triplet x64-clangcl-static

and observe stage-2 failure (again see failure logs panel below), but pls note this is seen only on  DEBUG mode and get this "fixed" I have added defined(__clang__) to enable the following section in AllocatorWithCleanup of secblock.h (again, check Additional Context below for patches):

#if (CRYPTOPP_MSC_VERSION >= 1500) || defined(__clang__)
 	AllocatorWithCleanup() {}
 	template <class V, bool A> AllocatorWithCleanup(const AllocatorWithCleanup<V, A> &) {}
#endif

7. re-run vcpkg again: 
PS C:\git\vcpkg> .\vcpkg.exe install cryptopp --triplet x64-clangcl-static

and observe stage-3 failure (both release and debug builds are effected) and this can be fixed with adding !defined(__clang__) to zdeflate.cpp:

#if defined(_STDEXT_BEGIN) && !(defined(CRYPTOPP_MSC_VERSION) && (CRYPTOPP_MSC_VERSION < 1400 || CRYPTOPP_MSC_VERSION >= 1600)) && !defined(_STLPORT_VERSION) && !defined(__clang__)
   stdext::unchecked_mismatch
#else
   std::mismatch

8. re-run vcpkg again: 
PS C:\git\vcpkg> .\vcpkg.exe install cryptopp --triplet x64-clangcl-static

and should be grand

"fixes" - cos I'm not familiar with cryptopp code to trust my changes, but with those applied and cryptest.exe built (using clang-cl) have run the suggested self-tests with all passed:

./cryptest.exe vv
./cryptest.exe tv all

Failure logs

• stage-1 failure (with ASM not disabled):

[22/180] C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\blake2b_simd.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\blake2b_simd.cpp
FAILED: cryptopp/CMakeFiles/cryptopp.dir/__/blake2b_simd.cpp.obj 
C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\blake2b_simd.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\blake2b_simd.cpp
clang-cl: warning: argument unused during compilation: '/MP' [-Wunused-command-line-argument]
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\blake2b_simd.cpp(484,5): error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be inlined into function 'BLAKE2_Compress64_SSE4' that is compiled without support for 'ssse3'
  484 |     BLAKE2B_ROUND(0);
      |     ^

• stage-2 failure on DEBUG build only (with ASM disabled):

to compare on command line level debug vs release:

Release (builds OK):

[58/154] C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -DCRYPTOPP_DISABLE_ASM=1 -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /MD /O2 /Oi /Gy /DNDEBUG /Z7  -MD /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\gzip.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\gzip.cpp

Debug (fails):

[60/154] C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -DCRYPTOPP_DISABLE_ASM=1 -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\gzip.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\gzip.cpp
FAILED: cryptopp/CMakeFiles/cryptopp.dir/__/gzip.cpp.obj 
C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -DCRYPTOPP_DISABLE_ASM=1 -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\gzip.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\gzip.cpp
clang-cl: warning: argument unused during compilation: '/MP' [-Wunused-command-line-argument]
In file included from C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\gzip.cpp:4:
In file included from C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/gzip.h:9:
In file included from C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/cryptlib.h:106:
In file included from C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/stdcpp.h:21:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include\vector(609,38): error: no matching conversion for static_cast from '_Alty' (aka 'AllocatorWithCleanup<CryptoPP::HuffmanDecoder::CodeInfo, false>') to '_Rebind_alloc_t<_Alty, _Container_proxy>' (aka 'AllocatorWithCleanup<std::_Container_proxy, false>')
  609 |         _Mypair._Myval2._Alloc_proxy(_GET_PROXY_ALLOCATOR(_Alty, _Getal()));
      |                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include\xmemory(1500,42): note: expanded from macro '_GET_PROXY_ALLOCATOR'
 1500 | #define _GET_PROXY_ALLOCATOR(_Alty, _Al) static_cast<_Rebind_alloc_t<_Alty, _Container_proxy>>(_Al)
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/zinflate.h(48,2): note: in instantiation of member function 'std::vector<CryptoPP::HuffmanDecoder::CodeInfo, CryptoPP::AllocatorWithCleanup<CryptoPP::HuffmanDecoder::CodeInfo>>::vector' requested here
   48 |         HuffmanDecoder() : m_maxCodeBits(0), m_cacheBits(0), m_cacheMask(0), m_normalizedCacheMask(0) {}
      |         ^
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/secblock.h(187,7): note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'AllocatorWithCleanup<CryptoPP::HuffmanDecoder::CodeInfo, [...]>' to 'const AllocatorWithCleanup<std::_Container_proxy, [...]>' for 1st argument
  187 | class AllocatorWithCleanup : public AllocatorBase<T>
      |       ^~~~~~~~~~~~~~~~~~~~
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/secblock.h(187,7): note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'AllocatorWithCleanup<CryptoPP::HuffmanDecoder::CodeInfo, [...]>' to 'AllocatorWithCleanup<std::_Container_proxy, [...]>' for 1st argument
  187 | class AllocatorWithCleanup : public AllocatorBase<T>
      |       ^~~~~~~~~~~~~~~~~~~~
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean/secblock.h(187,7): note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided

• stage-3 failure (both debug and release):

[148/154] C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -DCRYPTOPP_DISABLE_ASM=1 -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /MD /O2 /Oi /Gy /DNDEBUG /Z7  -MD /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\zdeflate.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\zdeflate.cpp
FAILED: cryptopp/CMakeFiles/cryptopp.dir/__/zdeflate.cpp.obj 
C:\PROGRA~1\LLVM\bin\clang-cl.exe   -TP -DCRYPTOPP_DISABLE_ASM=1 -D_WIN32_WINNT=0x0A00 -IC:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean -IC:\git\vcpkg\buildtrees\cryptopp\src /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc   -m64 /MD /O2 /Oi /Gy /DNDEBUG /Z7  -MD /FIwinapifamily.h /MP /showIncludes /Focryptopp\CMakeFiles\cryptopp.dir\__\zdeflate.cpp.obj /Fdcryptopp\CMakeFiles\cryptopp.dir\cryptopp.pdb -c -- C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\zdeflate.cpp
clang-cl: warning: argument unused during compilation: '/MP' [-Wunused-command-line-argument]
C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\zdeflate.cpp(417,5): error: use of undeclared identifier 'stdext'
  417 |                                 stdext::unchecked_mismatch
      |                                 ^
1 error generated.

Additional context

• to fix the above and make it success to compile the following new ports/cryptopp/cryptopp.patch have been created:

diff --git a/secblock.h b/secblock.h
index 5ab920f9..74d939cf 100644
--- a/secblock.h
+++ b/secblock.h
@@ -270,7 +270,7 @@ public:
 	/// \details VS.NET STL enforces the policy of "All STL-compliant allocators
 	///  have to provide a template class member called rebind".
     template <class V> struct rebind { typedef AllocatorWithCleanup<V, T_Align16> other; };
-#if (CRYPTOPP_MSC_VERSION >= 1500)
+#if (CRYPTOPP_MSC_VERSION >= 1500) || defined(__clang__)
 	AllocatorWithCleanup() {}
 	template <class V, bool A> AllocatorWithCleanup(const AllocatorWithCleanup<V, A> &) {}
 #endif
diff --git a/zdeflate.cpp b/zdeflate.cpp
index b3514b55..20717c24 100644
--- a/zdeflate.cpp
+++ b/zdeflate.cpp
@@ -413,7 +413,7 @@ unsigned int Deflator::LongestMatch(unsigned int &bestMatch) const
 		{
 			CRYPTOPP_ASSERT(scan[2] == match[2]);
 			unsigned int len = (unsigned int)(
-#if defined(_STDEXT_BEGIN) && !(defined(CRYPTOPP_MSC_VERSION) && (CRYPTOPP_MSC_VERSION < 1400 || CRYPTOPP_MSC_VERSION >= 1600)) && !defined(_STLPORT_VERSION)
+#if defined(_STDEXT_BEGIN) && !(defined(CRYPTOPP_MSC_VERSION) && (CRYPTOPP_MSC_VERSION < 1400 || CRYPTOPP_MSC_VERSION >= 1600)) && !defined(_STLPORT_VERSION) && !defined(__clang__)
 				stdext::unchecked_mismatch
 #else
 				std::mismatch

• and the following modification have been made to existing ports/cryptopp/portfile.cmake to accommodate new patch and disable ASM:

diff --git a/ports/cryptopp/portfile.cmake b/ports/cryptopp/portfile.cmake
index f68cd038d..752341b9b 100644
--- a/ports/cryptopp/portfile.cmake
+++ b/ports/cryptopp/portfile.cmake
@@ -15,7 +15,7 @@ vcpkg_from_github(
   REF "CRYPTOPP_${CRYPTOPP_VERSION}"
   SHA512 28a67141155c9c15e3e6a2173b3a8487cc38a2a2ade73bf4a09814ca541be6b06e9a501be26f7e2f42a2f80df21b076aa5d8ad4224dc0a1f8d7f3b24deae465e
   HEAD_REF master
-  PATCHES patch.patch
+  PATCHES patch.patch cryptopp.patch
 )
 
 file(COPY "${CMAKE_SOURCE_PATH}/cryptopp" DESTINATION "${SOURCE_PATH}")
@@ -47,6 +47,10 @@ elseif(NOT DEFINED CRYPTOPP_DISABLE_ASM) # Allow disabling using a triplet file
     set(CRYPTOPP_DISABLE_ASM "OFF")
 endif()
 
+if(VCPKG_PLATFORM_TOOLSET MATCHES "ClangCL")
+	set(CRYPTOPP_DISABLE_ASM "ON")
+endif()
+
 # Dynamic linking should be avoided for Crypto++ to reduce the attack surface,
 # so generate a static lib for both dynamic and static vcpkg targets.
 # See also:

also this change was tested with cryptest.exe without any error / faults reported .. this exe was built from standalone cryptopp official git repo after applying same changes from above:

./cryptest.exe vv
./cryptest.exe tv all

[Neumann-A]

Not a vcpkg issue.

C:\git\vcpkg\buildtrees\cryptopp\src\TOPP_8_9_0-d4af216c8d.clean\blake2b_simd.cpp(484,5): error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be inlined into function 'BLAKE2_Compress64_SSE4' that is compiled without support for 'ssse3'

That is an LLVM bug not correctly reflecting MSVC behavior. You can edit the immintrin.h header of llvm to have defined(_MSC_VER) || !defined(__SCE__) instead of !(defined(_MSC_VER) || defined(__SCE__)) (or explicitly pass -mssse3 etc)
(see llvm/llvm-project#53520)

The rest is a cryptopp issue.

[vovus]

The rest is a cryptopp issue.

you must be right, i have checked standalone build (non-vcpkg) and experienced similar issues, also have found discussion on cryptopp on releated topic and cryptopp wiki says:

The library supports MSVC, Intel ICC, MinGW and Cygwin compilers on Windows. MSVC is limited to Visual Studio .Net compilers and above. Clang is not supported.

fair enough, from this perspective i don't see anything needs or could to be done and this one should be closed i guess

[jimwang118]

OK, if there's nothing we need to fix, I'll close this issue. If you have new questions, please submit a new issue.