CryptAcquireContext, CryptGenRandom and CryptReleaseContext are deprecated

[mabuchner]

On Windows libtomcrypt uses CryptAcquireContext, CryptGenRandom and CryptReleaseContext to generate random numbers.

https://github.com/libtom/libtomcrypt/blob/v1.18.2/src/prngs/rng_get_bytes.c#L105

According to the documentation here, here and here, those functions are deprecated.

Important This API is deprecated. New and existing software should start using Cryptography Next Generation APIs. Microsoft may remove this API in future releases.

In fact, I'm opening this issue, because I failed to compile libtomcrypt for UWP where those functions were already removed.

libtomcrypt should probably replace the deprecated functions with the mentioned Cryptography Next Generation API.

[mabuchner]

I've reimplemented the _rng_win32 function with bcrypt.

# CMakeLists.txt
if(MSVC)
    target_link_libraries(tomcrypt PRIVATE Bcrypt)
endif()

/* rng_get_bytes.c */
#include <bcrypt.h>

static unsigned long _rng_win32(unsigned char *buf, unsigned long len,
                               void (*callback)(void))
{
   LTC_UNUSED_PARAM(callback);

   BCRYPT_ALG_HANDLE hProv = 0;
   if (!BCRYPT_SUCCESS(BCryptOpenAlgorithmProvider(&hProv, BCRYPT_RNG_ALGORITHM, NULL, 0))) {
      return 0;
   }

   if (!BCRYPT_SUCCESS(BCryptGenRandom(hProv, buf, len, 0))) {
      BCryptCloseAlgorithmProvider(hProv, 0);
      return 0;
   }

   BCryptCloseAlgorithmProvider(hProv, 0);
   return len;
}

It compiles now, but I don't have a Windows machine here to test it.

[assarbad]

I'll try, I've got Windows to try it out (although I really want to avoid testing VS2008 for something like that), I happen to have VS2005 still installed, so I'll give it a try on that. Chances are it'll open the project and solution with only minor changes.

Also at least for libraries like this one I'd probably use #pragma comment(lib, "...") [1] to tell the linker about the bcrypt dependency. However, I do not know if that is acceptable. One upside of Bcrypt, as this is CNG even kernel mode code on Windows could use this.

[1] That said, there are reasons why you may not want to do this without also providing an option to turn the behavior off. One being that if you ever request a dependency like this, the respective record ends up in the object file and ultimately a static lib and could annoy you if you wanted to supply the symbols any other way.

[assarbad]

I also noticed another function named s_read_wincsp (in s_mp_rand_platform.c) exists which also makes use of CSP.

What's off about the general code around these locations is that they default to WINVER for XP in one place, but WINVER for NT 4.0 (and actually 9x/ME) in another 🤨

[assarbad]

@sjaeckel @leite @karel-m Could you please chime in exactly what you'd like to see here? I'd be happy to contribute the following changes on top of what @mabuchner has provided (building atop his work). The list below is intentionally numbered, so we can discuss the points. Some of those are more or less "policy" which is why I am asking you as (visible) libtom project members to chime in.

1. Try libtomcrypt and libtommath under VS2005, fix any evident issues; optionally supply said VS2005 solution
2. Try the two libs under VS2019 and VS2022 and supply the respective fixed up solutions/projects for static libs
   1. My goal here would be to put most of the items into a property sheet (.props file) or .targets file and limit the rest to the absolute minimum; such a property sheet or .targets file should easily used by anyone else using VS from versions around VS2012/VS2013 to pull in the libraries as a dependency.
3. Fix the function mentioned by @mabuchner and the additional one I found
   1. Retain the older functionality (will be tested using VS2005 and potentially an old DDK) based on CSP and decide based on the WINVER and/or _WIN32_WINNT value (see notes below)
4. Establish Windows 6.0 (Vista) as a baseline for WINVER and/or _WIN32_WINNT (only one should be set and both values should match) without preventing builds targeting down to Windows 2000 or so (this is about how far back I can test without going through some really tiny hoops 😉)
5. Drop the proposed CMakeLists.txt change in favor of #pragma comment(lib, "bcrypt") for the newer code (review if we can do the same for the older ... this pragma has existed for ages!)

Thanks for any input you can provide, so I can get going.

[sjaeckel]

Thanks for picking this up!

That plan sounds good.

Regarding the solution files for newer VS versions than the 2008 we already have: please don't add this. We won't start adding a solution for each new VS version. If someone wants to use it in VS, they can convert the 2008 solution or simply use the makefile.msvc as already done in CI for multiple VS versions.

Regarding VS2005: it should be sufficient to test via makefile.msvc or not?

Regarding backwards compatibility: Sounds good! We can keep the existing implementation and use it for all Windows versions that don't have "bcrypt" yet and use the new version if possible.

[assarbad]

That plan sounds good.

👍 ... and thanks for the swift response.

Regarding the solution files for newer VS versions than the 2008 we already have: please don't add this. We won't start adding a solution for each new VS version.

I can see why and this wasn't what I was proposing (one solution/project per VS major version). However, you should be aware that VS since around VS2010 has been using MSBuild under the hood. That's a departure from prior versions (incl. VS2008) and it would be good to offer some MSBuild-based project, since all these modern VS versions have structurally the same project file format.

The conversion exists and works, but converted projects typically have some peculiarities embedded, such as property sheets which will suppress certain warnings or enable old (MSVC-specific) standard-violating behavior to not cause any "friction" when converting to the newer version. I looked over the auto-converted .vcxproj for libtomcrypt and it's not too much cruft, so I guess it won't do any damage. But modern versions of VS would completely leave out some of the artifacts one can find in the converted projects. So while working, I'd never recommend working with a converted project without thorough manual review if you want to use the result for anything serious 😁.

But I won't impose on you. I will anyway use a forked version of premake4 (Git clone here) to generate the VS solutions for my work on this. I may just as well contribute the resulting premake4.lua file instead of the solutions (plus some documentation on how to use that).

If someone wants to use it in VS, they can convert the 2008 solution or simply use the makefile.msvc as already done in CI for multiple VS versions.

The Makefile seems like a good idea in general (I love GNU make for other purposes), but when in Rome ... so I think a (modern) VS-based project is still the better option. But again, I won't impose. I'll work on this a bit, send a pull request and then we can hone it further to suit your needs.

Regarding VS2005: it should be sufficient to test via makefile.msvc or not?

Might work, yes. But with premake4 I don't have to care (see above). Again, you don't have to care about this part and whether you use the resulting premake4.lua is something you can decide down the road.

Regarding backwards compatibility: Sounds good! We can keep the existing implementation and use it for all Windows versions that don't have "bcrypt" yet and use the new version if possible.

Excellent. I hope I can make it work across a range of versions. However, one small wish: could we update the minimum to something like Windows XP? Theoretically the code does impose XP as minimum already in s_mp_rand_platform.c, but NT 4.0 in rng_get_bytes.c. So it stands to reason it hasn't been tried for such old versions for a very long time. But having a single consistent minimum baseline would be appreciated. XP got released in 2001.

As an unrelated side note: I am working on amalgamating libtomcrypt and libtommath into a single .c file (in the hope that I can make it work both on C and C++ compilers). libtommath is painless in that regard, just paste them together. libtomcrypt has plenty issues and I have to see how to work around this in a non-invasive way. Questions: 1.) any advice you can share? 2.) Would the project accept renaming of preprocessor and C symbols across unrelated .c files or is this something that would not be accepted (just gauging interest and pondering my best path forward)?

Thanks.

[sjaeckel]

The conversion exists and works, but [...] I'd never recommend working with a converted project [...]
I may just as well contribute the resulting premake4.lua file

OK, IMO we can also replace the 2008 solution with a premake4.lua file.

The Makefile seems like a good idea in general (I love GNU make for other purposes), but when in Rome ... so I think a (modern) VS-based project is still the better option.

OK!? even though it's an nmake file which is natively supported from VS?!

one small wish: could we update the minimum to something like Windows XP?

IMO: yes

Theoretically the code does impose XP as minimum already in s_mp_rand_platform.c, but NT 4.0 in rng_get_bytes.c

Please feel free to bring both to the same state!

[...] amalgamating libtomcrypt and libtommath into a single .c file [...] libtommath is painless in that regard [...]
1.) any advice you can share?

you know there's also a pre_gen/mpi.c? it's only included in the release, but can also be generated from the Git repo by make pre_gen

then there's https://github.com/eduardsui/tlse which already uses an outdated amalgamation of ltc and can maybe give some hints?

2.) Would the project accept renaming of preprocessor and C symbols across unrelated .c files or is this something that would not be accepted (just gauging interest and pondering my best path forward)?

sure! as we're breaking API&ABI on develop anyways, we can clean everything up as required.

Thanks.

Thanks back to you :)

[assarbad]

I didn't even receive a notification of your last message, but I am now on it now. Sorry for the delay.

[assarbad]

@sjaeckel A few observations. Please note, I am starting with libtommath, as it's a prerequisite for libtomcrypt. It should be quicker to take the results from the former and apply them to the latter, once everything is in place.

Since you seem to strive for extensive backward compatibility it's with great sadness (😉) I have to break it to you that even the unchanged solution/.vcproj cannot build on VS2008. I tried. Yep, I am right now in the process of installing an array of VS versions (VS2003, 2005, 2008, 2010, 2012, 2013, 2015, 2017, 2019, 2022). I had already installed those in bold and have since installed VS2003 and VS2008. After trying to "re-target" the vanilla VS2008 solution (by manually editing, I have some experience with this exercise) to VS2005 and failing. The error is the same on VS2005 and 2008:

libtommath\tommath.h(8) : fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory

There is indeed no such header in either of these VS versions. Alternatives exist. I'll pick one which creates the least noise on VS2003 through 2008 and is "license"-compatible. VS2010 allegedly has the header (haven't installed it as of yet).

Further reading on the topic:

• https://web.archive.org/web/20101104120614/http://snipplr.com/view/18199/stdinth/
• https://android.googlesource.com/platform/external/libusb_aah/+/master/msvc/stdint.h (newer version of the above)
• This Q&A on StackOverflow is a treasure trove of information. You can find a comment, saying VS2010 is the first to have the header. There is another saying VS2015 is the first VS to claim near-full compatibility with VS2015 and from experience I'd say this is about right.

Clearly the presence of a VS2008 solution would suggest that one is able to build libtommath (in this case) using it. So perhaps not such a good idea to provide such an old solution. By the way, stdbool.h also doesn't exist, but I have yet to determine the exact version range in which this is the case (for now I'll go with the assumption that it's the same). It's a trivial addition, though.

Anyway, making slow progress. I'll create a branch on my end, so I can rebase on develop as needed and you can peek at what I'm doing.

Please also note that in all my tests I will use the very latest patch level of the VS versions available to me. If I feel like it (and find the time), I may even try it out with VS6 and the DDKs/WDKs 😁.

PS: if you don't know this website, have a look at predef.sf.net for information on how to identify compilers, OSs and libraries etc.

[assarbad]

/wd5045, given in the makefile.msvc also errors out on VS2003 through VS2008. The warning code range ends at 4999.

Also, I don't know how /WX (i.e. "treat warnings as errors"; also in makefile.msvc) is supposed to work. I am getting at least three distinct warnings (of which there are multiple instances) which cause the build to fail with /WX. VS2005 is a worse offender than VS2008 in that regard, but still.

[sjaeckel]

Theoretically the code does impose XP as minimum already in s_mp_rand_platform.c, but NT 4.0 in rng_get_bytes.c

Please feel free to bring both to the same state!

Oh I'm not sure if you misunderstood my statement here! I don't think we have to force compatibility to really ancient versions of VS with the makefile.msvc, we should set a sane limit. Maybe the ones supported natively on appveyor!?

Source-code compatibility is another question. IMO we don't have to build against such old versions if we keep the code intact and still compile it against some newer VS version. If someone really wants to use the libraries on such an old platform they can still disable those warnings or do whatever else is necessary to have a successful build. I don't see it as our duty to really test this.

[assarbad]

Ah, fair enough. However, now I've already done the bulk of it for libtommath, so it doesn't really hurt to go through it. Especially since I spent hours on end installing all VS versions between 2003 and 2022 (inclusive). The versions that I have not yet tested well are VS2010 through 2017, but I am on it.

Edit: the nice part is that those old versions won't change. Even VS2017 is unlikely to see another release. So everything I find out about these versions will hold true in the future all the same.

I have one question regarding your remark about makefile being the authoritative source of truth. Could you please put this into perspective to me. As far as I can tell it contains at least two potentially different definitions of truth. There is pre_gen which concatenates files according to a particular wildcard (*mp_*.c) whereas the list of OBJECTS is hardcoded (and presumably updated via helper.pl?!). Which one is right? Also, why not use something along the lines of:

OBJECTS:=$(patsubst %.c,%.o, $(wildcard *mp_*.c))

... for the makefile which seems to aim at GNU make anyway? (I also mentioned in another place that naming the file GNUmakefile would have its benefits, as GNU make will prefer it in that case over any other variation of the name makefile and the makefile.unix could take its place and name. This would mean when on Linux - where make == GNU make - the GNUmakefile is used, but on BSDs the makefile would be used instead.)

---

The above could also be done more explicitly like:

SOURCES:=$(wildcard *mp_*.c)
OBJECTS:=$(SOURCES:.c=.o)

[sjaeckel]

Can you please check whether #626 correctly fixes this?

[mabuchner]

Sorry, but I can't really test it anymore. Unfortunately, I had to find a new employer. Now I have neither access to my ex-employers codebase nor a Windows system.

Thanks for taking the time to fix this issue. I'm sure it will benefit other developers.

[assarbad]

Can you please check whether #626 correctly fixes this?

I'll try to have a look at the change from a Windows box tonight.

PS: Sorry, but I lost sight of the other topic I had worked on, since libtomcrypt in the end didn't suit my needs for the particular issue, but I'll try to finish up my contribution to have proper solutions/projects for various VS versions.

[assarbad]

@sjaeckel took one day longer, but appears to work as desired from my tests. As far as I can tell this solves it. Compiles without warnings, and works when running it.

PS: I also like that you went with a solution not requiring to initialize the crypto provider.

[pineappleiceberg]

This errors out on a fresh configure with Unknown CMake command "cmake_push_check_state" and Unknown CMake command "cmake_check_symbol_exists". This can be fixed with
include (CMakePushCheckState)
include(CheckSymbolExists).