GCC build errors reported on Windows

[snej]

I have a bug report against my Nimdbx wrapper, that mdbx.c (the amalgamation prebuilt by me with make dist) fails to compile with GCC on Windows. I do know Nimdbx builds on macOS and Ubuntu. The first two errors are:

mdbx.c:16:10: error: #include expects "FILENAME" or <FILENAME>
 #include MDBX_CONFIG_H
          ^~~~~~~~~~~~~
mdbx.c:1371:29: error: unknown type name 'FILE_INFO_BY_HANDLE_CLASS'
     _In_ HANDLE hFile, _In_ FILE_INFO_BY_HANDLE_CLASS FileInformationClass,
                             ^~~~~~~~~~~~~~~~~~~~~~~~~

I've never seen a preprocessor symbol used as the parameter of #include before; is it a GCC extension? But it seems to work for me with Clang, and on Github Actions CI with GCC on Ubuntu.

I grepped the libmdbx "dist" sources, and there is no definition of FILE_INFO_BY_HANDLE_CLASS anywhere. The code that uses it is conditionalized for Windows only. Is this a libmdbx bug? Or is that symbol supposed to come from a Windows platform header?

Unrelated, but I also noticed a missing line break on line 1 of mdbx.c:

#define MDBX_ALLOY 1n#define MDBX_BUILD_SOURCERY 9854b516cd42bd97a2447d2f4adfda8cd99464906b15a3646a1409c129fbb1bc_v0_9_2_21_g1c70814

It looks like the "n" before the second #define was supposed to be a "\\n"...

(This is using libmdbx commit 3a441d6 from 4 Jan.)

[erthink]

Ok, I'll see what's going on.

[erthink]

Briefly:

• Seems the problem (incompatibility) is in the windows version of the utilities (sed, bash, echo, etc) used for make dist and building libmdbx by make.
• I am leaning towards the decision to close this issue as "wontfix", while retaining the recommendations to use CMake for building on Windows and create amalgamated source code (i.e. running make dist) only on modern Unix.

[erthink]

More details:

1. libmdbx supposes the two build methods: by CMake with provided CMakeLists.txt (a portable way), or by GNU Make with provided GNUmakefile (requires GNU Linux or modern Unix environment).
2. GNUmakefile uses commands like
   $(CC) $(CFLAGS) $(MDBX_OPTIONS) '-DMDBX_CONFIG_H="config.h"' -DLIBMDBX_EXPORTS=1 -c mdbx.c -o $@ to build, which expended to cc -O2 -g -Wall -Werror -Wextra -Wpedantic -ffunction-sections -fPIC -fvisibility=hidden -std=gnu11 -pthread -Wno-error=attributes -DNDEBUG=1 '-DMDBX_CONFIG_H="config.h"' -ULIBMDBX_EXPORTS -c mdbx.c -o mdbx-static.o, etc.
   This is correct, but requires properly handling command line options in Unix style, and this work perfect when mdbx.c (and other files) has proper header like this:

#define MDBX_ALLOY 1
#define MDBX_BUILD_SOURCERY 0a3f81552735496c153bc690bc12de...
#ifdef MDBX_CONFIG_H
#include MDBX_CONFIG_H
#endif

3. You noted that header is incorrect, and the #define MDBX_ALLOY 1n#define MDBX_BUILD_SOURCERY 9854b5... means that at least sed tool don't proceed as expected (i.e. failed to handle extended syntax properly).

So it is not a libmdbx's error, but sed and/or other tools (including wrong command line options processing by windows version of GNU make and bash).

Nothing can be fixed inside libmdbx to avoid this.
I can only recommend using CMake to build libmdbx on Windows (either don't use Windows at all).

[erthink]

Additionally, it is worth noting that the macro MDBX_CONFIG_H should be defined as "filename.ext" but not as filename.ext.
I.e. the macro value should be a valid C-string within double quotes in order for the #include MDBX_CONFIG_H construct to be valid for the C-preprocessor.

[snej]

You noted that header is incorrect ... at least sed tool don't proceed as expected (i.e. failed to handle extended syntax properly). So it is not a libmdbx's error, but sed and/or other tools (including wrong command line options processing by windows version of GNU make and bash).

Actually that glitch occurred when I ran make dist on macOS 10.15. (Those sources are checked into my repo.) I don't know the exact version of sed, but it seems to be based on the FreeBSD version.

use CMake for building on Windows and create amalgamated source code (i.e. running make dist) only on modern Unix

The CMake build fails unless the sources be in a git repo. This was causing problems for me with the way Nim installs packages ... basically by the time CMake runs, my package's source tree (and submodules) has been copied and isn't a git repo any more. Obviously not your problem; I'll try to find a workaround.

Thanks for your help!

[erthink]

Perhaps I'm confusing something, but AFAIR, I checked the work of make dist on Freebsd.
Nonetheless, at least, I will add a check of the make dist result to detect problem immediately.

[erthink]

And, of course, all suggestions on how to make the sed script more portable are welcome.

[erthink]

@snej, I just added an amalgamated source code check during make dist, and these changes have passed CI-check on Travis-CI, including for MacOS and FreeBSD.

This means that my amalgamation script do not have any troubles, it work fine on Linux, FreeBSD and MacOS.

[AskAlexSharov]

Looks like Go doesn't support linking to MSVC files ( golang/go#20982 ). So, i'm trying to build with mingw.
But build with mingw failed with error from this ticket unknown type name 'FILE_INFO_BY_HANDLE_CLASS'
https://github.com/torquem-ch/mdbx-go/runs/2533036546?check_suite_focus=true#step:6:63

But I didn't use amalgamated code, I run next on "libmdbx" git submodule: cmake -G "MinGW Makefiles" . then cmake --build .

[erthink]

The first problem was that when building with MinGW, the target version of Windows is selected too old, in which this enum is not yet defined. I made a couple of commits to fix this.

However, the second problem error: unknown conversion type character 'z' in format still remains.
Suggested solutions worth trying:

1. Add somithink like this to here:

   #ifdef __MINGW32__
   #define __USE_MINGW_ANSI_STDIO 1
   #endif 
   

2. Use the latest CMake which (may be) known how to enable C11 for MinGW.
3. Modifiy CMakeLists.txt to force add -std=c11 for MinGW.

@AskAlexSharov, could you try these solutions (since I haven't MinGW for now) ?

[erthink]

@AskAlexSharov, another option: you may prefer to build libmdbx by MSVC as a shared library (i.e. dll) with MDBX_WITHOUT_MSVC_CRT option for avoid using any additional runtime/CRT(s).
Sure GoLand should be able to use a dll with the C-header (i.e. mdbx.h).

[AskAlexSharov]

1. cmake version 3.20.1
2. Applied patch erigontech/mdbx-go@17e87d1 - result https://github.com/torquem-ch/mdbx-go/runs/2533764261?check_suite_focus=true#step:6:176
3. "Modifiy CMakeLists.txt to force add -std=c11 for MinGW." - sorry, not sure how to
4. "since I haven't MinGW for now" - I using next github actions config: https://github.com/torquem-ch/mdbx-go/blob/win/.github/workflows/test.yml
5. "as a shared library" - I will try today.

[erthink]

I've added a few more fixes, but I am more and more inclined to close this issue as "won't fix" since MinGW don't support:

• the __try and __except statements for native/structural exception handling.
• the z printf's format flag for size_t.
• the /* fall through */ comment for explicit marking a case statements which are fall through.

[erthink]

Now it (likely) should be possible to build libmdbx by MinGW with (a lot of) warnings, but I still suggest you to use a dll built by MSVC.

[AskAlexSharov]

cmake -DBUILD_SHARED:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=ON -DMDBX_WITHOUT_MSVC_CRT:BOOL=ON .
cmake --build .

Some warnings of MSVC build: https://github.com/torquem-ch/mdbx-go/runs/2533871563?check_suite_focus=true#step:5:108

[erthink]

Windows, MSVC and MinGW must die.
All together.

[AskAlexSharov]

I got a bit same conclusion today. Will see if I can over-sleep it.