make check - no test ran

[emazv72]

Symptom: Running make check doesn't run any test on my environment.

$ make check
make check -C test
make[1]: Entering directory '/hbdev/test/ziplib/json/test'
Executing test-json_patch...
===============================================================================
No tests ran

The issue seems to be in test/Makefile

if I replace

TEST_PATTERN ?= "*"

with

TEST_PATTERN ?= ""

The issue seems fixed

OS: Windows 10
Plat: MSYS2 - GNU Make 4.2.1 - gcc version 7.2.0 (Rev1, Built by MSYS2 project)

Thanks

[nlohmann]

Thanks for reporting. We recently switched to Cmake to compile/execute the test suite, so you could also try

$ mkdir build
$ cd build
$ cmake ..
$ cmake --build .
$ ctest

However, we did not change the existing Makefile, and pattern * should be ok... Strange...

Anyway, please try with cmake/ctest.

[nlohmann]

I cannot reproduce the issue. make check executes the test suite without problems. Changing TEST_PATTERN ?= "" is wrong: it would give make check the same behavior as make check-fast to only execute a smaller test suite.

[emazv72]

Thanks Niel for your time. I didn't have much time to dig into make issues, so I switched to cmake, can build successfully on develop branch now.

After running the the test suite I've the following failures:

35 - test-inspection_default (Failed)
61 - test-regression_default (Failed)
65 - test-testsuites_default (Failed)

Building on master branch gives me the following errors:

Scanning dependencies of target catch_main
[ 2%] Building CXX object test/CMakeFiles/catch_main.dir/src/unit.cpp.o
[ 2%] Built target catch_main
[ 5%] Generating CXX prefix source test/cotire/json_unit_CXX_prefix.cxx
[ 7%] Generating CXX prefix header test/cotire/json_unit_CXX_prefix.hxx
[ 10%] Building CXX precompiled header test/cotire/json_unit_CXX_prefix.hxx.gch
json/build/test/cotire/json_unit_CXX_prefix.hxx:4:10: fatal error: json/build/test/cotire/json_unit_CXX_prefix.cxx: No such file or directory
#include "/build/test/cotire/json_unit_CXX_prefix.cxx"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CMake Error at /json/cmake/cotire.cmake:1848 (message):
cotire: error 1 precompiling
/build/test/cotire/json_unit_CXX_prefix.hxx.
Call Stack (most recent call first):
/json/cmake/cotire.cmake:3512 (cotire_precompile_prefix_header)

make[2]: *** [test/CMakeFiles/json_unit.dir/build.make:64: test/cotire/json_unit_CXX_prefix.hxx.gch] Error 1
make[1]: *** [CMakeFiles/Makefile2:122: test/CMakeFiles/json_unit.dir/all] Error 2
make: *** [Makefile:139: all] Error 2

[nlohmann]

Could you try the develop version?

[emazv72]

Just tested the develop version again and got the following output ( didn't change since last run on develop )

The following tests FAILED:
35 - test-inspection_default (Failed)
61 - test-regression_default (Failed)
65 - test-testsuites_default (Failed)

[nlohmann]

Can you please provide the error logs so we can have a look why the tests fail?

[emazv72]

Hope this LastTest.log helps, can't see any other relevant file generated in the build tree.

[nlohmann]

Could you please run the tests with

ctest -O test.log --output-on-failure

and attach file test.log?

[emazv72]

This is the test result test.log

[nlohmann]

It seems that most errors are due to MingGW's formatting of exponents with a leading zero, cf.

C:/Data/e.zavallone/test/ziplib/json/json/test/src/unit-testsuites.cpp:328: FAILED:
  CHECK( j.dump() == json_string )
with expansion:
  "[1.2345e+030]" == "[1.2345e+30]"
with messages:
  filename := "test/data/json_roundtrip/roundtrip32.json"
  json_string := "[1.2345e+30]"

or

C:/Data/e.zavallone/test/ziplib/json/json/test/src/unit-regression.cpp:772: FAILED:
  CHECK( j.dump() == "1.66020696663386e+20" )
with expansion:
  "1.66020696663386e+020"
  ==
  "1.66020696663386e+20"

We had a ticket on this before, but if I remember correctly, MinGW's behavior is not conformant here.

[emazv72]

The effect of this issue is negligible on my domain. I briefly inspected the other failed test ( parsing in particular ) and they look like corner cases ( like unit-regression.cpp, SECTION("second example from #529") )

I'm going to integrate your library - easy to use and well documented - in my app, thanks for helping me.

[nlohmann]

No worries. I really would like to provide better support here, but I found no way yet to run MinGW on a CI server.

In any way - if you need further assistance, just open an issue.