Windows with Visual Studio regression test - test fail

[ILDENIEL]

Hi, I'm having problems with the tests.
I did everything explained in the section 3.2 of the OpenFAST Documentation (Release v3.2.0), but some tests failed.
In particular I tried to typed
python manualRegressionTest.py ..\build\bin\openfast_x64_Double.exe 1e-5 0.5e-5
but this was the result

Then I tried
PS C:\OFNREL\openfast\reg_tests> python manualRegressionTest.py ..\build\bin\openfast_x64_Double.exe Intel 1e-5 0.5e-5
usage: manualRegressionTest.py [-h] [-p [Plotting-Flag]] [-n [No-Execution]] [-v [Verbose-Flag]] [-case [Case-Name]] [-module [Module-Name]]
Executable-Name Relative-Tolerance Absolute-Tolerance
manualRegressionTest.py: error: argument Relative-Tolerance: invalid float value: 'Intel'

Finally I tried changing the tolerance
PS C:\OFNREL\openfast\reg_tests> python manualRegressionTest.py ..\build\bin\openfast_x64_Double.exe 1e-5 0.5e-4
and most of the tests succeded, but not all of them.
How can I solve this issue?

Regression test execution completed with these results:
AWT_YFix_WSt PASS
AWT_WSt_StartUp_HighSpShutDown PASS
AWT_YFree_WSt PASS
AWT_YFree_WTurb PASS
AWT_WSt_StartUpShutDown PASS
AOC_WSt PASS
AOC_YFree_WTurb PASS
AOC_YFix_WSt PASS
UAE_Dnwind_YRamp_WSt PASS
UAE_Upwind_Rigid_WRamp_PwrCurve PASS
WP_VSP_WTurb_PitchFail PASS
WP_VSP_ECD PASS
WP_VSP_WTurb PASS
SWRT_YFree_VS_EDG01 PASS
SWRT_YFree_VS_EDC01 PASS
SWRT_YFree_VS_WTurb FAIL
5MW_Land_DLL_WTurb PASS
5MW_OC3Mnpl_DLL_WTurb_WavesIrr PASS
5MW_OC3Trpd_DLL_WSt_WavesReg PASS
5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth PASS
5MW_ITIBarge_DLL_WTurb_WavesIrr FAIL
5MW_TLP_DLL_WTurb_WavesIrr_WavesMulti PASS
5MW_OC3Spar_DLL_WTurb_WavesIrr PASS
5MW_OC4Semi_WSt_WavesWN PASS
5MW_OC4Jckt_ExtPtfm PASS
5MW_Land_BD_DLL_WTurb PASS
StC_test_OC4Semi PASS
IEA_LB_RWT-AeroAcoustics PASS
HelicalWake_OLAF PASS
EllipticalWing_OLAF PASS
WP_Stationary_Linear PASS
5MW_Land_BD_Linear PASS
5MW_OC4Semi_Linear PASS
Ideal_Beam_Fixed_Free_Linear PASS
Ideal_Beam_Free_Free_Linear PASS
StC_test_OC4Semi_Linear_Nac PASS
StC_test_OC4Semi_Linear_Tow PASS
MHK_RM1_Fixed PASS
Total PASSING tests - 36
Total FAILING tests - 2

[ietqlw]

Dear @bjonkman and @jjonkman，

Recently, I have upgraded openfast from Version 3.0.0 to Version 3.4.1. Anything went well with Version 3.0.0. When I employed visual studio 2019 to compile openfast 3.4.1, I was told to upgrade “Windows SDK Version” to 10.0. I selected to upgrade. However, "Configuration Type" in “MAP_dll Property Pages ” was altered as “Static library (.lib)” automatically. Hence, when I rebuilt the solution, MAP_x64.dll did not generate. Other files could generated normally. I tried to change the "Configuration Type" as “Dynamic Library (.dll)” and then rebuilt the solution. MAP_x64.dll could generated normally while MAP_x64.lib was missing. As a result, error was reported as "Severity Code Description Project File Line Suppression State Error fatal error LNK1181: cannot open input file '....\build\bin\MAP_x64.lib' LINK".

I came up a solution to this bug:

compile "MAP_dll" project alone to generate MAP_x64.dll by setting "Configuration Type" in “MAP_dll Property Pages ” as “Dynamic Library (.dll)”.
build the whole solution (instead of “rebuild”) by setting "Configuration Type" in “MAP_dll Property Pages ”as “Static library (.lib)”.
By doing this, all files including MAP_x64.dll and MAP_x64.lib can generate normally. But I do not know whether it has potential deficiencies. I have conducted a regression test the compiled program. 36 cases passed and 2 case failed.

[bjonkman]

@ietqlw , As of OpenFAST v3.1.0, the MAP library has been linked statically into the main OpenFAST executable, so there is no need for a separate MAP dll file. See https://github.com/OpenFAST/openfast/releases/tag/v3.1.0.

Also, as of OpenFAST v3.3.0, the testing system changed to use different criteria for passing. See #1217 for details on this change, including default values for tolerances.

[andrew-platt]

Regarding the two tests that are failing -- these tests both use AeroDyn 14 are unstable enough that they show slight variations depending on the system the are run on. We don't actually run either test in our automated testing because of this.

• 5MW_ITIBarge_DLL_WTurb_WavesIrr -- We have removed it from our automated testing.
• SWRT_YFree_VS_WTurb -- this test is also rather unstable and we will be removing it entirely from the test suite.

[ILDENIEL]

Ok thanks, so can I consider that everything is working properly and I correctly did all the steps?