Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase size of TimeoutType from 8 to 16 bits; maybe allow per test timeouts. #57

Closed
brewmanz opened this issue Sep 14, 2020 · 3 comments

Comments

@brewmanz
Copy link
Contributor

brewmanz commented Sep 14, 2020

Timeout limit of 255 seconds too small; I wanted around 10 minutes so set it to 999 (no compiler warning) and of course it timed out.
I suggest that TimeoutType be changed from uint8_t to uint16_t. I have not checked what follow-on changes may be needed.

PS Maybe allow per test timings?

@bxparks
Copy link
Owner

bxparks commented Sep 14, 2020

Can you expand on why you need such a long timeout? I never imagined unit tests on microcontrollers taking 10 minutes. I've never needed to increase the default timeout of 10 seconds.

Compiler warnings: You need to set Compiler Warnings to All. Setting setTimeout(999) in tests/AUnitMetaTest.ino, gives me the following warning on Arduino IDE 1.8.13:

AUnitMetaTest.ino: In function 'void setup()':
AUnitMetaTest.ino:357:29: warning: large integer implicitly truncated to unsigned type [-Woverflow]

Per test timeout: It used to be on my todo list (see issue #10) but I closed it since I have never needed it, and realistically don't have the time to implement it.

@brewmanz
Copy link
Contributor Author

I am surprised that the writer of a Unit Test suite should ask why someone wants to run a lot of tests. The fact that they do run on a slow microcontroller is one reason why they run for so long. The reason why the Pentium chip had a hardware bug was because they didn't do enough testing. Similarly with HP's calculators, which I was using back in the '70's.

Unit Testing is very important to me. In some circles I am known as 'Mister Unit Tests'; I have reported two new bugs to Microsoft with Excel, several to Ashton-Tate / Borland with dBase 3 & 4, raised bugs with RedHat with the gcc compiler handling of atexit(), and found bugs with the dotNet optimiser.

I am writing a couple of functions for the Arduino (an extension to Rob Tillaart's FastMap, and a faster integer-based sine calculation) and unexpectedly encountered some rounding issues that occur maybe 1% of the time; my theoretical approach was flawed; I see why now, and so I am being pragmatic about more in-depth testing. For 16-bit maths my tests take 7.8 secs; for 32-bit they take quite a bit longer (even though they're not as exhaustive) at 65 seconds a test, and there are 12 of them.

And so, lots of testing takes lots of CPU cycles.

Another problem is that you make the datatype for timeout seconds opaque; rather than using a built-in integer type, the function has a signature with datatype TimeoutType, so a second lookup jump is needed to find what the real datatype is: uint8_t. Thou doth abstract too much, methinks.

@bxparks
Copy link
Owner

bxparks commented Sep 15, 2020

No need to be defensive, I commend you for writing unit tests. I am asking why your tests are taking a long time, not why you are writing so many tests. None of mine take more than 10 seconds.

TimeoutType is a typedef, not separate type, and it is identical to uint8_t as far as the compiler is concerned. Obviously, it will allow changing its type to a different type more easily. If you want help from me, who is a complete stranger to you and don't owe you anything, I think you should refrain from making smug side comments.

bxparks added a commit that referenced this issue Sep 15, 2020
… uint8_t, for an upper limit of 18 hours (see #57)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants