[feature request] Thread-safety for asserts and logging facilities #4
Comments
|
If you are doing assertions in multiple threads this is almost the same as running several test in parallel, isn't it? One additional question. Do you have any pointers to related issues in CATCH or Boost.Test? |
|
This will definitely be added to the roadmap. Currently there is no synchronization happening. |
|
@viboes The feature to run the tests in parallel would require more work than that on doctest I think. But as you said, I'm just asking if doctest is threadsafe (and that's indeed a better way to put it). About the related issue on catch, see here, I don't know if an issue is open on Boost.Test. @onqtam Thanks! |
|
@blastrock if no one creates an issue to Boost.Test no fix could be expected ;-) The question is also if Boost.Test document that it is thread-safe. @onqtam Adding synchronization always will make the test execution slower ("Don't pay for what you don't use"). It would be great if the tester could opt-in this feature and let most of the test continue to run as fast as possible. |
|
@viboes yes - the default case will be without synchronization :) But no guarantees when this will be done. |
|
@onqtam Making DocTest thread-safe will add additional dependencies or move to C++11. In addition it is not an easy task. I would suggest you to concentrate on a mono-threaded version. You can document that the user must avoid using the library on several threads and that it is up to them to do the synchronization between the different threads they are launching. The fact that other unit test frameworks are not thread-safe is a symptom that it is not simple. |
|
I've written a few test frameworks and all of them are not thread safe. The basic problem is that as soon as you have threading, you have an undefined synchronicity in timing, meaning that the reproductivity of your test results goes out the window. IE, your unit tests will be bad and hard to fix. Your tests should guarantee a single unique way of running; that's what a unit test is for quite literally, to ensure that you test that a given path works. If you have threading you should ensure that it does not interfere with the reliability of your tests, and that implicitly removes any use you can get out of this. Running multiple tests in parallel I'm in favor, but I would feel that you're likely to hit slightly non-threadsafe code and you'll get unreproducible problems right back at you. |
|
@dascandy In my case, I want to test a multithreading library. I know that I won't get reproducibility, but still, it's better to have tests that may fail when there is a bug than no tests at all. I guess that people that rely on these multithreading libraries may have the same constraints (that their code may run in parallel and in a non-deterministic way). |
|
I just pushed the thread safety support in commit 60e8a45 in the dev branch! asserts can now be safely used in multiple threads! what remains is the logging functionality... thanks to @blastrock for showing me how simple this was (once I switched to C++11 for 2.0) in his PR! I will not promise a release date however... but the work I have left for version 2.0 isn't much - all I have to do is implement an xml reporter with the new IReporter interface - but I should research xml encoding and other non-interesting things... :D (and document all new features...) so I hope that is... in the near month. |
…on - no need to allocate streams anymore - reusing a single one) - relates #4
…thread local storage (__thread and __declspec(thread) respectively) but it works only for POD types... relates #4
|
logging functionality can now be used in a thread-safe manner as well - at the cost of dropping GCC 4.7 and VS 2013 from the list of supported compilers. not documented yet, and also no thread-sanitizer CI builds yet. |
…reporter for windows when attached to a VS debugger) - relates #4
…support for thread_local - relates #4 - see this: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c11-thread-local-when-official-clang-supports
onqtam
changed the title
…on - no need to allocate streams anymore - reusing a single one) - relates #4
…thread local storage (__thread and __declspec(thread) respectively) but it works only for POD types... relates #4
…reporter for windows when attached to a VS debugger) - relates #4
…support for thread_local - relates #4 - see this: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c11-thread-local-when-official-clang-supports
|
Forgot to close this with the release of 2.0.0 |
Hi,
I would like to know if this framework supports tests that are multithreaded. To clarify, I am not talking about running multiple tests in parallel, but about running one single test that involves multiple threads doing assertions in parallel. I know that Catch and Boost.Test crash when we do that, but I didn't see anything in doctest's feature page nor in its roadmap about this. Is this supported? Will it be?
The text was updated successfully, but these errors were encountered: