-
Notifications
You must be signed in to change notification settings - Fork 195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PeriodicThread accumulates error over time #2488
Comments
cc @isorrentino this could explain the drift we saw today in velocity data |
Oops, the console output I pasted in the description must be mixing two different runs, if you compare start and end timestamps. In any case the total drift was indeed 0.15 s. I have just tested this out of curiosity on WSL 2 / Ubuntu focal / YARP 3.4.2+2-20210119.16+git444e855e9 and got a drift of 0.67 seconds in 1000 iterations (50 seconds total):
|
yarp/src/libYARP_os/src/yarp/os/PeriodicThread.cpp Lines 156 to 204 in 292193e
Question: shouldn't |
@PeterBowman Thanks for the analysis and the graphs. I remember discussing this with someone a long time ago, but I've been told that this is the way it is supposed to be, because the If there are actually 2 different use cases, it shouldn't be difficult to add an extra parameter to the constructor to switch between relative and absolute sleep time. |
I would very much like to use the other implementation, honestly. |
A small clarification @PeterBowman : in original post #2488 (comment) were you using a NetworkClock or not? If not, how is #2488 (comment) is related to the original problem? |
Sorry, I'm not sure to understand the reason why you would want this... Using the system clock in a simulation, means that if the system clock is faster than the network clock, the thread would be resumed, it would check if enough time has passed (network time, not real time, which is not passed yet), and therefore go back to sleep for several times. Instead, the network clock is used, and (if I remember correctly) the network clock implementation checks which threads should be resumed whenever a new clock signal is received. Therefore this will adjust the actual sleep time properly, according to the simulation.
I'm not sure to understand this, are you using the network clock or a custom class that derives from |
To my best understanding, the fact that the current implementation attempts to account for elapsed time tells me it was probably designed to ensure that exactly Real-time control aside, I find the
@traversaro I'm sorry, my previous comment (network clock + elapsed time) is not directly related to the original description (drifts, using system clock). I have posted it in case their respective solutions conflict with each other and thus they should be considered as a whole. @drdanz let me prepare a sample app to illustrate/double-check my point. Shortly, I believe the behavior you describe in this sentence is actually the opposite to my experience:
|
In the following example I build two executables: the clock app which starts a network clock with a refresh rate of 5 milliseconds, and the reader app which iterates over a loop with the exact same period. Real use case and rationale: clock publishes a sync event over the network and an external controller (reader) catches up, acts accordingly and sends a setpoint back to the robot, awaiting the next synchronization (roboticslab-uc3m/yarp-devices#254). CMakeLists.txtcmake_minimum_required(VERSION 3.12 FATAL_ERROR)
project(yarp-clock LANGUAGES CXX)
find_package(YARP 3.4 REQUIRED COMPONENTS os)
add_executable(clock main-clock.cpp)
target_link_libraries(clock YARP::YARP_os YARP::YARP_init)
add_executable(reader main-reader.cpp)
target_link_libraries(reader YARP::YARP_os YARP::YARP_init)
set_target_properties(clock reader PROPERTIES COMPILE_DEFINITIONS CLOCK_PERIOD=0.005) main-clock.cpp#include <cmath>
#include <yarp/os/Bottle.h>
#include <yarp/os/BufferedPort.h>
#include <yarp/os/Network.h>
#include <yarp/os/SystemClock.h>
int main()
{
yarp::os::Network yarp(yarp::os::YARP_CLOCK_SYSTEM);
yarp::os::BufferedPort<yarp::os::Bottle> port;
port.open("/myclock");
auto count = 0;
const auto initial = yarp::os::SystemClock::nowSystem();
while (true)
{
// Publish current system time (network time for listeners).
auto now = yarp::os::SystemClock::nowSystem();
int secs = std::trunc(now);
int nsecs = std::trunc((now - std::trunc(now)) * 1e9);
port.prepare() = {yarp::os::Value(secs), yarp::os::Value(nsecs)};
port.write();
// Compute delay for current step. Use initial time as reference
// and get current time again to account for elapsed time in this step.
auto delay = initial + CLOCK_PERIOD * ++count - yarp::os::SystemClock::nowSystem();
yarp::os::SystemClock::delaySystem(delay);
}
return 0;
} main-reader.cpp#include <yarp/os/LogStream.h>
#include <yarp/os/Network.h>
#include <yarp/os/SystemClock.h>
#include <yarp/os/Time.h>
int main()
{
yarp::os::Network yarp;
yarp::os::Time::useNetworkClock("/myclock");
int count = 0;
while (count++ < 2000)
{
auto now = yarp::os::SystemClock::nowSystem();
yInfo("[%d] Network time: %f", count, yarp::os::Time::now());
auto elapsed = yarp::os::SystemClock::nowSystem() - now;
yarp::os::Time::delay(CLOCK_PERIOD - elapsed);
}
return 0;
} The network clock caches the last timestamp received via "/myclock", so any call to Now, please note I am accounting for real (system clock-driven) elapsed time in the delay, thus substracting a few microseconds from the clock period. Had I used Since for-loops are ugly, I would mostly like to rewrite reader using a main-reader.cpp (now with a PeriodicThread)#include <yarp/os/LogStream.h>
#include <yarp/os/Network.h>
#include <yarp/os/PeriodicThread.h>
#include <yarp/os/SystemClock.h>
#include <yarp/os/Time.h>
class Worker : public yarp::os::PeriodicThread
{
public:
Worker() : yarp::os::PeriodicThread(CLOCK_PERIOD) {}
protected:
void run() override
{
yInfo("[%d] Network time: %f, period: %f, used: %f",
getIterations() + 1, yarp::os::Time::now(), getEstimatedPeriod(), getEstimatedUsed());
}
};
int main()
{
yarp::os::Network yarp;
yarp::os::Time::useNetworkClock("/myclock");
Worker worker;
worker.start();
while (worker.getIterations() < 2000) yarp::os::SystemClock::delaySystem(0.1);
worker.stop();
return 0;
} Output:
Almost every second step is missed because of not accounting for real elapsed time: see how reported period is nearly 0.01 milliseconds instead. Used (elapsed) time is zero since internal calls to After applying the fix proposed in PeterBowman@77f41d3:
Misses still may occur rarely (my real use case involves using a real timer instead of a loop over the network clock, and doing a bit more work in the reader thread so that elapsed time will be noticeable), but my point here is that elapsed time is making a difference. Besides, stats work again ( |
I'm not sure about this. The catch here is that the thread is not actually performing a delay, but it is blocked, waiting for a signal from the network clock thread. Whenever it receives that signal, the thread restarts without checking if the time is actually passed. This is the theoretic working principles.
[*] assuming that the clock did not receive a new time in the meanwhile which for a task that is shorter than the period should happen most of the times If this does not happen, then there is probably a bug in the implementation, but the system time does not have any role here. I think you are assuming that the system time and the network time can be replaced because your clock is running exactly at the system time, but you should consider that this is not the only use case. The network time can be faster or slower than the system time (which is totally expected in a simulation). Actually, this is a very specific use case since you also want the period to be exactly the same as the network clock, and you have a task that takes a very small time. You should make some easy experiment, changing the period in the thread, adding a delay in your run() and adding a multiply factor to your clock, i.e. something like // Publish current system time (network time for listeners).
static constexpr double realtimefactor = 0.5; // Or 2 for a clock that is faster than real time
auto now = yarp::os::SystemClock::nowSystem() * realtimefactor;
int secs = std::trunc(now);
int nsecs = std::trunc((now - std::trunc(now)) * 1e9);
port.prepare() = {yarp::os::Value(secs), yarp::os::Value(nsecs)};
port.write(); For example, assuming
Therefore if you are using a network clock you will request a delay of TL;DR: I think that there might be be a bug in current implementation, but using the system clock here is wrong. |
Thank you very much for taking your time to review this, @drdanz, I agree with you. I have realized my assumptions were incomplete in two fundamental ways:
I am pursuing a specific functionality which the
The I brought up this question because I feel something around elapsed time should be done while fixing the original problem (drifts). In fact, I didn't need to account for it while estimating the delay that drives the loop in main-clock.cpp. So perhaps something like this might work (ref): delayFunc(initialTime + period * count - nowFunc()); |
Describe the bug
yarp::os::PeriodicThread
wraps a callback method (run()
) executed repeatedly in a separate thread given a fixed period. It accounts for the time elapsed while processing this callback so that the thread is slept until the next iteration without exceeding said period, i.e.sleepTime = fixedPeriod - elapsedTime
. However, I have noticed a drift is introduced due to inaccuracies in estimating the sleep time.To Reproduce
The following program instantiates a
PeriodicThread
with a fixed period of 50 ms. On each step, it prints the current iteration, current real time, and expected time.Output showing first few iterations, and ~1000:
As you can see, the timer has drifted away ~0.15 seconds, i.e. 3 times the period, over 1000 iterations (50 seconds). (edit: initial timestamps correspond to a previous run, sorry, in any case the results were consistent)
Expected behavior
Certain applications require strict timing using a fixed period, e.g. xPDO cyclic synchronous communication in CANopen standard. I expect that
yarp::os::PeriodicThread
(also used byyarp::os::Timer
under the hood) callsrun
at exactlyperiod*N_iter
seconds from start, no matter how long it has been running. We've noticed this problem because a timer implemented by hand as shown below does the trick (for simple cases):I am therefore proposing to set an initial static time reference from which to substract on each iteration.
Screenshots
In this plot two sequences are depicted: red is for a correct fixed period as in the above Python snippet, and blue for a faulty
PeriodicThread
. Y-values should come in blue-red pairs at constant offset (initial instants differ), but a drift in X-values (timestamps) is noticeable. Zoom in/out (double click) and pan the graph to check how blue points are shifted along the entire time domain.Configuration
The text was updated successfully, but these errors were encountered: