Prevent runaway lock timeout

[dfarr]

On lock heartbeat we set the new lock timeout as follows:

timeout = timeout + expiry

However, heartbeat requests may occur more frequently than once each time the lock is set to expire. For example, here is a lock that is acquired at t=0 with an initial timeout of t=60000 (one minute) where we heartbeat every 15s. The intent is for the timeout to always stay one minute ahead of the most recent heartbeat request, but we see it runaway.

heartbeat	expiry_in_ms	timeout	timeout' (after pr)
	60,000	60,000	60,000
15,000	60,000	120,000	75,000
30,000	60,000	180,000	90,000
45,000	60,000	240,000	105,000
60,000	60,000	300,000	120,000

This change is achieved by changing the calculation for the new timeout to:

timeout = tick_time + expiry

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.27%. Comparing base (6a41367) to head (147ccea).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #280   +/-   ##
=======================================
  Coverage   62.27%   62.27%           
=======================================
  Files         113      113           
  Lines        9706     9707    +1     
=======================================
+ Hits         6044     6045    +1     
  Misses       3169     3169           
  Partials      493      493           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[guergabo]

lgtm