Flush async worker queue immediately on exit

[c24t]

This diff changes the behavior of AsyncTransport such that its worker thread starts exporting items in its queue immediately when the process exits.

Currently the worker thread exports a batch of items from the queue at an interval of _WAIT_PERIOD, and waits for up to grace_period on exit to finish draining the queue. Since we don't trigger an export on exit this means that the thread can block for up to _WAIT_PERIOD before doing any work, while preventing the process from exiting.

#354 changed _WAIT_PERIOD from 1s to 60s. The default grace_period is 5s. Before #354, any process that spawned a worker thread to be killed on exit would block for up to 1s (plus the time to export pending items), after the change these processes would block for up to 5s. As a result, tests that created a lot of AsyncTransports -- like TestStackdriverStatsExporter -- would block for a long time on exit.

I think this is the behavior we want, but it is a significant change. On exit, the transport will now export data in a tight loop for up to grace_period, when before it would never export items at a rate faster than batch_size / _WAIT_PERIOD.

[c24t]

Compare some offending tests on this branch and master:

python -m pytest -s 'tests/unit/stats/exporter/test_stackdriver_stats.py::TestStackdriverStatsExporter'

On my machine this hangs for over a minute after the tests complete on master and finishes in under a second on this branch.

[liyanhui1228]

LGTM

[c24t]

Thanks @liyanhui1228. I'll make the same changes in the trace package with #386.

[ocervell]

Thanks for this. Have we released a OC version with this change in ?

[c24t]

@ocervell not yet, and we don't have an ETA for the next release.

Obviously not a great solution, but if you want to use bleeding-edge unreleased changes in the meantime you can do an editable install, e.g.:

pip install -e '[email protected]:census-instrumentation/opencensus-python.git@32ee2c8#egg=opencensus-python'