async API

[douglas-raillard-arm]

Add an async API to devlib to take advantage of concurrency more easily, both at a coarse and fine grain.

Note: This currently requires python 3.7

Based on #568

[douglas-raillard-arm]

Preliminary results on:

        with target.cpufreq.use_governor('performance'):
            pass

1.4x faster on a SSH target (juno r0):

• on this branch: 3.7s (avg over 10 runs)
• on master: 5.3s (avg over 10 runs)

2.2x faster on a local target:

• on this branch: ~9s
• on master: ~20s

[douglas-raillard-arm]

I checked all instances of threading.Lock in devlib and it should be safe, as none is in the path used when running a background command or any other coroutine.

If a function taking a threading.Lock was being called by 2 coroutines that end up running concurrently, the 2nd coroutine would wait for the lock, thereby blocking the main thread and deadlocking.

[douglas-raillard-arm]

I also tried various combinations of thread pools, including running a blocking execute() in a different thread (so with separate SSH connections) and I did not get any real timing variations.

What I did get is some errors probably because I was maintaining too many connections on the SSH server, so the current approach with one connection and many concurrent SSH channels seems to be the best. I could not saturate the server with open channels (maybe there is no limit). If that was to happen, we could use an asyncio.Semaphore to limit the number of bg commands in flight.

[douglas-raillard-arm]

There are a few things to cleanup (docstring etc) so if you are happy with the API as it stands I will proceed with:

• adding docstrings
• renaming asyn.parallel into asyn.concurrently
• remove the .concurrent attribute, as it is not that useful in practice I think (it allows running the function concurrently, only blocking to get the result when it is awaited. The "normal" couroutine behavior is to not do anything until the return value is awaited). asyn.concurrently is usually better as it makes it obvious what runs concurrently and avoids any kind of "task leak".

EDIT: cleanup done. Beyond possible high level doc and converting more bits, the change is somewhat ready.

[douglas-raillard-arm]

While working on a bug fix for another problem, it came to my attention that SSH servers accept a fixed number of "sessions" per connection. This "sessions" seem to map to paramiko's channels. This means that for a given connection, we might be limited to e.g. 10 concurrent channels (default of OpenSSH). This is configurable with MaxSessions on OpenSSH side and results in an exception on paramiko side: Could not open an SSH channel: ChannelException(2, 'Connect failed')

I'll probably need to handle that in that PR to limit the number of opened channels. There does not seem to be a standard way of getting the max number of channels, but we can probably handle the exception as a hint, or attempt to read OpenSSH config and parse it to find the MaxSessions value (a bit hacky though).

[douglas-raillard-arm]

@setrofim @marcbonnici I've updated the PR with an automatic detection of number of allowed background commands, along with some automatic limiting in the async API to use at most half of the available channels.

Next step:

• Decide what version of Python we want to support
• "instrument" read_value() and write_value() such that we can detect when we are trying to read/write to the same files concurrently. This should catch most issues where we are trying to do some setup concurrently, which is where the bulk of the speed up will come from.

[douglas-raillard-arm]

Updated the PR with checks to make sure that no file access conflicts when executing concurrent asyncio tasks. An artificial example:

from devlib.utils.asyn import *

m = ConcurrentResourceManager()

r1 = FileResource('host', 'file1', 'w')
r2 = FileResource('host', 'file1', 'r')

async def f1():
    # m.track_resource(r1)
    m.track_resource(r2)
    print('from f1')


async def f2():
    m.track_resource(r1)
    m.track_resource(r2)
    print('from f2')

async def f3():
    # m.track_resource(r1)
    print('from f3')

run(
    m.concurrently(
        (
            f1(),
            m.concurrently(
                (
                    f2(),
                    f3(),
                )
            )
        )
    )
)

In the PR, ConcurrentResourceManager is target.resource_manager, which is thread-local

Note: this depends on Python >= 3.7 for asyncio.current_task()

[douglas-raillard-arm]

FWIW we have been using this PR in Lisa for a month now (Lisa repo vendorizes devlib as a git subtree) and so far I haven't observed or heard of any issue related to that

[douglas-raillard-arm]

@marcbonnici PR updated with:

• Fix to connect(max_async)
• Fix to cpufreq.use_governor() racy file write
• Fix to async.asyncf() decorator for async generators: async generators are now consumed completely when crossing a blocking boundary, rather than failing to await on them. We could maybe do something a bit smarter to preserve lazyness provided that we can asyncio.run() anywhere, so I'll have a look later.
• Added asyn.memoized_method() decorator that works for both async and non-async code. This does not memoize non-hashable data so it will avoid issues described here: Surprising memoized behavior for mutable containers #341
• Converted the final bits of the cpufreq module to async

EDIT: Forgot to add the last bullet entry

[douglas-raillard-arm]

PR re-updated with a lazy async generator blocking shim. This preserves the laziness of the async gen, made possible by nest_asyncio package.