Instability with executing commands and file operations in LXD virtual machine.

[yhaliaw]

Encountering errors in the log when executing commands in virtual machine instance. Python script, env info, and sample logs is included in the end.

A virtual machine instance is created and started with wait=True.
Executing commands with execute will return with return code -1.
File operations will still fail with LXD VM agent isn't currently running.

This part seems to be related to LXD VM agent. Is there a way to ensure it is running after instance is started with wait=True?

After the LXD VM agent seems to be running as file operation no longer fails, logs such as the follow can still be observed when executing commands:

DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/543f05e1-8b56-4c02-b6f8-7427802ae727 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/543f05e1-8b56-4c02-b6f8-7427802ae727 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'

environment:
ubuntu-22.04

package versions:

❯ pip list            
Package             Version
------------------- ---------
black               23.3.0
certifi             2022.12.7
cffi                1.15.1
charset-normalizer  3.1.0
click               8.1.3
cryptography        40.0.2
idna                3.4
mypy-extensions     1.0.0
packaging           23.1
pathspec            0.11.1
pip                 22.2.2
platformdirs        3.2.0
pycparser           2.21
pylxd               2.3.1
python-dateutil     2.8.2
requests            2.28.2
requests-toolbelt   0.10.1
requests-unixsocket 0.3.0
setuptools          63.2.0
six                 1.16.0
tomli               2.0.1
urllib3             1.26.15
ws4py               0.5.1

Script:

#!/usr/bin/env python

import logging
import random
import tempfile
import time

import pylxd
import pylxd.models

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)


TEST_FILE = "/home/ubuntu/test_file.bin"
FILE_SIZE = 10 * 1000 * 1000  # 10 MB


def launch_instance(client: pylxd.Client, name: str) -> pylxd.models.Instance:
    instance_config = {
        "name": name,
        "type": "virtual-machine",
        "source": {
            "type": "image",
            "mode": "pull",
            "server": "https://cloud-images.ubuntu.com/daily",
            "protocol": "simplestreams",
            "alias": "jammy",
        },
        "ephemeral": True,
        "profiles": ["default"],
    }

    return client.instances.create(config=instance_config, wait=True)


def write_random_data_file(instance: pylxd.models.Instance, filename: str, size: int):
    data = random.randbytes(size)

    instance.files.put(filename, data)
    logger.info("Written random data to %s", filename)


def read_file(instance: pylxd.models.Instance, filename: str, size: int):
    data = instance.files.get(filename)
    logger.info("Read %i bytes of data from %s: %s", size, filename, data[:size].hex())


def echo_hello(instance: pylxd.models.Instance):
    result = instance.execute(["/usr/bin/echo", "hello"])
    logger.info(
        "Echoing hello: [%i, %s, %s]",
        result.exit_code,
        result.stdout,
        result.stderr,
    )


def main():
    logger.info("Starting up...")

    client = pylxd.Client()
    instance = launch_instance(client, "test")
    instance.start(wait=True)

    logger.info("Instance started")

    logger.info("Testing some operations...")
    for _ in range(10):
        try:
            echo_hello(instance)
            write_random_data_file(instance, TEST_FILE, FILE_SIZE)
            read_file(instance, TEST_FILE, 10)
        except Exception:
            logger.exception("Errors encountered.")

        time.sleep(5)

    instance.stop(wait=True)

    logger.info("Cleanup completed.")


if __name__ == "__main__":
    main()

Sample log:

❯ python pylxd_test.py
INFO:__main__:Starting up...
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0 HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances HTTP/1.1" 202 514
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/ba4a7039-a1d8-4c7c-a93a-612442e3bb88 HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/ba4a7039-a1d8-4c7c-a93a-612442e3bb88/wait HTTP/1.1" 200 613
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/ba4a7039-a1d8-4c7c-a93a-612442e3bb88 HTTP/1.1" 200 613
DEBUG:urllib3.connectionpool:http://localhost:None "PUT /1.0/instances/test/state HTTP/1.1" 202 514
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/08c0ee13-0fa3-410d-a4e5-9830a098c21b HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/08c0ee13-0fa3-410d-a4e5-9830a098c21b/wait HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/08c0ee13-0fa3-410d-a4e5-9830a098c21b HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test HTTP/1.1" 200 1916
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test HTTP/1.1" 200 1916
INFO:__main__:Instance started
INFO:__main__:Testing some operations...
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
ERROR:ws4py:Terminating websocket [Bound to ''] due to exception: BrokenPipeError(32, 'Broken pipe') in once method
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Terminating websocket [Bound to ''] due to exception: BrokenPipeError(32, 'Broken pipe') in once method
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/3d738cbf-b9a5-482b-be42-f27d7dccdc24 HTTP/1.1" 200 987
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [-1, , ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 500 141
ERROR:__main__:Errors encountered.
Traceback (most recent call last):
  File "/home/user/work/tmp/pylxd_test.py", line 72, in main
    write_random_data_file(instance, TEST_FILE, FILE_SIZE)
  File "/home/user/work/tmp/pylxd_test.py", line 40, in write_random_data_file
    instance.files.put(filename, data)
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/models/instance.py", line 114, in put
    response = self._endpoint.post(
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/client.py", line 216, in post
    self._assert_response(response, allowed_status_codes=(200, 201, 202))
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/client.py", line 145, in _assert_response
    raise exceptions.LXDAPIException(response)
pylxd.exceptions.LXDAPIException: LXD VM agent isn't currently running
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
ERROR:ws4py:Terminating websocket [Bound to ''] due to exception: BrokenPipeError(32, 'Broken pipe') in once method
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Terminating websocket [Bound to ''] due to exception: BrokenPipeError(32, 'Broken pipe') in once method
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/37f202ec-c1cc-471e-b819-a16532df6766 HTTP/1.1" 200 987
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [-1, , ]
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 500 141
ERROR:__main__:Errors encountered.
Traceback (most recent call last):
  File "/home/user/work/tmp/pylxd_test.py", line 72, in main
    write_random_data_file(instance, TEST_FILE, FILE_SIZE)
  File "/home/user/work/tmp/pylxd_test.py", line 40, in write_random_data_file
    instance.files.put(filename, data)
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/models/instance.py", line 114, in put
    response = self._endpoint.post(
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/client.py", line 216, in post
    self._assert_response(response, allowed_status_codes=(200, 201, 202))
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/pylxd/client.py", line 145, in _assert_response
    raise exceptions.LXDAPIException(response)
pylxd.exceptions.LXDAPIException: LXD VM agent isn't currently running
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1000
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/31869010-d3f6-4f57-94b0-ae625cfaad0b HTTP/1.1" 200 937
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/31869010-d3f6-4f57-94b0-ae625cfaad0b HTTP/1.1" 200 949
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: d199e204821c5ed196bc
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/29270d47-211f-49e9-a928-1aa615296ab1 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/29270d47-211f-49e9-a928-1aa615296ab1 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: bb1e44c925f72685979f
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/4f2e9e21-41aa-4702-942d-03c860b13723 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/4f2e9e21-41aa-4702-942d-03c860b13723 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: a7d404771490d02c82cb
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/104ba683-6bfa-44a1-8442-197fa96776aa HTTP/1.1" 200 939
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/104ba683-6bfa-44a1-8442-197fa96776aa HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: 8f2c42f4cf2c75ff620f
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/da8e1579-80bf-4042-9f4e-1fab6a743f45 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/da8e1579-80bf-4042-9f4e-1fab6a743f45 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: 06a126f754e76114d997
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/06a10cf3-996c-4d40-a580-41c7a4352526 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/06a10cf3-996c-4d40-a580-41c7a4352526 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: 1e24c1ac08f0afb6b3b2
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/247745dc-d303-4367-884f-7f2cdbc874ae HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/247745dc-d303-4367-884f-7f2cdbc874ae HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: 96ad3b63dbbe41e86bc2
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/exec HTTP/1.1" 202 1002
INFO:ws4py:Using epoll
INFO:ws4py:Managing websocket [Bound to '']
INFO:ws4py:Managing websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/543f05e1-8b56-4c02-b6f8-7427802ae727 HTTP/1.1" 200 939
INFO:ws4py:Terminating websocket [Bound to '']
ERROR:ws4py:Failed to receive data
Traceback (most recent call last):
  File "/home/user/work/tmp/venv/lib/python3.10/site-packages/ws4py/websocket.py", line 394, in once
    b = self.sock.recv(self.reading_buffer_size)
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:ws4py:Terminating websocket [Bound to '']
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/543f05e1-8b56-4c02-b6f8-7427802ae727 HTTP/1.1" 200 950
INFO:ws4py:Closing all websockets with [1001] 'Server is shutting down'
INFO:__main__:Echoing hello: [0, hello
, ]
DEBUG:urllib3.connectionpool:http://localhost:None "POST /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 108
INFO:__main__:Written random data to /home/ubuntu/test_file.bin
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/instances/test/files?path=%2Fhome%2Fubuntu%2Ftest_file.bin HTTP/1.1" 200 10000000
INFO:__main__:Read 10 bytes of data from /home/ubuntu/test_file.bin: a98f105fe3de7e2e0991
DEBUG:urllib3.connectionpool:http://localhost:None "PUT /1.0/instances/test/state HTTP/1.1" 202 514
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/9bece8d3-1496-48c0-9b27-9110316f70e6 HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/9bece8d3-1496-48c0-9b27-9110316f70e6/wait HTTP/1.1" 200 451
DEBUG:urllib3.connectionpool:http://localhost:None "GET /1.0/operations/9bece8d3-1496-48c0-9b27-9110316f70e6 HTTP/1.1" 200 451
INFO:__main__:Cleanup completed.

[stgraber]

I can't really comment on pylxd/ws4py, but for the LXD agent, you need to setup some kind of loop to see when the VM is connected. LXD itself doesn't know when the agent is ready as that would require frequent polling of all the VMs. Instead it just tries to interact with the agent as needed.

In shell, you'd probably want to do something like:

lxc list -cN -fcsv NAME

If you get -1, no agent is responding, if you get a value higher than 0, then you're talking to an agent.
An alternative is a busy loop of lxc exec NAME /bin/true 2>/dev/null

[sirkadirov]

@yhaliaw, are you using pylxd package available from pip? In my case #376 all errors were caused by an ols bug with pip version of the package. Try to use latest package from GitHub repo and check if this solves your issue.

P.S. Here's the command to install latest pylxd package

pip install git+https://github.com/lxc/pylxd