From bdc7de9ae9cb57cea0f15d48136287e2f4f23b3a Mon Sep 17 00:00:00 2001 From: David Minarsch Date: Sun, 3 Nov 2019 21:08:48 +0000 Subject: [PATCH 01/52] Minor doc updates --- docs/app-areas.md | 2 +- docs/core-components.md | 2 +- docs/gym-skill.md | 2 +- docs/index.md | 2 +- docs/oef-ledger.md | 2 +- docs/protocol.md | 4 ++-- docs/trust.md | 2 +- docs/weather-skills.md | 4 ++-- 8 files changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/app-areas.md b/docs/app-areas.md index ee9ba036f1..6faad3ff02 100644 --- a/docs/app-areas.md +++ b/docs/app-areas.md @@ -8,7 +8,7 @@ There are five general application areas for Fetch.ai AEAs. * **Digital data sales agents**: pure software agents that attach to data sources and sell it via the open economic framework. * **Representative**: an agent which represents an individual's activities on the Fetch.ai network. -## Multi-agent system vs agent-based modelling +## Multi-agent system versus agent-based modelling The Fetch.ai multi agent system is a real world multi-agent technological system and, although there is some overlap, it is not the same as agent based modelling where the goal is scientific behaviourial observation rather than practical economic gain. diff --git a/docs/core-components.md b/docs/core-components.md index 1ab263f1a5..6ed4bc56bb 100644 --- a/docs/core-components.md +++ b/docs/core-components.md @@ -56,7 +56,7 @@ A skill encapsulates implementations of the abstract base classes `Handler`, `Be * `Handler`: each skill has none, one or more `Handler` objects, each responsible for the registered messaging protocol. Handlers implement agents' reactive behaviour. If the agent understands the protocol referenced in a received `Envelope`, the `Handler` reacts appropriately to the corresponding message. Each `Handler` is responsible for only one protocol. A `Handler` is also capable of dealing with internal messages. * `Behaviour`: none, one or more `Behaviours` encapsulate actions that cause interactions with other agents initiated by the agent. Behaviours implement agents' proactiveness. -* `Task`: none, one or more Tasks encapsulate background work internal to the agent. +* `Task`: none, one or more `Tasks` encapsulate background work internal to the agent. ## Agent diff --git a/docs/gym-skill.md b/docs/gym-skill.md index 8212af2e19..96f7e78da9 100644 --- a/docs/gym-skill.md +++ b/docs/gym-skill.md @@ -1,4 +1,4 @@ -The AEA gym skill demonstrates how a custom Reinforcement Learning agent, that uses openai's gym library, may be embedded into an Autonomous Economic Agent. +The AEA gym skill demonstrates how a custom Reinforcement Learning agent, that uses OpenAI's gym library, may be embedded into an Autonomous Economic Agent. ## Demo instructions diff --git a/docs/index.md b/docs/index.md index 9b324b8e84..d196504d6a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,7 +2,7 @@ Do you want to create software to work for you and enrich your life? -Autonomous Economic Agents - or AEAs - work continously for your benefit without you having to do anything more than write them and start them up. +Autonomous Economic Agents (AEAs) work continously for your benefit without you having to do anything more than write them and start them up. AEAs are able to act independent of your constant input and to autonomously develop new capabilities. Their goal is to create economic gain for you, their owner. diff --git a/docs/oef-ledger.md b/docs/oef-ledger.md index b0266a6669..7fb081e404 100644 --- a/docs/oef-ledger.md +++ b/docs/oef-ledger.md @@ -1,4 +1,4 @@ In the AEA framework universe, agents run alongside OEF search and discovery nodes against the Fetch.ai ledger and external ledger systems. -
![The AEA, OEF, and Ledger systems](assets/oef-ledger.png)
\ No newline at end of file +
![The AEA, OEF, and Ledger systems](assets/oef-ledger.png)
diff --git a/docs/protocol.md b/docs/protocol.md index 9cd7c89399..4a6ed45cc2 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -88,7 +88,7 @@ A `models.py` module is provided by the `oef` protocol which includes classes an The `fipa` protocol definition includes a `FIPAMessage` class which gets a `protocol_id` of `fipa`. -It defines FIPA negotiating terms by way of a `Peformative(Enum)`. +It defines FIPA negotiating terms by way of a `Performative(Enum)`. ``` python class Performative(Enum): @@ -165,4 +165,4 @@ The serialisation methods `encode` and `decode` implement transformations from ` -
\ No newline at end of file +
diff --git a/docs/trust.md b/docs/trust.md index 0e94045207..84a9fcc364 100644 --- a/docs/trust.md +++ b/docs/trust.md @@ -6,4 +6,4 @@ A step up, if you run the weather skills demo with a ledger (Fetch.ai or Ethereu One could expand trustlessness even further by incorporating a third party as an arbitrator or some escrow contract. However, in the weather skills demo there are limits to trustlessness as the station ultimately offers unverifiable data. -Finally, in the case of (non-fungible) token transactions where there is an atomic swap, full trustlessness is apparent. This is demonstrated in the TAC. \ No newline at end of file +Finally, in the case of (non-fungible) token transactions where there is an atomic swap, full trustlessness is apparent. This is demonstrated in the TAC. diff --git a/docs/weather-skills.md b/docs/weather-skills.md index 2f63c6ea7b..5e33ff2d35 100644 --- a/docs/weather-skills.md +++ b/docs/weather-skills.md @@ -26,8 +26,8 @@ svn export https://github.com/fetchai/agents-aea.git/trunk/packages svn export https://github.com/fetchai/agents-aea.git/trunk/scripts ``` -## Launch the OEF Node (for search and discovery): -In a separate terminal, launch an OEF node locally: +## Launch the OEF Node: +In a separate terminal, launch an OEF node (for search and discovery) locally: ``` bash python scripts/oef/launch.py -c ./scripts/oef/launch_config.json ``` From 1fcd7d2a036d57c6b9d18608c30eaf4660716868 Mon Sep 17 00:00:00 2001 From: Aristotelis Date: Mon, 4 Nov 2019 10:08:16 +0000 Subject: [PATCH 02/52] 100% on ledger_apis --- aea/crypto/ledger_apis.py | 6 +- tests/test_crypto/test_ledger_apis.py | 167 +++++++++++++++++--------- 2 files changed, 113 insertions(+), 60 deletions(-) diff --git a/aea/crypto/ledger_apis.py b/aea/crypto/ledger_apis.py index e3e10d94b7..1373530d31 100644 --- a/aea/crypto/ledger_apis.py +++ b/aea/crypto/ledger_apis.py @@ -112,7 +112,7 @@ def token_balance(self, identifier: str, address: str) -> int: except Exception: logger.warning("An error occurred while attempting to get the current balance.") balance = 0 - else: # pragma: no cover + else: # pragma: no cover balance = 0 return balance @@ -162,12 +162,12 @@ def transfer(self, identifier: str, crypto_object: Crypto, destination_address: logger.info("transaction validated - exiting") tx_digest = hex_value.hex() break - except web3.exceptions.TransactionNotFound: + except web3.exceptions.TransactionNotFound: # pragma: no cover logger.info("transaction not found - sleeping for 3.0 seconds") time.sleep(3.0) return tx_digest - else: # pragma: no cover + else: # pragma: no cover tx_digest = None return tx_digest diff --git a/tests/test_crypto/test_ledger_apis.py b/tests/test_crypto/test_ledger_apis.py index f7bd8d676e..8ca056ba52 100644 --- a/tests/test_crypto/test_ledger_apis.py +++ b/tests/test_crypto/test_ledger_apis.py @@ -21,19 +21,26 @@ """This module contains the tests for the crypto/helpers module.""" import logging # from unittest import mock +import os +from unittest import mock import pytest +from hexbytes import HexBytes -from aea.crypto.ethereum import ETHEREUM -from aea.crypto.fetchai import FETCHAI -from aea.crypto.ledger_apis import LedgerApis, DEFAULT_FETCHAI_CONFIG # , _try_to_instantiate_fetchai_ledger_api, _try_to_instantiate_ethereum_ledger_api - +from aea.crypto.ethereum import ETHEREUM, EthereumCrypto +from aea.crypto.fetchai import FETCHAI, FetchAICrypto +from aea.crypto.ledger_apis import LedgerApis, DEFAULT_FETCHAI_CONFIG, \ + _try_to_instantiate_fetchai_ledger_api, \ + _try_to_instantiate_ethereum_ledger_api +from tests.conftest import CUR_PATH logger = logging.getLogger(__name__) DEFAULT_ETHEREUM_CONFIG = ("https://ropsten.infura.io/v3/f00f7b3ba0e848ddbdc8941c527447fe", 3) fet_address = "B3t9pv4rYccWqCjeuoXsDoeXLiKxVAQh6Q3CLAiNZZQ2mtqF1" eth_address = "0x21795D753752ccC1AC728002D23Ba33cbF13b8b0" +GAS_PRICE = '50' +GAS_ID = 'gwei' class TestLedgerApis: @@ -48,56 +55,102 @@ def test_initialisation(self): assert ledger_apis.has_ethereum unknown_config = ("UknownPath", 8080) with pytest.raises(ValueError): - ledger_apis = LedgerApis({"UNKNOWN": unknown_config}) - - # def test_token_balance(self): - # """Test the token_balance for the different tokens.""" - # ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, - # FETCHAI: DEFAULT_FETCHAI_CONFIG}) - - # with mock.patch.object(ledger_apis, 'token_balance', return_value=10): - # balance = ledger_apis.token_balance(FETCHAI, eth_address) - # assert balance == 10 - # balance = ledger_apis.token_balance(ETHEREUM, eth_address) - # assert balance == 10, "The specific address has some eth" - # with mock.patch.object(ledger_apis, 'token_balance', return_value=0): - # balance = ledger_apis.token_balance(ETHEREUM, fet_address) - # assert balance == 0, "Should trigger the Exception and the balance will be 0" - # with mock.patch.object(ledger_apis, 'token_balance', return_value=Exception): - # balance = ledger_apis.token_balance(ETHEREUM, fet_address) - # assert balance == 0, "Should trigger the Exception and the balance will be 0" - # with pytest.raises(AssertionError): - # balance = ledger_apis.token_balance("UNKNOWN", fet_address) - # assert balance == 0, "Unknown identifier so it will return 0" - # def test_transfer(self): - # """Test the transfer function for the supported tokens.""" - # private_key_path = os.path.join(CUR_PATH, "data", "eth_private_key.txt") - # eth_obj = EthereumCrypto(private_key_path=private_key_path) - # private_key_path = os.path.join(CUR_PATH, 'data', "fet_private_key.txt") - # fet_obj = FetchAICrypto(private_key_path=private_key_path) - # ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, - # FETCHAI: DEFAULT_FETCHAI_CONFIG}) - # - # with mock.patch.object(ledger_apis, 'transfer', - # return_value= "97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35"): - # tx_digest = ledger_apis.transfer(FETCHAI, fet_obj, fet_address, amount=10, tx_fee=10) - # assert tx_digest is not None - # with mock.patch.object(ledger_apis, 'is_tx_settled', return_value= True): - # assert ledger_apis.is_tx_settled(identifier=FETCHAI, tx_digest=tx_digest, amount=10) - # with mock.patch.object(ledger_apis, 'is_tx_settled', return_value= False): - # assert not ledger_apis.is_tx_settled(identifier=FETCHAI, tx_digest=tx_digest, amount=10) - # with mock.patch.object(ledger_apis, 'transfer', - # return_value="97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35"): - # tx_digest = ledger_apis.transfer(ETHEREUM, eth_obj, eth_address, amount=10, tx_fee=200000) - # assert tx_digest is not None - # with mock.patch.object(ledger_apis, 'is_tx_settled', return_value= True): - # assert ledger_apis.is_tx_settled(identifier=FETCHAI, tx_digest=tx_digest, amount=10) - # with mock.patch.object(ledger_apis, 'is_tx_settled', return_value= False): - # assert not ledger_apis.is_tx_settled(identifier=FETCHAI, tx_digest=tx_digest, amount=10) - # def test_try_to_instantiate_fetchai_ledger_api(self): - # """Test the instantiation of the fetchai ledger api.""" - # _try_to_instantiate_fetchai_ledger_api(addr="127.0.0.1", port=80) - - # def test__try_to_instantiate_ethereum_ledger_api(self): - # """Test the instantiation of the ethereum ledger api.""" - # _try_to_instantiate_ethereum_ledger_api(addr="127.0.0.1", port=80) + LedgerApis({"UNKNOWN": unknown_config}) + + def test_token_balance(self): + """Test the token_balance for the different tokens.""" + ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, + FETCHAI: DEFAULT_FETCHAI_CONFIG}) + + with mock.patch.object(ledger_apis, 'token_balance', return_value=10): + balance = ledger_apis.token_balance(FETCHAI, fet_address) + assert balance == 10 + balance = ledger_apis.token_balance(ETHEREUM, eth_address) + assert balance == 10 + + balance = ledger_apis.token_balance(FETCHAI, eth_address) + assert balance == 0, "This must be 0 since the address is wrong" + balance = ledger_apis.token_balance(ETHEREUM, fet_address) + assert balance == 0, "This must be 0 since the address is wrong" + + with pytest.raises(AssertionError): + balance = ledger_apis.token_balance("UNKNOWN", fet_address) + assert balance == 0, "Unknown identifier so it will return 0" + + def test_transfer(self): + """Test the transfer function for the supported tokens.""" + private_key_path = os.path.join(CUR_PATH, "data", "eth_private_key.txt") + eth_obj = EthereumCrypto(private_key_path=private_key_path) + private_key_path = os.path.join(CUR_PATH, 'data', "fet_private_key.txt") + fet_obj = FetchAICrypto(private_key_path=private_key_path) + ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, + FETCHAI: DEFAULT_FETCHAI_CONFIG}) + + with mock.patch.object(ledger_apis.apis.get(FETCHAI).tokens, 'transfer', + return_value="97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35"): + tx_digest = ledger_apis.transfer(FETCHAI, fet_obj, fet_address, amount=10, tx_fee=10) + assert tx_digest is not None + + with mock.patch.object(ledger_apis.apis.get(ETHEREUM).eth, 'getTransactionCount', return_value=5): + transaction = { + 'nonce': ledger_apis.apis.get(ETHEREUM).eth.getTransactionCount(), + 'chainId': 3, + 'to': eth_address, + 'value': 50, + 'gas': 2000000, + 'gasPrice': ledger_apis.apis[ETHEREUM].toWei(GAS_PRICE, GAS_ID) + } + signed = ledger_apis.apis[ETHEREUM].eth.account.signTransaction(transaction, eth_obj._account.privateKey) + with mock.patch.object(ledger_apis.apis.get(ETHEREUM).eth.account, 'signTransaction', + return_value=signed): + with pytest.raises(ValueError): + ledger_apis.apis[ETHEREUM].eth.sendRawTransaction(signed.rawTransaction) + result = HexBytes('0xf85f808082c35094d898d5e829717c72e7438bad593076686d7d164a80801ba005c2e99ecee98a12fbf28ab9577423f42e9e88f2291b3acc8228de743884c874a077d6bc77a47ad41ec85c96aac2ad27f05a039c4787fca8a1e5ee2d8c7ec1bb6a') + with mock.patch.object(ledger_apis.apis.get(ETHEREUM).eth, 'sendRawTransaction', + return_value=result): + with mock.patch.object(ledger_apis.apis.get(ETHEREUM).eth, "getTransactionReceipt", + return_value=b'0xa13f2f926233bc4638a20deeb8aaa7e8d6a96e487392fa55823f925220f6efed'): + tx_digest = ledger_apis.transfer(ETHEREUM, eth_obj, eth_address, amount=10, tx_fee=200000) + assert tx_digest is not None + + def test_is_tx_settled(self): + """Test if the transaction is settled.""" + ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, + FETCHAI: DEFAULT_FETCHAI_CONFIG}) + tx_digest = "97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35" + with pytest.raises(AssertionError): + ledger_apis.is_tx_settled("Unknown", tx_digest=tx_digest, amount=10) + + with mock.patch.object(ledger_apis.apis[FETCHAI].tx, "status", return_value='Submitted'): + is_successful = ledger_apis.is_tx_settled(FETCHAI, tx_digest=tx_digest, amount=10) + assert is_successful + + with mock.patch.object(ledger_apis.apis[FETCHAI].tx, "status", side_effect=Exception): + is_successful = ledger_apis.is_tx_settled(FETCHAI, tx_digest=tx_digest, amount=10) + assert not is_successful + + result = HexBytes( + '0xf85f808082c35094d898d5e829717c72e7438bad593076686d7d164a80801ba005c2e99ecee98a12fbf28ab9577423f42e9e88f2291b3acc8228de743884c874a077d6bc77a47ad41ec85c96aac2ad27f05a039c4787fca8a1e5ee2d8c7ec1bb6a') + with mock.patch.object(ledger_apis.apis[ETHEREUM].eth, "getTransactionReceipt", return_value=result): + is_successful = ledger_apis.is_tx_settled(ETHEREUM, tx_digest=tx_digest, amount=10) + assert is_successful + + with mock.patch.object(ledger_apis.apis[ETHEREUM].eth, "getTransactionReceipt", side_effect=Exception): + is_successful = ledger_apis.is_tx_settled(ETHEREUM, tx_digest=tx_digest, amount=10) + assert not is_successful + + def test_try_to_instantiate_fetchai_ledger_api(self): + """Test the instantiation of the fetchai ledger api.""" + _try_to_instantiate_fetchai_ledger_api(addr="127.0.0.1", port=80) + from fetchai.ledger.api import LedgerApi + with mock.patch.object(LedgerApi, "__init__", side_effect=Exception): + with pytest.raises(SystemExit): + _try_to_instantiate_fetchai_ledger_api(addr="127.0.0.1", port=80) + + def test__try_to_instantiate_ethereum_ledger_api(self): + """Test the instantiation of the ethereum ledger api.""" + _try_to_instantiate_ethereum_ledger_api(addr="127.0.0.1", port=80) + from web3 import Web3 + with mock.patch.object(Web3, "__init__", side_effect=Exception): + with pytest.raises(SystemExit): + _try_to_instantiate_ethereum_ledger_api(addr="127.0.0.1", port=80) From fd7203a02092c25cff7f7fbf2aa0cfd8a0d3dbca Mon Sep 17 00:00:00 2001 From: Aristotelis Date: Mon, 4 Nov 2019 10:11:15 +0000 Subject: [PATCH 03/52] Removed unused libary --- tests/test_crypto/test_ledger_apis.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/test_crypto/test_ledger_apis.py b/tests/test_crypto/test_ledger_apis.py index 8ca056ba52..6c6f2f94c8 100644 --- a/tests/test_crypto/test_ledger_apis.py +++ b/tests/test_crypto/test_ledger_apis.py @@ -20,7 +20,6 @@ """This module contains the tests for the crypto/helpers module.""" import logging -# from unittest import mock import os from unittest import mock From 52e7410abf84c33e3bdbc56df262919724a6191d Mon Sep 17 00:00:00 2001 From: Aristotelis Date: Mon, 4 Nov 2019 10:51:09 +0000 Subject: [PATCH 04/52] Added ignore to Hexbytes --- tests/test_crypto/test_ledger_apis.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_crypto/test_ledger_apis.py b/tests/test_crypto/test_ledger_apis.py index 6c6f2f94c8..a9fbb8f99e 100644 --- a/tests/test_crypto/test_ledger_apis.py +++ b/tests/test_crypto/test_ledger_apis.py @@ -24,7 +24,7 @@ from unittest import mock import pytest -from hexbytes import HexBytes +from hexbytes import HexBytes # type: ignore from aea.crypto.ethereum import ETHEREUM, EthereumCrypto from aea.crypto.fetchai import FETCHAI, FetchAICrypto From caac6f2ffa0b6d14b88d0ee09a996b54ae68cbf5 Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 13:24:11 +0100 Subject: [PATCH 05/52] add resources as AEA parameter. --- aea/aea.py | 9 ++++++--- aea/registries/base.py | 11 ++++++++--- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/aea/aea.py b/aea/aea.py index bdc8e0bba9..f581a4c886 100644 --- a/aea/aea.py +++ b/aea/aea.py @@ -45,6 +45,7 @@ def __init__(self, name: str, timeout: float = 0.0, debug: bool = False, max_reactions: int = 20, + resources: Optional[Resources] = None, directory: str = '') -> None: """ Instantiate the agent. @@ -56,6 +57,7 @@ def __init__(self, name: str, :param timeout: the time in (fractions of) seconds to time out an agent between act and react :param debug: if True, run the agent in debug mode. :param max_reactions: the processing rate of messages per iteration. + :param resources: the resources of the agent. If None, the directory parameter will be considered. :param directory: the path to the agent's resource directory. | If None, we assume the directory is in the working directory of the interpreter. @@ -81,7 +83,7 @@ def __init__(self, name: str, self.decision_maker.ownership_state, self.decision_maker.preferences, self.decision_maker.is_ready_to_pursuit_goals) - self._resources = None # type: Optional[Resources] + self._resources = resources # type: Optional[Resources] self._filter = None # type: Optional[Filter] @property @@ -112,8 +114,9 @@ def setup(self) -> None: :return: None """ - self._resources = Resources.from_resource_dir(self._directory, self.context) - assert self._resources is not None, "No resources initialized. Error in setup." + if self._resources is None: + self._resources = Resources.from_resource_dir(self._directory, self.context) + assert self._resources is not None, "No resources initialized. Error in setup." self._resources.setup() self._filter = Filter(self.resources, self.decision_maker.message_out_queue) diff --git a/aea/registries/base.py b/aea/registries/base.py index 16b6ee859e..bffba48b23 100644 --- a/aea/registries/base.py +++ b/aea/registries/base.py @@ -478,9 +478,14 @@ def from_resource_dir(cls, directory: str, agent_context: AgentContext) -> Optio :return: None """ resources = Resources() - resources.protocol_registry.populate(directory) - resources.populate_skills(directory, agent_context) - return resources + try: + resources.protocol_registry.populate(directory) + resources.populate_skills(directory, agent_context) + return resources + except Exception as e: + logger.warning(str(e)) + logger.warning("Not loaded") + return None def populate_skills(self, directory: str, agent_context: AgentContext) -> None: """ From 7c01d7d7aa8ad911d66bb94c46fbf8f236b3e069 Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 15:53:50 +0100 Subject: [PATCH 06/52] add tests. --- aea/aea.py | 13 ++++-- tests/conftest.py | 4 ++ tests/test_aea.py | 100 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 113 insertions(+), 4 deletions(-) diff --git a/aea/aea.py b/aea/aea.py index f581a4c886..d367b11861 100644 --- a/aea/aea.py +++ b/aea/aea.py @@ -42,10 +42,10 @@ def __init__(self, name: str, mailbox: MailBox, wallet: Wallet, ledger_apis: LedgerApis, + resources: Optional[Resources] = None, timeout: float = 0.0, debug: bool = False, max_reactions: int = 20, - resources: Optional[Resources] = None, directory: str = '') -> None: """ Instantiate the agent. @@ -54,12 +54,12 @@ def __init__(self, name: str, :param mailbox: the mailbox of the agent. :param wallet: the wallet of the agent. :param ledger_apis: the ledger apis of the agent. - :param timeout: the time in (fractions of) seconds to time out an agent between act and react - :param debug: if True, run the agent in debug mode. - :param max_reactions: the processing rate of messages per iteration. :param resources: the resources of the agent. If None, the directory parameter will be considered. :param directory: the path to the agent's resource directory. | If None, we assume the directory is in the working directory of the interpreter. + :param timeout: the time in (fractions of) seconds to time out an agent between act and react + :param debug: if True, run the agent in debug mode. + :param max_reactions: the processing rate of messages per iteration. :return: None """ @@ -102,6 +102,11 @@ def resources(self) -> Resources: assert self._resources is not None, "No resources initialized. Call setup." return self._resources + @resources.setter + def resources(self, resources: 'Resources'): + """Set resources.""" + self._resources = resources + @property def filter(self) -> Filter: """Get filter.""" diff --git a/tests/conftest.py b/tests/conftest.py index de8a415a15..0e139a7575 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -101,6 +101,10 @@ def send(self, envelope: 'Envelope'): """Send an envelope.""" self.out_queue.put(envelope) + def receive(self, envelope: 'Envelope'): + """Receive an envelope.""" + self.in_queue.put(envelope) + @classmethod def from_config(cls, public_key: str, connection_configuration: ConnectionConfig) -> 'Connection': """Return a connection obj fom a configuration.""" diff --git a/tests/test_aea.py b/tests/test_aea.py index 64eb6ece02..f7ea0a4211 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -22,15 +22,22 @@ from pathlib import Path from threading import Thread +import yaml + +from aea import AEA_DIR from aea.aea import AEA +from aea.configurations.base import ProtocolConfig from aea.connections.local.connection import LocalNode, OEFLocalConnection from aea.crypto.ledger_apis import LedgerApis from aea.crypto.wallet import Wallet from aea.mail.base import MailBox, Envelope +from aea.protocols.base import Protocol from aea.protocols.default.message import DefaultMessage from aea.protocols.default.serialization import DefaultSerializer from aea.protocols.fipa.message import FIPAMessage from aea.protocols.fipa.serialization import FIPASerializer +from aea.registries.base import Resources +from aea.skills.base import Skill from .conftest import CUR_PATH, DummyConnection @@ -180,3 +187,96 @@ def test_handle(): finally: agent.stop() t.join() + + +class TestInitializeAEAProgrammaticallyFromResourcesDir: + """Test that we can initialize the agent by providing the resource object loaded from dir.""" + + @classmethod + def setup_class(cls): + """Set the test up.""" + cls.agent_name = "MyAgent" + cls.private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") + cls.wallet = Wallet({'default': cls.private_key_pem_path}) + cls.ledger_apis = LedgerApis({}) + cls.connection = DummyConnection() + cls.mailbox = MailBox(cls.connection) + + cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis) + cls.resources = Resources.from_resource_dir(os.path.join(CUR_PATH, "data", "dummy_aea"), cls.aea.context) + cls.aea.resources = cls.resources + + cls.expected_message = DefaultMessage(type=DefaultMessage.Type.BYTES, content=b"hello") + cls.connection.receive(Envelope(to="", sender="", protocol_id="default", message=DefaultSerializer().encode(cls.expected_message))) + + cls.t = Thread(target=cls.aea.start) + cls.t.start() + + def test_initialize_aea_programmatically(self): + """Test that we can initialize an AEA programmatically.""" + time.sleep(1.0) + dummy_behaviour = self.aea.resources.behaviour_registry.fetch("dummy")[0] + dummy_task = self.aea.resources.task_registry.fetch("dummy")[0] + dummy_handler = self.aea.resources.handler_registry.fetch("default")[0] + assert dummy_behaviour.nb_act_called > 0 + assert dummy_task.nb_execute_called > 0 + assert len(dummy_handler.handled_messages) == 1 + assert dummy_handler.handled_messages[0] == self.expected_message + + @classmethod + def teardown_class(cls): + """Tear the test down.""" + cls.aea.stop() + cls.t.join() + + +class TestInitializeAEAProgrammaticallyBuildResources: + """Test that we can initialize the agent by building the resource object.""" + + @classmethod + def setup_class(cls): + """Set the test up.""" + cls.agent_name = "MyAgent" + cls.private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") + cls.wallet = Wallet({'default': cls.private_key_pem_path}) + cls.ledger_apis = LedgerApis({}) + cls.connection = DummyConnection() + cls.mailbox = MailBox(cls.connection) + + cls.resources = Resources() + cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis, resources=cls.resources) + + cls.default_protocol_configuration = ProtocolConfig.from_json( + yaml.safe_load(open(Path(AEA_DIR, "protocols", "default", "protocol.yaml")))) + cls.default_protocol = Protocol("default", + DefaultSerializer(), + cls.default_protocol_configuration) + cls.resources.protocol_registry.register(("default", None), cls.default_protocol) + + cls.error_skill = Skill.from_dir(Path(AEA_DIR, "skills", "error"), cls.aea.context) + cls.dummy_skill = Skill.from_dir(Path(CUR_PATH, "data", "dummy_skill"), cls.aea.context) + cls.resources.add_skill(cls.dummy_skill) + cls.resources.add_skill(cls.error_skill) + + cls.expected_message = DefaultMessage(type=DefaultMessage.Type.BYTES, content=b"hello") + cls.connection.receive(Envelope(to="", sender="", protocol_id="default", message=DefaultSerializer().encode(cls.expected_message))) + + cls.t = Thread(target=cls.aea.start) + cls.t.start() + + def test_initialize_aea_programmatically(self): + """Test that we can initialize an AEA programmatically.""" + time.sleep(1.0) + dummy_behaviour = self.aea.resources.behaviour_registry.fetch("dummy")[0] + dummy_task = self.aea.resources.task_registry.fetch("dummy")[0] + dummy_handler = self.aea.resources.handler_registry.fetch("default")[0] + assert dummy_behaviour.nb_act_called > 0 + assert dummy_task.nb_execute_called > 0 + assert len(dummy_handler.handled_messages) == 1 + assert dummy_handler.handled_messages[0] == self.expected_message + + @classmethod + def teardown_class(cls): + """Tear the test down.""" + cls.aea.stop() + cls.t.join() From c524f142b1878ccd541ecd49cb79b40f0e5b690c Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 16:36:19 +0100 Subject: [PATCH 07/52] explicitly select components in tests. --- tests/test_aea.py | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/tests/test_aea.py b/tests/test_aea.py index f7ea0a4211..1933eaa639 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -215,14 +215,19 @@ def setup_class(cls): def test_initialize_aea_programmatically(self): """Test that we can initialize an AEA programmatically.""" time.sleep(1.0) - dummy_behaviour = self.aea.resources.behaviour_registry.fetch("dummy")[0] - dummy_task = self.aea.resources.task_registry.fetch("dummy")[0] - dummy_handler = self.aea.resources.handler_registry.fetch("default")[0] + + dummy_behaviour = next(filter(lambda x: x.__class__.__name__ == "DummyBehaviour", self.aea.resources.behaviour_registry.fetch("dummy")), None) + assert dummy_behaviour is not None assert dummy_behaviour.nb_act_called > 0 + + dummy_task = next(filter(lambda x: x.__class__.__name__ == "DummyTask", self.aea.resources.task_registry.fetch("dummy")), None) + assert dummy_task is not None assert dummy_task.nb_execute_called > 0 + + dummy_handler = next(filter(lambda x: x.__class__.__name__ == "DummyHandler", self.aea.resources.handler_registry.fetch("default")), None) + assert dummy_handler is not None assert len(dummy_handler.handled_messages) == 1 assert dummy_handler.handled_messages[0] == self.expected_message - @classmethod def teardown_class(cls): """Tear the test down.""" @@ -267,11 +272,17 @@ def setup_class(cls): def test_initialize_aea_programmatically(self): """Test that we can initialize an AEA programmatically.""" time.sleep(1.0) - dummy_behaviour = self.aea.resources.behaviour_registry.fetch("dummy")[0] - dummy_task = self.aea.resources.task_registry.fetch("dummy")[0] - dummy_handler = self.aea.resources.handler_registry.fetch("default")[0] + + dummy_behaviour = next(filter(lambda x: x.__class__.__name__ == "DummyBehaviour", self.aea.resources.behaviour_registry.fetch("dummy")), None) + assert dummy_behaviour is not None assert dummy_behaviour.nb_act_called > 0 + + dummy_task = next(filter(lambda x: x.__class__.__name__ == "DummyTask", self.aea.resources.task_registry.fetch("dummy")), None) + assert dummy_task is not None assert dummy_task.nb_execute_called > 0 + + dummy_handler = next(filter(lambda x: x.__class__.__name__ == "DummyHandler", self.aea.resources.handler_registry.fetch("default")), None) + assert dummy_handler is not None assert len(dummy_handler.handled_messages) == 1 assert dummy_handler.handled_messages[0] == self.expected_message From 1eb19b0f0d9d22952df1e695f4bdc12e36dcd41f Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 17:18:04 +0100 Subject: [PATCH 08/52] remove 'directory' parameter in AEA class. --- aea/aea.py | 14 ++++---------- aea/cli/run.py | 3 ++- aea/registries/base.py | 27 ++++++++++++++++----------- tests/test_aea.py | 14 +++++++++----- tests/test_registries.py | 4 ++-- tests/test_skills/test_base.py | 8 ++++---- tests/test_skills/test_error.py | 3 ++- 7 files changed, 39 insertions(+), 34 deletions(-) diff --git a/aea/aea.py b/aea/aea.py index d367b11861..b2c458df6c 100644 --- a/aea/aea.py +++ b/aea/aea.py @@ -42,11 +42,10 @@ def __init__(self, name: str, mailbox: MailBox, wallet: Wallet, ledger_apis: LedgerApis, - resources: Optional[Resources] = None, + resources: Resources = None, timeout: float = 0.0, debug: bool = False, - max_reactions: int = 20, - directory: str = '') -> None: + max_reactions: int = 20) -> None: """ Instantiate the agent. @@ -55,8 +54,6 @@ def __init__(self, name: str, :param wallet: the wallet of the agent. :param ledger_apis: the ledger apis of the agent. :param resources: the resources of the agent. If None, the directory parameter will be considered. - :param directory: the path to the agent's resource directory. - | If None, we assume the directory is in the working directory of the interpreter. :param timeout: the time in (fractions of) seconds to time out an agent between act and react :param debug: if True, run the agent in debug mode. :param max_reactions: the processing rate of messages per iteration. @@ -66,7 +63,6 @@ def __init__(self, name: str, super().__init__(name=name, wallet=wallet, timeout=timeout, debug=debug) self.max_reactions = max_reactions - self._directory = directory if directory else str(Path(".").absolute()) self.mailbox = mailbox self._decision_maker = DecisionMaker(self.name, @@ -119,10 +115,8 @@ def setup(self) -> None: :return: None """ - if self._resources is None: - self._resources = Resources.from_resource_dir(self._directory, self.context) - assert self._resources is not None, "No resources initialized. Error in setup." - self._resources.setup() + self.resources.load(self.context) + self.resources.setup() self._filter = Filter(self.resources, self.decision_maker.message_out_queue) def act(self) -> None: diff --git a/aea/cli/run.py b/aea/cli/run.py index cd19680744..bc6cd7c2fa 100644 --- a/aea/cli/run.py +++ b/aea/cli/run.py @@ -42,6 +42,7 @@ from aea.crypto.ledger_apis import LedgerApis, _try_to_instantiate_fetchai_ledger_api, _try_to_instantiate_ethereum_ledger_api, SUPPORTED_LEDGER_APIS from aea.crypto.wallet import Wallet, DEFAULT, SUPPORTED_CRYPTOS from aea.mail.base import MailBox +from aea.registries.base import Resources def _verify_or_create_private_keys(ctx: Context) -> None: @@ -209,7 +210,7 @@ def run(click_context, connection_name: str, env_file: str, install_deps: bool): click_context.invoke(install) mailbox = MailBox(connection) - agent = AEA(agent_name, mailbox, wallet, ledger_apis, directory=str(Path("."))) + agent = AEA(agent_name, mailbox, wallet, ledger_apis, resources=Resources(str(Path(".")))) try: agent.start() except KeyboardInterrupt: diff --git a/aea/registries/base.py b/aea/registries/base.py index bffba48b23..ce5c5c576d 100644 --- a/aea/registries/base.py +++ b/aea/registries/base.py @@ -28,7 +28,7 @@ from abc import ABC, abstractmethod from queue import Queue from pathlib import Path -from typing import Optional, List, Dict, Any, Tuple, cast +from typing import Optional, List, Dict, Any, Tuple, cast, Union from aea.configurations.base import ProtocolId, SkillId, ProtocolConfig, DEFAULT_PROTOCOL_CONFIG_FILE from aea.configurations.loader import ConfigLoader @@ -458,8 +458,9 @@ def teardown(self) -> None: class Resources(object): """This class implements the resources of an AEA.""" - def __init__(self): + def __init__(self, directory: Optional[Union[str, os.PathLike]] = None): """Instantiate the resources.""" + self._directory = directory if str(Path(directory).absolute()) is not None else str(Path(".").absolute()) self.protocol_registry = ProtocolRegistry() self.handler_registry = HandlerRegistry() self.behaviour_registry = BehaviourRegistry() @@ -468,6 +469,11 @@ def __init__(self): self._registries = [self.protocol_registry, self.handler_registry, self.behaviour_registry, self.task_registry] + @property + def directory(self) -> str: + """Get the directory.""" + return self._directory + @classmethod def from_resource_dir(cls, directory: str, agent_context: AgentContext) -> Optional['Resources']: """ @@ -477,15 +483,14 @@ def from_resource_dir(cls, directory: str, agent_context: AgentContext) -> Optio :param agent_context: the agent's context object :return: None """ - resources = Resources() - try: - resources.protocol_registry.populate(directory) - resources.populate_skills(directory, agent_context) - return resources - except Exception as e: - logger.warning(str(e)) - logger.warning("Not loaded") - return None + resources = Resources(directory) + resources.load(agent_context) + return resources + + def load(self, agent_context: AgentContext) -> None: + """Load all the resources.""" + self.protocol_registry.populate(self.directory) + self.populate_skills(self.directory, agent_context) def populate_skills(self, directory: str, agent_context: AgentContext) -> None: """ diff --git a/tests/test_aea.py b/tests/test_aea.py index 1933eaa639..f468bc0cb9 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -18,6 +18,7 @@ # ------------------------------------------------------------------------------ """This module contains the tests for aea/aea.py.""" import os +import tempfile import time from pathlib import Path from threading import Thread @@ -49,7 +50,7 @@ def test_initialise_AEA(): private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) - my_AEA = AEA("Agent0", mailbox1, wallet, ledger_apis, directory=str(Path(CUR_PATH, "aea"))) + my_AEA = AEA("Agent0", mailbox1, wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "aea")))) assert AEA("Agent0", mailbox1, wallet, ledger_apis), "Agent is not initialised" assert my_AEA.context == my_AEA._context, "Cannot access the Agent's Context" my_AEA.setup() @@ -72,7 +73,7 @@ def test_act(): mailbox, wallet, ledger_apis, - directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) t = Thread(target=agent.start) try: t.start() @@ -109,7 +110,7 @@ def test_react(): mailbox, wallet, ledger_apis, - directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) t = Thread(target=agent.start) try: t.start() @@ -148,7 +149,7 @@ def test_handle(): mailbox, wallet, ledger_apis, - directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) t = Thread(target=agent.start) try: t.start() @@ -228,6 +229,7 @@ def test_initialize_aea_programmatically(self): assert dummy_handler is not None assert len(dummy_handler.handled_messages) == 1 assert dummy_handler.handled_messages[0] == self.expected_message + @classmethod def teardown_class(cls): """Tear the test down.""" @@ -248,7 +250,8 @@ def setup_class(cls): cls.connection = DummyConnection() cls.mailbox = MailBox(cls.connection) - cls.resources = Resources() + cls.temp = tempfile.mkdtemp(prefix="test_aea_resources") + cls.resources = Resources(cls.temp) cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis, resources=cls.resources) cls.default_protocol_configuration = ProtocolConfig.from_json( @@ -291,3 +294,4 @@ def teardown_class(cls): """Tear the test down.""" cls.aea.stop() cls.t.join() + Path(cls.temp).rmdir() diff --git a/tests/test_registries.py b/tests/test_registries.py index b7b71e20c7..07ce029004 100644 --- a/tests/test_registries.py +++ b/tests/test_registries.py @@ -138,7 +138,7 @@ def setup_class(cls): private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) - cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, directory=cls.agent_folder) + cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, resources=Resources(cls.agent_folder)) cls.resources = Resources.from_resource_dir(os.path.join(cls.agent_folder), cls.aea.context) cls.expected_skills = {"dummy", "error"} @@ -298,7 +298,7 @@ def setup_class(cls): private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) - cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, directory=cls.agent_folder) + cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, resources=Resources(cls.agent_folder)) def test_handle_internal_messages(self): """Test that the internal messages are handled.""" diff --git a/tests/test_skills/test_base.py b/tests/test_skills/test_base.py index c69e325230..8078862a11 100644 --- a/tests/test_skills/test_base.py +++ b/tests/test_skills/test_base.py @@ -26,7 +26,7 @@ import aea.registries.base -from aea.aea import AEA +from aea.aea import AEA, Resources from aea.crypto.wallet import Wallet from aea.crypto.ledger_apis import LedgerApis from aea.decision_maker.base import OwnershipState, Preferences @@ -41,7 +41,7 @@ def test_agent_context_ledger_apis(): wallet = Wallet({'default': private_key_pem_path}) mailbox1 = MailBox(DummyConnection()) ledger_apis = LedgerApis({"fetchai": ('alpha.fetch-ai.com', 80)}) - my_aea = AEA("Agent0", mailbox1, wallet, ledger_apis, directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + my_aea = AEA("Agent0", mailbox1, wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) assert set(my_aea.context.ledger_apis.apis.keys()) == {"fetchai"} fetchai_ledger_api_obj = my_aea.context.ledger_apis.apis["fetchai"] @@ -59,7 +59,7 @@ def setup_class(cls): cls.wallet = Wallet({'default': private_key_pem_path}) cls.ledger_apis = LedgerApis({"fetchai": ("alpha.fetch-ai.com", 80)}) cls.mailbox1 = MailBox(DummyConnection()) - cls.my_aea = AEA("Agent0", cls.mailbox1, cls.wallet, cls.ledger_apis, directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + cls.my_aea = AEA("Agent0", cls.mailbox1, cls.wallet, cls.ledger_apis, resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) cls.skill_context = SkillContext(cls.my_aea.context) def test_agent_name(self): @@ -137,7 +137,7 @@ def setup_class(cls): cls.wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) cls.mailbox1 = MailBox(DummyConnection()) - cls.my_aea = AEA("agent_name", cls.mailbox1, cls.wallet, ledger_apis, directory=str(Path(CUR_PATH, "data", "dummy_aea"))) + cls.my_aea = AEA("agent_name", cls.mailbox1, cls.wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "data", "dummy_aea")))) cls.agent_context = cls.my_aea.context def test_missing_handler(self): diff --git a/tests/test_skills/test_error.py b/tests/test_skills/test_error.py index 6bdfe83191..6fcd82dc24 100644 --- a/tests/test_skills/test_error.py +++ b/tests/test_skills/test_error.py @@ -30,6 +30,7 @@ from aea.protocols.fipa.message import FIPAMessage from aea.protocols.fipa.serialization import FIPASerializer from aea.protocols.oef.message import OEFMessage +from aea.registries.base import Resources from aea.skills.base import SkillContext from aea.skills.error.behaviours import ErrorBehaviour from aea.skills.error.handlers import ErrorHandler @@ -53,7 +54,7 @@ def setup_class(cls): cls.connection = DummyConnection() cls.mailbox1 = MailBox(cls.connection) cls.my_aea = AEA(cls.agent_name, cls.mailbox1, cls.wallet, cls.ledger_apis, timeout=2.0, - directory=str(Path(CUR_PATH, "data/dummy_aea"))) + resources=Resources(str(Path(CUR_PATH, "data/dummy_aea")))) cls.skill_context = SkillContext(cls.my_aea._context) cls.my_error_handler = ErrorHandler(skill_context=cls.skill_context) From 7ee83a30e25dc2de422899cd23b8c1ea8bd440b1 Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 17:53:45 +0100 Subject: [PATCH 09/52] address PR comments. - remove obsolete method 'Resources.from_dir' - remove obsolete asserts, e.g. checking whether 'resources' is None (not it is a mandatory argument). - instantiate Filter in the AEA constructor. --- aea/aea.py | 13 ++++--------- aea/registries/base.py | 18 ++---------------- tests/test_aea.py | 7 +++---- tests/test_registries.py | 5 +++-- 4 files changed, 12 insertions(+), 31 deletions(-) diff --git a/aea/aea.py b/aea/aea.py index b2c458df6c..9f4395635b 100644 --- a/aea/aea.py +++ b/aea/aea.py @@ -19,7 +19,6 @@ """This module contains the implementation of an Autonomous Economic Agent.""" import logging -from pathlib import Path from typing import Optional, cast from aea.agent import Agent @@ -31,7 +30,6 @@ from aea.registries.base import Filter, Resources from aea.skills.error.handlers import ErrorHandler - logger = logging.getLogger(__name__) @@ -42,7 +40,7 @@ def __init__(self, name: str, mailbox: MailBox, wallet: Wallet, ledger_apis: LedgerApis, - resources: Resources = None, + resources: Resources, timeout: float = 0.0, debug: bool = False, max_reactions: int = 20) -> None: @@ -53,7 +51,7 @@ def __init__(self, name: str, :param mailbox: the mailbox of the agent. :param wallet: the wallet of the agent. :param ledger_apis: the ledger apis of the agent. - :param resources: the resources of the agent. If None, the directory parameter will be considered. + :param resources: the resources of the agent. :param timeout: the time in (fractions of) seconds to time out an agent between act and react :param debug: if True, run the agent in debug mode. :param max_reactions: the processing rate of messages per iteration. @@ -79,8 +77,8 @@ def __init__(self, name: str, self.decision_maker.ownership_state, self.decision_maker.preferences, self.decision_maker.is_ready_to_pursuit_goals) - self._resources = resources # type: Optional[Resources] - self._filter = None # type: Optional[Filter] + self._resources = resources + self._filter = Filter(self.resources, self.decision_maker.message_out_queue) @property def decision_maker(self) -> DecisionMaker: @@ -95,7 +93,6 @@ def context(self) -> AgentContext: @property def resources(self) -> Resources: """Get resources.""" - assert self._resources is not None, "No resources initialized. Call setup." return self._resources @resources.setter @@ -106,7 +103,6 @@ def resources(self, resources: 'Resources'): @property def filter(self) -> Filter: """Get filter.""" - assert self._filter is not None, "No filter initialized. Call setup." return self._filter def setup(self) -> None: @@ -117,7 +113,6 @@ def setup(self) -> None: """ self.resources.load(self.context) self.resources.setup() - self._filter = Filter(self.resources, self.decision_maker.message_out_queue) def act(self) -> None: """ diff --git a/aea/registries/base.py b/aea/registries/base.py index ce5c5c576d..8b3d28cca8 100644 --- a/aea/registries/base.py +++ b/aea/registries/base.py @@ -460,7 +460,7 @@ class Resources(object): def __init__(self, directory: Optional[Union[str, os.PathLike]] = None): """Instantiate the resources.""" - self._directory = directory if str(Path(directory).absolute()) is not None else str(Path(".").absolute()) + self._directory = str(Path(directory).absolute()) if directory is not None else str(Path(".").absolute()) self.protocol_registry = ProtocolRegistry() self.handler_registry = HandlerRegistry() self.behaviour_registry = BehaviourRegistry() @@ -474,19 +474,6 @@ def directory(self) -> str: """Get the directory.""" return self._directory - @classmethod - def from_resource_dir(cls, directory: str, agent_context: AgentContext) -> Optional['Resources']: - """ - Parse the resource directory. - - :param directory: the agent's resources directory. - :param agent_context: the agent's context object - :return: None - """ - resources = Resources(directory) - resources.load(agent_context) - return resources - def load(self, agent_context: AgentContext) -> None: """Load all the resources.""" self.protocol_registry.populate(self.directory) @@ -577,7 +564,6 @@ def __init__(self, resources: Resources, decision_maker_out_queue: Queue): @property def resources(self) -> Resources: """Get resources.""" - assert self._resources is not None, "No resources initialized. Call setup." return self._resources @property @@ -585,7 +571,7 @@ def decision_maker_out_queue(self) -> Queue: """Get decision maker (out) queue.""" return self._decision_maker_out_queue - def get_active_handlers(self, protocol_id: str) -> List[Handler]: + def get_active_handlers(self, protocol_id: str) -> Optional[List[Handler]]: """ Get active handlers. diff --git a/tests/test_aea.py b/tests/test_aea.py index f468bc0cb9..e6009f5472 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -51,7 +51,6 @@ def test_initialise_AEA(): wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) my_AEA = AEA("Agent0", mailbox1, wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "aea")))) - assert AEA("Agent0", mailbox1, wallet, ledger_apis), "Agent is not initialised" assert my_AEA.context == my_AEA._context, "Cannot access the Agent's Context" my_AEA.setup() assert my_AEA.resources is not None,\ @@ -203,9 +202,9 @@ def setup_class(cls): cls.connection = DummyConnection() cls.mailbox = MailBox(cls.connection) - cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis) - cls.resources = Resources.from_resource_dir(os.path.join(CUR_PATH, "data", "dummy_aea"), cls.aea.context) - cls.aea.resources = cls.resources + cls.resources = Resources(os.path.join(CUR_PATH, "data", "dummy_aea")) + cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis, cls.resources) + cls.resources.load(cls.aea.context) cls.expected_message = DefaultMessage(type=DefaultMessage.Type.BYTES, content=b"hello") cls.connection.receive(Envelope(to="", sender="", protocol_id="default", message=DefaultSerializer().encode(cls.expected_message))) diff --git a/tests/test_registries.py b/tests/test_registries.py index 07ce029004..875a4544e3 100644 --- a/tests/test_registries.py +++ b/tests/test_registries.py @@ -138,8 +138,9 @@ def setup_class(cls): private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") wallet = Wallet({'default': private_key_pem_path}) ledger_apis = LedgerApis({}) - cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, resources=Resources(cls.agent_folder)) - cls.resources = Resources.from_resource_dir(os.path.join(cls.agent_folder), cls.aea.context) + cls.resources = Resources(os.path.join(cls.agent_folder)) + cls.aea = AEA(cls.agent_name, mailbox, wallet, ledger_apis, resources=cls.resources) + cls.resources.load(cls.aea.context) cls.expected_skills = {"dummy", "error"} From 09d7fad082f2870b0913fb2751c66a586ab07d60 Mon Sep 17 00:00:00 2001 From: Diarmid Campbell Date: Mon, 4 Nov 2019 17:35:24 +0000 Subject: [PATCH 10/52] started on tests for gui --- .gitignore | 3 + aea/cli_gui/__init__.py | 17 +++- tests/test_gui/__init__.py | 20 ++++ tests/test_gui/test_agents.py | 110 ++++++++++++++++++++++ tests/test_gui/test_base.py | 152 +++++++++++++++++++++++++++++++ tests/test_gui/test_items.py | 42 +++++++++ tests/test_gui/test_run_agent.py | 72 +++++++++++++++ 7 files changed, 414 insertions(+), 2 deletions(-) create mode 100644 tests/test_gui/__init__.py create mode 100644 tests/test_gui/test_agents.py create mode 100644 tests/test_gui/test_base.py create mode 100644 tests/test_gui/test_items.py create mode 100644 tests/test_gui/test_run_agent.py diff --git a/.gitignore b/.gitignore index 1a937174a7..73328df4d3 100644 --- a/.gitignore +++ b/.gitignore @@ -111,4 +111,7 @@ venv.bak/ data/* temp_private_key.pem .idea/ +packages +aea_scripts + diff --git a/aea/cli_gui/__init__.py b/aea/cli_gui/__init__.py index 2fd2b2a791..82cf539676 100644 --- a/aea/cli_gui/__init__.py +++ b/aea/cli_gui/__init__.py @@ -379,9 +379,8 @@ def _kill_running_oef_nodes(): subprocess.call(['docker', 'kill', oef_node_name]) -def run(): +def create_app(): """Run the flask server.""" - _kill_running_oef_nodes() CUR_DIR = os.path.abspath(os.path.dirname(__file__)) app = connexion.FlaskApp(__name__, specification_dir=CUR_DIR) flask.app.oef_process = None @@ -410,8 +409,22 @@ def favicon(): return flask.send_from_directory( os.path.join(app.root_path, 'static'), 'favicon.ico', mimetype='image/vnd.microsoft.icon') + return app + +def run(): + + _kill_running_oef_nodes() + app = create_app() app.run(host='127.0.0.1', port=8080, debug=False) + return app + +def run_test(): + + #_kill_running_oef_nodes() + app = create_app() + return app.app.test_client() + # If we're running in stand alone mode, run the application if __name__ == '__main__': diff --git a/tests/test_gui/__init__.py b/tests/test_gui/__init__.py new file mode 100644 index 0000000000..4bbd0b1527 --- /dev/null +++ b/tests/test_gui/__init__.py @@ -0,0 +1,20 @@ +# -*- coding: utf-8 -*- +# ------------------------------------------------------------------------------ +# +# Copyright 2018-2019 Fetch.AI Limited +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ------------------------------------------------------------------------------ + +"""The tests for the cli gui.""" diff --git a/tests/test_gui/test_agents.py b/tests/test_gui/test_agents.py new file mode 100644 index 0000000000..66adea61ef --- /dev/null +++ b/tests/test_gui/test_agents.py @@ -0,0 +1,110 @@ +# -*- coding: utf-8 -*- +# ------------------------------------------------------------------------------ +# +# Copyright 2018-2019 Fetch.AI Limited +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ------------------------------------------------------------------------------ + +"""This test module contains the tests for the `aea create` sub-command.""" +import json + +from .test_base import TestBase + + +class TestHomePageExits(TestBase): + """Test that the gui home page exits and has the correct title.""" + + def test_home_page_exits(self): + """Test that the home-page exits.""" + # sends HTTP GET request to the application + # on the specified path + result = self.app.get('/') + + # assert the status code of the response + assert result.status_code == 200 + assert "Fetch.AI AEA CLI REST API" in str(result.data) + + +class TestCreateAndList(TestBase): + """Test that we can create an agent and get list of agents.""" + + def test_agents_create_and_list(self): + agent_name = "test_agent" + + # Make sure there are no agents in the directory + response_list = self.app.get( + 'api/agent', + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert len(data) == 0 + assert response_list.status_code == 200 + + # Make an agent + assert self.create_agent(agent_name).status_code == 201 + + # Ensure there is now one agent + response_list = self.app.get( + 'api/agent', + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) == 1 + assert data[0]['id'] == agent_name + + # Delete the agent + response_delete = self.app.delete( + 'api/agent/' + agent_name, + data=None, + content_type='application/json', + ) + assert response_delete.status_code == 200 + + # Ensure there are now agents + response_list = self.app.get( + 'api/agent', + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) == 0 + + + + +class TestDuplicateAgentError(TestBase): + """Test that we can create an agent and get list of agents.""" + + def test_duplicate_agent_error(self): + # Make sure there are no agents in the directory + response_list = self.app.get( + 'api/agent', + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert len(data) == 0 + assert response_list.status_code == 200 + + # Make an agent + assert self.create_agent("test_agent").status_code == 201 + + # Attempt tp make the same agent again + assert self.create_agent("test_agent").status_code == 400 + diff --git a/tests/test_gui/test_base.py b/tests/test_gui/test_base.py new file mode 100644 index 0000000000..3c0afaa15a --- /dev/null +++ b/tests/test_gui/test_base.py @@ -0,0 +1,152 @@ +# -*- coding: utf-8 -*- +# ------------------------------------------------------------------------------ +# +# Copyright 2018-2019 Fetch.AI Limited +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ------------------------------------------------------------------------------ + +"""This test module contains the tests for the `aea create` sub-command.""" +import os +import shutil +import json + +import tempfile + +import aea.cli_gui + + +class TestBase: + """Base class for testing gui entry points.""" + + @classmethod + def setup_class(cls): + """Set the test up.""" + cls.temp_dir = tempfile.mkdtemp() + cls.cwd = os.getcwd() + os.chdir(cls.temp_dir) + + cls.app = aea.cli_gui.run_test() + cls.app.debug = True + cls.app.testing = True + + def create_agent(self, name): + return self.app.post( + 'api/agent', + content_type='application/json', + data=json.dumps(name)) + + @classmethod + def teardown_class(cls): + """Teardowm the test.""" + os.chdir(cls.cwd) + try: + shutil.rmtree(cls.temp_dir) + except (OSError, IOError): + pass + + def _test_create_and_list(self, item_type, new_item_name): + agent_name = "test_agent" + + # Make an agent + assert self.create_agent(agent_name).status_code == 201 + + # Get list of items on this agent + response_list = self.app.get( + 'api/agent/' + agent_name + "/" + item_type, + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + prev_count = len(data) + + # Get list of items from the package + response_list = self.app.get( + 'api/' + item_type, + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) > 0 + + # Add a item + response_create = self.app.post( + 'api/agent/' + agent_name + "/" + item_type, + content_type='application/json', + data=json.dumps(new_item_name) + ) + assert response_create.status_code == 201 + + # Get list of items + response_list = self.app.get( + 'api/agent/' + agent_name + "/" + item_type, + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) == prev_count + 1 + new_connection_exists = False + for element in data: + assert element['id'] != "" + assert element['description'] != "" + if element['id'] == new_item_name: + new_connection_exists = True + assert new_connection_exists + + # Remove the item + response_delete = self.app.delete( + 'api/agent/' + agent_name + "/" + item_type + "/" + new_item_name, + data=None, + content_type='application/json', + ) + assert response_delete.status_code == 201 + + # Get list of items + response_list = self.app.get( + 'api/agent/' + agent_name + "/" + item_type, + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) == prev_count + + # Scaffold item + scaffold_item_name = "scaffold_item" + response_create = self.app.post( + 'api/agent/' + agent_name + "/" + item_type + "/scaffold", + content_type='application/json', + data=json.dumps(scaffold_item_name) + ) + assert response_create.status_code == 201 + + # Get list of items + response_list = self.app.get( + 'api/agent/' + agent_name + "/" + item_type, + data=None, + content_type='application/json', + ) + data = json.loads(response_list.get_data(as_text=True)) + assert response_list.status_code == 200 + assert len(data) == prev_count + 1 + new_connection_exists = False + for element in data: + assert element['id'] != "" + assert element['description'] != "" + if element['id'] == scaffold_item_name: + new_connection_exists = True + assert new_connection_exists diff --git a/tests/test_gui/test_items.py b/tests/test_gui/test_items.py new file mode 100644 index 0000000000..d49e1bb1ce --- /dev/null +++ b/tests/test_gui/test_items.py @@ -0,0 +1,42 @@ +# -*- coding: utf-8 -*- +# ------------------------------------------------------------------------------ +# +# Copyright 2018-2019 Fetch.AI Limited +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ------------------------------------------------------------------------------ + +"""This test module contains the tests for the `aea create` sub-command.""" +from .test_base import TestBase + + +class TestCreateConnectionAndList(TestBase): + """Test that the gui home page exits and has the correct title.""" + + def test_create_and_list_connections(self): + self._test_create_and_list("connection", "local") + +class TestCreateProtocolAndList(TestBase): + """Test that the gui home page exits and has the correct title.""" + + def test_create_and_list_protocols(self): + self._test_create_and_list("protocol", "fipa") + + +class TestCreateSkillAndList(TestBase): + """Test that the gui home page exits and has the correct title.""" + + def test_create_and_list_skills(self): + self._test_create_and_list("skill", "scaffold") + diff --git a/tests/test_gui/test_run_agent.py b/tests/test_gui/test_run_agent.py new file mode 100644 index 0000000000..e5997cae58 --- /dev/null +++ b/tests/test_gui/test_run_agent.py @@ -0,0 +1,72 @@ +# -*- coding: utf-8 -*- +# ------------------------------------------------------------------------------ +# +# Copyright 2018-2019 Fetch.AI Limited +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ------------------------------------------------------------------------------ + +"""This test module contains the tests for the `aea create` sub-command.""" +import json +import time +from .test_base import TestBase + + +class TestRunAgent(TestBase): + """Test for running and agent, reading TTY and errors.""" + + def test_create_and_run_agent(self): + agent_name = "test_agent" + + # Make an agent + assert self.create_agent(agent_name).status_code == 201 + + # Add the local connection + response_add = self.app.post( + 'api/agent/' + agent_name + "/connection", + content_type='application/json', + data=json.dumps("local") + ) + assert response_add.status_code == 201 + + # run the agent + response_run = self.app.post( + 'api/agent/' + agent_name + "/run", + data=None, + content_type='application/json', + ) + assert response_run.status_code == 201 + + time.sleep(2) + + response_status = self.app.get( + 'api/agent/' + agent_name + "/run", + data=None, + content_type='application/json', + ) + assert response_status.status_code == 200 + data = json.loads(response_status.get_data(as_text=True)) + + assert data["error"] == "" + assert "RUNNING" in data["status"] + assert "do connected finished" in data["tty"] + + response_stop = self.app.delete( + 'api/agent/' + agent_name + "/run", + data=None, + content_type='application/json', + ) + assert response_stop.status_code == 200 + + From 8996cdb18d7e8d118008387534944ec173cc036d Mon Sep 17 00:00:00 2001 From: MarcoFavorito Date: Mon, 4 Nov 2019 18:51:22 +0100 Subject: [PATCH 11/52] remove useless 'resource.load' instruction in test_aea --- tests/test_aea.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/test_aea.py b/tests/test_aea.py index e6009f5472..bb82443dd7 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -204,7 +204,6 @@ def setup_class(cls): cls.resources = Resources(os.path.join(CUR_PATH, "data", "dummy_aea")) cls.aea = AEA(cls.agent_name, cls.mailbox, cls.wallet, cls.ledger_apis, cls.resources) - cls.resources.load(cls.aea.context) cls.expected_message = DefaultMessage(type=DefaultMessage.Type.BYTES, content=b"hello") cls.connection.receive(Envelope(to="", sender="", protocol_id="default", message=DefaultSerializer().encode(cls.expected_message))) From ac36032155b4a083e88fa37f82306523bc15b391 Mon Sep 17 00:00:00 2001 From: Aristotelis Date: Tue, 5 Nov 2019 11:02:43 +0000 Subject: [PATCH 12/52] Changes based on the PR comments --- aea/crypto/ledger_apis.py | 7 +++---- setup.cfg | 3 +++ tests/test_crypto/test_ledger_apis.py | 23 ++++++++++++++++------- 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/aea/crypto/ledger_apis.py b/aea/crypto/ledger_apis.py index 1373530d31..afe8ca7a8b 100644 --- a/aea/crypto/ledger_apis.py +++ b/aea/crypto/ledger_apis.py @@ -113,7 +113,7 @@ def token_balance(self, identifier: str, address: str) -> int: logger.warning("An error occurred while attempting to get the current balance.") balance = 0 else: # pragma: no cover - balance = 0 + raise Exception("Ledger id is not known") return balance def transfer(self, identifier: str, crypto_object: Crypto, destination_address: str, amount: int, tx_fee: int) -> Optional[str]: @@ -154,8 +154,7 @@ def transfer(self, identifier: str, crypto_object: Crypto, destination_address: } signed = api.eth.account.signTransaction(transaction, crypto_object.entity.privateKey) hex_value = api.eth.sendRawTransaction(signed.rawTransaction) - print("TX Hash: ", hex_value.hex()) - print("connect_to https://ropsten.etherscan.io/tx/{}".format(hex_value.hex())) + logger.info("TX Hash: {}".format(str(hex_value.hex()))) while True: try: api.eth.getTransactionReceipt(hex_value) @@ -168,7 +167,7 @@ def transfer(self, identifier: str, crypto_object: Crypto, destination_address: return tx_digest else: # pragma: no cover - tx_digest = None + raise Exception("Ledger id is not known") return tx_digest def is_tx_settled(self, identifier: str, tx_digest: str, amount: int) -> bool: diff --git a/setup.cfg b/setup.cfg index d83be73407..532d2c8d3b 100644 --- a/setup.cfg +++ b/setup.cfg @@ -70,6 +70,9 @@ ignore_missing_imports = True [mypy-eth_keys.*] ignore_missing_imports = True +[mypy-hexbytes] +ignore_missing_imports = True + # Per-module options for examples dir: [mypy-numpy] diff --git a/tests/test_crypto/test_ledger_apis.py b/tests/test_crypto/test_ledger_apis.py index a9fbb8f99e..988668e2fe 100644 --- a/tests/test_crypto/test_ledger_apis.py +++ b/tests/test_crypto/test_ledger_apis.py @@ -24,7 +24,7 @@ from unittest import mock import pytest -from hexbytes import HexBytes # type: ignore +from hexbytes import HexBytes from aea.crypto.ethereum import ETHEREUM, EthereumCrypto from aea.crypto.fetchai import FETCHAI, FetchAICrypto @@ -76,10 +76,8 @@ def test_token_balance(self): balance = ledger_apis.token_balance("UNKNOWN", fet_address) assert balance == 0, "Unknown identifier so it will return 0" - def test_transfer(self): - """Test the transfer function for the supported tokens.""" - private_key_path = os.path.join(CUR_PATH, "data", "eth_private_key.txt") - eth_obj = EthereumCrypto(private_key_path=private_key_path) + def test_transfer_fetchai(self): + """Test the transfer function for fetchai token.""" private_key_path = os.path.join(CUR_PATH, 'data', "fet_private_key.txt") fet_obj = FetchAICrypto(private_key_path=private_key_path) ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, @@ -90,6 +88,12 @@ def test_transfer(self): tx_digest = ledger_apis.transfer(FETCHAI, fet_obj, fet_address, amount=10, tx_fee=10) assert tx_digest is not None + def test_transfer_ethereum(self): + """Test the transfer function for ethereum token.""" + private_key_path = os.path.join(CUR_PATH, "data", "eth_private_key.txt") + eth_obj = EthereumCrypto(private_key_path=private_key_path) + ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, + FETCHAI: DEFAULT_FETCHAI_CONFIG}) with mock.patch.object(ledger_apis.apis.get(ETHEREUM).eth, 'getTransactionCount', return_value=5): transaction = { 'nonce': ledger_apis.apis.get(ETHEREUM).eth.getTransactionCount(), @@ -112,8 +116,8 @@ def test_transfer(self): tx_digest = ledger_apis.transfer(ETHEREUM, eth_obj, eth_address, amount=10, tx_fee=200000) assert tx_digest is not None - def test_is_tx_settled(self): - """Test if the transaction is settled.""" + def test_is_tx_settled_fetchai(self): + """Test if the transaction is settled for fetchai.""" ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, FETCHAI: DEFAULT_FETCHAI_CONFIG}) tx_digest = "97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35" @@ -128,6 +132,11 @@ def test_is_tx_settled(self): is_successful = ledger_apis.is_tx_settled(FETCHAI, tx_digest=tx_digest, amount=10) assert not is_successful + def test_is_tx_settled_ethereum(self): + """Test if the transaction is settled for eth.""" + ledger_apis = LedgerApis({ETHEREUM: DEFAULT_ETHEREUM_CONFIG, + FETCHAI: DEFAULT_FETCHAI_CONFIG}) + tx_digest = "97fcacaaf94b62318c4e4bbf53fd2608c15062f17a6d1bffee0ba7af9b710e35" result = HexBytes( '0xf85f808082c35094d898d5e829717c72e7438bad593076686d7d164a80801ba005c2e99ecee98a12fbf28ab9577423f42e9e88f2291b3acc8228de743884c874a077d6bc77a47ad41ec85c96aac2ad27f05a039c4787fca8a1e5ee2d8c7ec1bb6a') with mock.patch.object(ledger_apis.apis[ETHEREUM].eth, "getTransactionReceipt", return_value=result): From de8c5a92718979a288b47f2680a2f717fe8692eb Mon Sep 17 00:00:00 2001 From: Katharine Murphy Date: Tue, 5 Nov 2019 12:57:42 +0000 Subject: [PATCH 13/52] Draft of programming an AEA, proof read overall --- docs/app-areas.md | 6 +- docs/car-park.md | 4 +- docs/cli-gui.md | 45 +++++---- docs/core-components.md | 2 +- docs/design-principles.md | 4 +- docs/diagram.md | 11 ++- docs/fipa-skill.md | 2 +- docs/gym-skill.md | 4 +- docs/hacking-an-agent.md | 202 +++++++++++++++++++++++++++++++++++++- docs/index.md | 6 +- docs/logging.md | 28 +++--- docs/oef-ledger.md | 4 +- docs/protocol.md | 4 +- docs/quickstart.md | 9 +- docs/skill-guide.md | 25 ++--- docs/steps.md | 4 +- docs/tac.md | 2 +- docs/trust.md | 11 ++- docs/weather-skills.md | 63 ++++++------ 19 files changed, 319 insertions(+), 117 deletions(-) diff --git a/docs/app-areas.md b/docs/app-areas.md index ee9ba036f1..07e39b36b0 100644 --- a/docs/app-areas.md +++ b/docs/app-areas.md @@ -8,13 +8,13 @@ There are five general application areas for Fetch.ai AEAs. * **Digital data sales agents**: pure software agents that attach to data sources and sell it via the open economic framework. * **Representative**: an agent which represents an individual's activities on the Fetch.ai network. -## Multi-agent system vs agent-based modelling +## Multi-agent system versus agent-based modelling -The Fetch.ai multi agent system is a real world multi-agent technological system and, although there is some overlap, it is not the same as agent based modelling where the goal is scientific behaviourial observation rather than practical economic gain. +The Fetch.ai multi agent system is a real world multi-agent technological system and, although there is some overlap, it is not the same as agent based modelling where the goal is scientific behavioural observation rather than practical economic gain. Moreover, there is no restriction to *multi*. Single-agent applications are also possible. -
\ No newline at end of file +
diff --git a/docs/car-park.md b/docs/car-park.md index aa40a2d7d0..b5698f34bc 100644 --- a/docs/car-park.md +++ b/docs/car-park.md @@ -70,7 +70,7 @@ Set up your Pi to physically view the car park. We'll leave that to you. ### Potential issues -1. Make sure you use the first port, `HDMI 0` on the Pi for the initial setup monitor. +1. Make sure you use the first port, `HDMI 0` on the Pi for the initial set up monitor. 2. If you install the Pi with a used SD card, you will need to reformat the card with NOOBS: https://www.raspberrypi.org/downloads/noobs/. 3. Fix the screen resolution issues by editing the configuration. @@ -196,7 +196,7 @@ Add the Pi's ip address. You will be prompted for the Raspberry Pi password. The ### Get your remote desktop ip -Follow the instructiona to get your remote ip address. +Follow the instructions to get your remote ip address. ### Connect to the Raspberry Pi diff --git a/docs/cli-gui.md b/docs/cli-gui.md index 5194d65546..9a9c316008 100644 --- a/docs/cli-gui.md +++ b/docs/cli-gui.md @@ -1,49 +1,52 @@ -The AEA Command Line Interface (CLI) can also be invoked from a Graphical User Interface (GUI) which can be access from a web browser. +You can invoke the AEA Command Line Interface (CLI) from a Graphical User Interface (GUI) accessed from a web browser. -These instructions will take you through building an agent, starting an OEF Node and running the agent - all from the GUI. Once you can do this, the other operations should be fairly self-explanatory. +These instructions will take you through building an agent, starting an OEF Node, and running the agent - all from the GUI. ## Preliminaries -Ensure you have the framework installed and the CLI is working by following the [quick-start guide](quickstart.md). +Follow the Preliminaries and Installation instructions here. -Please install the extra dependencies for the CLI GUI: - ```python - pip install aea[cli_gui] - ``` +Install the extra dependencies for the CLI GUI. + +```python +pip install aea[cli_gui] +``` ## Starting the GUI -Go to your working folder, where you want to create new agents. If you followed the quick start guide, this will be in the my_aea directory. Start the local web-server: +Go to the directory in which you will create new agents. If you followed the quick start guide, this will be `my_aea`. + +Start the local web-server. ``` bash aea gui ``` - Open this page in a browser: [http://127.0.0.1:8080](http://127.0.0.1:8080) -You should see the following page displayed: +You should see the following page.
![new gui screen](assets/cli_gui01_clean.png)
-On the left-hand side we can see any agents you have created and any protocols, connections and skills they have. Initially this will be empty - or if you have created an agent using the CLI in the quick-start guide and not deleted it then that should be listed. +On the left-hand side we can see any agents you have created and the protocols, connections, and skills they have. Initially this will be empty - unless you have run the quick start previously and not deleted those agents. -On the right-hand side is the Registry which shows all the protocols, connections and skills which are available to you to construct your agents out of. +On the right-hand side is the Registry which shows all the protocols, connections, and skills which are available to your agent. -To create a new agent and run it, follow these steps: +To create a new agent and run it, follow these steps.
![gui sequence](assets/cli_gui02_sequence.png)
1. In the [Create Agent id] box on the left. type the name of your agent - e.g. my_new_agent. This should now be the currently selected agent - but you can click on its name in the list to make sure. -2. Click the [Create Agent] button - the newly created agent should appear in the [Local Agents] table -3. On the right hand side, find the Echo skill and click on it - this will select it -4. Click on the [Add skill] button - which should actually now say "Add echo skill to my_new_agent agent" -5. Start an OEF Node, by clicking on the [Start OEF Node] button. Wait for the text saying "A thing of beauty is a joy forever..." to appear. This shows that the node has started successfully +2. Click the [Create Agent] button - the newly created agent should appear in the [Local Agents] table. +3. On the right hand side, find the Echo skill and click on it - this will select it. +4. Click on the [Add skill] button - which should now say "Add echo skill to my_new_agent agent". +5. Start an OEF Node by clicking on the [Start OEF Node] button. Wait for the text saying "A thing of beauty is a joy forever..." to appear. When you see that, the node has started successfully. + +
![start node](assets/cli_gui03_oef_node.png)
-
![start node](assets/cli_gui03_oef_node.png)
+6. Start the agent running by clicking on the [start agent] button. You should see the output from the echo agent appearing on the screen. -6. Start the agent running, by clicking on the [start agent] button - you should see the output from the echo agent appearing on the screen +
![start agent](assets/cli_gui04_new_agent.png)
-
![start agent](assets/cli_gui04_new_agent.png)
+This is how your whole page should look if you followed the instructions correctly. -This is how your whole page should look if you followed the instructions correctly
![whole screen running](assets/cli_gui05_full_running_agent.png)
diff --git a/docs/core-components.md b/docs/core-components.md index 1ab263f1a5..6ed4bc56bb 100644 --- a/docs/core-components.md +++ b/docs/core-components.md @@ -56,7 +56,7 @@ A skill encapsulates implementations of the abstract base classes `Handler`, `Be * `Handler`: each skill has none, one or more `Handler` objects, each responsible for the registered messaging protocol. Handlers implement agents' reactive behaviour. If the agent understands the protocol referenced in a received `Envelope`, the `Handler` reacts appropriately to the corresponding message. Each `Handler` is responsible for only one protocol. A `Handler` is also capable of dealing with internal messages. * `Behaviour`: none, one or more `Behaviours` encapsulate actions that cause interactions with other agents initiated by the agent. Behaviours implement agents' proactiveness. -* `Task`: none, one or more Tasks encapsulate background work internal to the agent. +* `Task`: none, one or more `Tasks` encapsulate background work internal to the agent. ## Agent diff --git a/docs/design-principles.md b/docs/design-principles.md index 2d10192b01..dd3ac85c5c 100644 --- a/docs/design-principles.md +++ b/docs/design-principles.md @@ -1,6 +1,6 @@ -We have collated 8 principles which guide AEA framework development: +Eight principles guide AEA framework development: -* **Accessibility**: easy of use. +* **Accessibility**: ease of use. * **Modularity**: encourages module creation and sharing and reuse. * **Openness**: easily extensible with third party libraries. * **Conciseness**: conceptually simple. diff --git a/docs/diagram.md b/docs/diagram.md index 837fd3b034..e50e88e6c7 100644 --- a/docs/diagram.md +++ b/docs/diagram.md @@ -2,15 +2,18 @@ Work in progress. -The framework can be divided into two parts: a) a **core** part that is developed by the Fetch.ai team as well as external contributors, -and b) **extensions** (also known as **packages**) developed by any developer. This allows for a modular and scalable framework. -Currently, the framework supports three types of packages which can be added to the core part as modules: +The framework has two distinctive parts. + +* A **core** that is developed by the Fetch.ai team as well as external contributors. +* **extensions** (also known as **packages**) developed by any developer which promotes a modular and scalable framework. + +Currently, the framework supports three types of packages which can be added to the core as modules. * Skills * Protocols * Connections -The following figure illustrates the framework's architecture: +The following figure illustrates the framework's architecture.
![The AEA Framework Architecture](assets/framework-architecture.png)
diff --git a/docs/fipa-skill.md b/docs/fipa-skill.md index f0c1e50008..bce93c43ba 100644 --- a/docs/fipa-skill.md +++ b/docs/fipa-skill.md @@ -80,7 +80,7 @@ This class deals with representing potential transactions between agents. FIPA negotiation skill is not fully developed. -Follow the Preliminaries and Installation instructions here. +Follow the Preliminaries and Installation instructions here. Then, download the examples and packages directory. diff --git a/docs/gym-skill.md b/docs/gym-skill.md index 8212af2e19..adfe6a5f32 100644 --- a/docs/gym-skill.md +++ b/docs/gym-skill.md @@ -1,4 +1,4 @@ -The AEA gym skill demonstrates how a custom Reinforcement Learning agent, that uses openai's gym library, may be embedded into an Autonomous Economic Agent. +The AEA gym skill demonstrates how a custom Reinforcement Learning agent, that uses OpenAI's gym library, may be embedded into an Autonomous Economic Agent. ## Demo instructions @@ -76,4 +76,4 @@ aea delete my_gym_agent ``` -
\ No newline at end of file +
diff --git a/docs/hacking-an-agent.md b/docs/hacking-an-agent.md index 588cafe237..f10b451bc1 100644 --- a/docs/hacking-an-agent.md +++ b/docs/hacking-an-agent.md @@ -1,6 +1,200 @@ -!!! Warning - Not recommended. +## Preliminaries -!!! Note - Coming soon. +These instructions detail the Python code you need for running an AEA outside the `cli` tool. +!!! Note + You have already coded up your agent. See the build your own skill guide for a reminder. + + +## Imports + +First, import the necessary common Python libraries and classes. + +``` python +import os +import tempfile +import time +from pathlib import Path +from threading import Thread +import yaml +``` + +Then, import the application specific libraries. + +``` python +from aea import AEA_DIR +from aea.aea import AEA +from aea.configurations.base import ProtocolConfig +from aea.connections.local.connection import LocalNode, OEFLocalConnection +from aea.crypto.ledger_apis import LedgerApis +from aea.crypto.wallet import Wallet +from aea.mail.base import MailBox, Envelope +from aea.protocols.base import Protocol +from aea.protocols.default.message import DefaultMessage +from aea.protocols.default.serialization import DefaultSerializer +from aea.protocols.fipa.message import FIPAMessage +from aea.protocols.fipa.serialization import FIPASerializer +from aea.registries.base import Resources +from aea.skills.base import Skill +from .conftest import CUR_PATH, DummyConnection +``` + + + +## Initialise the agent + +Create a node. + +``` python +node = LocalNode() +``` + +Initialise a `MailBox` object with a public key and grab the private key and add it to a wallet. + +``` python +public_key_1 = "mailbox1" +mailbox1 = MailBox(OEFLocalConnection(public_key_1, node)) +private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") +wallet = Wallet({'default': private_key_pem_path}) +``` + +Using a variable for accessing running ledgers, initialise the agent. + +``` python +ledger_apis = LedgerApis({}) +my_AEA = AEA("Agent0", mailbox1, wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "aea")))) +``` + + +## Run the agent + +Running the agent invokes the `act()` function in a `Behaviour` object. + +Initialise the agent as above and add a mailbox to it. + +``` python +mailbox = MailBox(OEFLocalConnection(public_key, node)) +``` + +Create a thread and add the agent to it. + +``` python +t = Thread(target=my_AEA.start) +``` + +Start the agent and sleep it. + +``` python +t.start() +time.sleep(1) +``` + +Make sure the `act()` function was called by grabbing the agent's `Behaviour` object and running an assert on it. + +``` python +behaviour = agent.resources.behaviour_registry.fetch("dummy") +assert behaviour[0].nb_act_called > 0, "Act() wasn't called" +``` + +Finalise the agent thread. + +``` python +finally: + agent.stop() + t.join() +``` + + +## Create an envelope + +An envelope carries an encoded message. Create the message object with contents then serialise it. + +After that, add the serialised message to an envelope object. + +``` python +msg = DefaultMessage(type=DefaultMessage.Type.BYTES, content=b"hello") +message_bytes = DefaultSerializer().encode(msg) + +envelope = Envelope( + to="Agent1", + sender=public_key, + protocol_id="default", + message=message_bytes) +``` + + +## Ensure receipt of envelope messages + +Start up the agent thread as in all previous steps then add the envelope you just created in to the message queue. + +``` python +t.start() +agent.mailbox.inbox._queue.put(envelope) +time.sleep(1) +``` + +Grab a `Handler` from the agent `handler_registry` to make sure messages are coming in. + +``` python +handler = agent.resources.handler_registry.fetch_by_skill('default', "dummy") +assert msg in handler.handled_messages, "The message is not inside the handled_messages." +``` + + +## Test the `handle()` method + +Initialise and start the agent as in previous steps but this time create a `DummyConnection` object to add to the `MailBox` so you have a variable name for accessing the `Connection`. + +``` python +connection = DummyConnection() +mailbox = MailBox(connection) +``` + +Create the message envelope as before then start the agent in a `try` clause and add the envelope to the `DummyConnection`. + +``` python +t.start() +time.sleep(1.0) +connection.in_queue.put(envelope) +``` + +Check the out queue is functioning correctly. + +``` python +env = connection.out_queue.get(block=True, timeout=5.0) +assert env.protocol_id == "default" +``` + +``` python +# DECODING ERROR +msg = "hello".encode("utf-8") +envelope = Envelope( + to=public_key, + sender=public_key, + protocol_id='default', + message=msg) +connection.in_queue.put(envelope) +env = connection.out_queue.get(block=True, timeout=5.0) +assert env.protocol_id == "default" +``` + +``` python +# UNSUPPORTED SKILL +msg = FIPASerializer().encode( + FIPAMessage(performative=FIPAMessage.Performative.ACCEPT, + message_id=0, + dialogue_id=0, + destination=public_key, + target=1)) +envelope = Envelope( + to=public_key, + sender=public_key, + protocol_id="fipa", + message=msg) +connection.in_queue.put(envelope) +env = connection.out_queue.get(block=True, timeout=5.0) +assert env.protocol_id == "default" +``` + + +
\ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 9b324b8e84..290e42614b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,11 +2,11 @@ Do you want to create software to work for you and enrich your life? -Autonomous Economic Agents - or AEAs - work continously for your benefit without you having to do anything more than write them and start them up. +Autonomous Economic Agents (AEAs) work continuously for your benefit without you having to do anything more than write them and start them up. -AEAs are able to act independent of your constant input and to autonomously develop new capabilities. Their goal is to create economic gain for you, their owner. +AEAs act independently of constant input and autonomously develop new capabilities. Their goal is to create economic gain for you, their owner. -AEAs have a wide range of application areas. Check out the demo section for examples. +AEAs have a wide range of application areas. Check out the demo section for examples. Bridging Web 2.0 to Web 3.0, AEAs are the future, now. diff --git a/docs/logging.md b/docs/logging.md index d2c42f6d37..ed1f3f5d4c 100644 --- a/docs/logging.md +++ b/docs/logging.md @@ -1,8 +1,8 @@ -The AEA framework supports flexible logging capabilities with the standard [Python logging library](https://docs.python.org/3/library/logging.html). +The AEA framework supports flexible logging capabilities with the standard Python logging library. -In this tutorial, we will configure logging for an agent. +In this tutorial, we configure logging for an agent. -First of all, create your agent: +First of all, create your agent. ``` bash @@ -10,8 +10,9 @@ aea create my_agent cd my_agent ``` -The `aea-config.yaml` file should look like: -```yaml +The `aea-config.yaml` file should look like this. + +``` yaml aea_version: 0.1.6 agent_name: my_agent authors: '' @@ -32,16 +33,14 @@ logging_config: version: 1 ``` -By updating the `logging_config` section, you can configure -the loggers of your application. +By updating the `logging_config` section, you can configure the loggers of your application. + +The format of this section is specified in the `logging.config` module. -The format of this section is specified in the -[`logging.config`](https://docs.python.org/3/library/logging.config.html) -module. -At [this section](https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema) +At this section you'll find the definition of the configuration dictionary schema. -An example of `logging_config` value is reported below: +Below is an example of the `logging_config` value. ```yaml logging_config: @@ -69,10 +68,7 @@ logging_config: propagate: true ``` -This configuration will set up a logger with name `aea`, -print both on console (see `console` handler) and on file -(see `logfile` handler) with format specified by the -`standard` formatter. +This configuration will set up a logger with name `aea`. It prints both on console and on file with a format specified by the `standard` formatter.
\ No newline at end of file diff --git a/docs/oef-ledger.md b/docs/oef-ledger.md index b0266a6669..8f7293512e 100644 --- a/docs/oef-ledger.md +++ b/docs/oef-ledger.md @@ -1,4 +1,4 @@ -In the AEA framework universe, agents run alongside OEF search and discovery nodes against the Fetch.ai ledger and external ledger systems. +In the AEA framework universe, agents run alongside OEF search and discovery nodes which are running against the Fetch.ai ledger and external ledger systems. -
![The AEA, OEF, and Ledger systems](assets/oef-ledger.png)
\ No newline at end of file +
![The AEA, OEF, and Ledger systems](assets/oef-ledger.png)
diff --git a/docs/protocol.md b/docs/protocol.md index 9cd7c89399..4a6ed45cc2 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -88,7 +88,7 @@ A `models.py` module is provided by the `oef` protocol which includes classes an The `fipa` protocol definition includes a `FIPAMessage` class which gets a `protocol_id` of `fipa`. -It defines FIPA negotiating terms by way of a `Peformative(Enum)`. +It defines FIPA negotiating terms by way of a `Performative(Enum)`. ``` python class Performative(Enum): @@ -165,4 +165,4 @@ The serialisation methods `encode` and `decode` implement transformations from ` -
\ No newline at end of file +
diff --git a/docs/quickstart.md b/docs/quickstart.md index c4783ad867..2cd77ec94d 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -59,17 +59,16 @@ pip install aea[cli] ### Known issues -If the installation steps fail, it might be because some of - the dependencies cannot be built on your system. +If the installation steps fail, it might a dependency issue. The following hints can help: -- Ubuntu/Debian systems only: install Python 3.7 headers +- Ubuntu/Debian systems only: install Python 3.7 headers. ```bash sudo apt-get install python3.7-dev ``` -- Windows users: install [build tools for Visual Studio](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019). +- Windows users: install tools for Visual Studio. ## Echo skill demo @@ -94,13 +93,11 @@ cd my_first_agent ### Add the echo skill ``` bash - aea add skill echo ``` This copies the echo application code for the behaviours, handlers, and tasks into the skill, ready to run. - ### Add a stub connection AEAs use messages for communication. We will add a stub connection to send messages to and receive messages from the AEA. diff --git a/docs/skill-guide.md b/docs/skill-guide.md index ddfe458013..2a057f51cb 100644 --- a/docs/skill-guide.md +++ b/docs/skill-guide.md @@ -1,4 +1,4 @@ -The scaffolding tool allows you create the folder structure required for a skill. +The scaffolding tool allows you to create the folder structure required for a skill. !!! Note Before developing your first skill, please read the skill guide. @@ -6,14 +6,16 @@ The scaffolding tool allows you create the folder ## Step 1: Setup -Ensure, you have followed the preliminaries and installation. We will first create an agent and add a scaffold skill, which we call `my_search`: +Make sure you have followed the preliminaries and installation instructions from the quick start. + +We will first create an agent and add a scaffold skill, which we call `my_search`. ``` bash aea create my_agent && cd my_agent aea scaffold skill my_search ``` -In the following steps, we will replace each one of the scaffolded `Behaviour`, `Handler` and `Task` in `my_agent/skills/my_search` with our implementation. We will build a simple skill which lets the agent send a search query to the [OEF](https://docs.fetch.ai/oef/) and process the resulting response. +In the following steps, we replace each one of the scaffolded `Behaviour`, `Handler` and `Task` in `my_agent/skills/my_search` with our implementation. We will build a simple skill which lets the agent send a search query to the [OEF](https://docs.fetch.ai/oef/) and process the resulting response. ## Step 2: Develop a Behaviour @@ -79,7 +81,7 @@ class MySearchBehaviour(Behaviour): logger.info("[{}]: tearing down MySearchBehaviour".format(self.context.agent_name)) ``` -Searches are proactive and as such well placed in a `Behaviour`. +Searches are proactive and, as such, well placed in a `Behaviour`. We place this code in `my_agent/skills/my_search/behaviours.py`. @@ -200,9 +202,9 @@ We place this code in `my_agent/skills/my_search/tasks.py`. ## Step 5: Create the config file -Based on our skill components above, we create the following config file: +Based on our skill components above, we create the following config file. -```yaml +``` yaml name: my_search authors: Fetch.ai Limited version: 0.1.0 @@ -231,20 +233,20 @@ We place this code in `my_agent/skills/my_search/skill.yaml`. ## Step 6: Add the oef protocol -Our agent does not have the oef protocol yet. Hence, we add it like so: +Our agent does not have the oef protocol yet so let's add it. ```bash aea add protocol oef ``` ## Step 7: Run the agent -We first start an oef node (see the connection section for more details) in a separate terminal window: +We first start an oef node (see the connection section for more details) in a separate terminal window. ```bash python scripts/oef/launch.py -c ./scripts/oef/launch_config.json ``` -We can then launch our agent: +We can then launch our agent. ```bash aea run ``` @@ -252,8 +254,9 @@ aea run Stop the agent with `CTRL + C`. -## Now it's your turn: +## Now it's your turn + +We hope this step by step introduction has helped you develop your own skill. We are excited to see what you will build. -We hope this step by step introduction has helped you to develop your own skill. We are excited to see what you will build.
diff --git a/docs/steps.md b/docs/steps.md index 625a6e80a1..efb9e54b49 100644 --- a/docs/steps.md +++ b/docs/steps.md @@ -2,9 +2,9 @@
  • There are a number of ways to build an agent.
      • -
    • We recommended you build an AEA project with the CLI tool as mentioned in the quick start guide. See information on the CLI tool here.
      • +
    • We recommended you build an AEA project with the CLI tool as mentioned in the quick start guide. See information on the CLI tool here.
    • [Coming soon!] Using the CLI fetch command, pull in an already built project and run as normal.
      • -
    • The last option is to install the AEA without the CLI tool with pip install aea and, from there, import classes directly.
      • +
    • The last option is to install the AEA without the CLI tool with pip install aea and, from there, import classes directly. See the guide for this here.
    diff --git a/docs/tac.md b/docs/tac.md index 337ead70fb..6eec01a541 100644 --- a/docs/tac.md +++ b/docs/tac.md @@ -10,7 +10,7 @@ Make sure you are running diff --git a/docs/weather-skills.md b/docs/weather-skills.md index 2f63c6ea7b..516a370a47 100644 --- a/docs/weather-skills.md +++ b/docs/weather-skills.md @@ -1,4 +1,7 @@ -The AEA weather skills demonstrate an interaction between two AEAs; one as the provider of weather data (the weather station), the other as the seller of weather data (the weather client). +The AEA weather skills demonstrate an interaction between two AEAs. + +* The provider of weather data (the weather station). +* The seller of weather data (the weather client). ## Prerequisites @@ -26,8 +29,9 @@ svn export https://github.com/fetchai/agents-aea.git/trunk/packages svn export https://github.com/fetchai/agents-aea.git/trunk/scripts ``` -## Launch the OEF Node (for search and discovery): -In a separate terminal, launch an OEF node locally: + +## Launch an OEF node +In a separate terminal, launch a local OEF node (for search and discovery). ``` bash python scripts/oef/launch.py -c ./scripts/oef/launch_config.json ``` @@ -38,7 +42,7 @@ Keep it running for all the following demos. The AEAs negotiate and then transfer the data. No payment takes place. This demo serves as a demonstration of the negotiation steps. -### Create the weather station AEA: +### Create the weather station AEA In the root directory, create the weather station AEA. ``` bash aea create my_weather_station @@ -98,7 +102,7 @@ aea delete my_weather_client ## Demo 2: Fetch.ai ledger payment -A demo to run the same scenario but with a true ledger transaction on Fetch.ai test net. This demo assumes the weather client trusts the weather station to send the weather data upon successful payment. +A demo to run the same scenario but with a true ledger transaction on Fetch.ai `testnet`. This demo assumes the weather client trusts the weather station to send the weather data upon successful payment. ### Create the weather station (ledger version) @@ -110,9 +114,9 @@ cd my_weather_station aea add skill weather_station_ledger ``` -### Create the weather client (ledger version): +### Create the weather client (ledger version) -In another terminal, create the AEA that will query the weather station +In another terminal, create the AEA that will query the weather station. ``` bash aea create my_weather_client @@ -120,7 +124,7 @@ cd my_weather_client aea add skill weather_client_ledger ``` -Additionally, create the private key for the weather client AEA +Additionally, create the private key for the weather client AEA. ```bash aea generate-key fetchai ``` @@ -128,7 +132,7 @@ aea generate-key fetchai ### Update the AEA configs Both in `weather_station/aea-config.yaml` and -`weather_client/aea-config.yaml`, replace `ledger_apis: []` with: +`weather_client/aea-config.yaml`, replace `ledger_apis: []` with the following. ``` yaml ledger_apis: @@ -140,7 +144,7 @@ ledger_apis: ### Fund the weather client AEA -Create some wealth for your weather client on the Fetch.ai test net (it takes a while): +Create some wealth for your weather client on the Fetch.ai `testnet`. (It takes a while). ``` bash cd .. python scripts/fetchai_wealth_generation.py --private-key weather_client/fet_private_key.txt --amount 10000000 --addr alpha.fetch-ai.com --port 80 @@ -149,12 +153,12 @@ cd my_weather_client ### Run the AEAs -Run both AEAs, from their respective terminals +Run both AEAs from their respective terminals. ``` bash aea run ``` -You will see that the AEAs negotiate and then transact using the Fetch.ai test net. +You will see that the AEAs negotiate and then transact using the Fetch.ai `testnet`. ### Delete the AEAs @@ -168,7 +172,7 @@ aea delete my_weather_client ## Demo 3: Ethereum ledger payment -A demo to run the same scenario but with a true ledger transaction on Fetch.ai test net. This demo assumes the weather client trusts the weather station to send the weather data upon successful payment. +A demo to run the same scenario but with a true ledger transaction on the Ethereum Ropsten `testnet`. This demo assumes the weather client trusts the weather station to send the weather data upon successful payment. ### Create the weather station (ledger version) @@ -180,9 +184,9 @@ cd my_weather_station aea add skill weather_station_ledger ``` -### Create the weather client (ledger version): +### Create the weather client (ledger version) -In another terminal, create the AEA that will query the weather station +In another terminal, create the AEA that will query the weather station. ``` bash aea create my_weather_client @@ -190,7 +194,7 @@ cd my_weather_client aea add skill weather_client_ledger ``` -Additionally, create the private key for the weather client AEA +Additionally, create the private key for the weather client AEA. ```bash aea generate-key ethereum ``` @@ -198,7 +202,7 @@ aea generate-key ethereum ### Update the AEA configs Both in `weather_station/aea-config.yaml` and -`weather_client/aea-config.yaml`, replace `ledger_apis: []` with: +`weather_client/aea-config.yaml`, replace `ledger_apis: []` with the following. ``` yaml ledger_apis: @@ -210,41 +214,40 @@ ledger_apis: ### Update the skill configs -In the weather station skill config (`my_weather_station/skills/weather_station_ledger/skill.yaml`) under strategy change the `currency_pbk` and `ledger_id` as follows: -``` +In the weather station skill config (`my_weather_station/skills/weather_station_ledger/skill.yaml`) under strategy, amend the `currency_pbk` and `ledger_id` as follows. +``` bash currency_pbk: 'ETH' ledger_id: 'ethereum' ``` -and under ledgers change to: -``` +Amend `ledgers` to the following. +``` bash ledgers: ['ethereum'] ``` -In the weather client skill config (`my_weather_client/skills/weather_client_ledger/skill.yaml`) under strategy change the `currency_pbk` and `ledger_id` as follows: -``` +In the weather client skill config (`my_weather_client/skills/weather_client_ledger/skill.yaml`) under strategy change the `currency_pbk` and `ledger_id`. +``` bash max_buyer_tx_fee: 20000 currency_pbk: 'ETH' ledger_id: 'ethereum' ``` -and under ledgers change to: -``` +Amend `ledgers` to the following. +``` basgh ledgers: ['ethereum'] ``` ### Fund the weather client AEA -Create some wealth for your weather client on the Ethereum Ropsten test net: +Create some wealth for your weather client on the Ethereum Ropsten test net. -Go to Metamask [Faucet](https://faucet.metamask.io) and request some test ETH for the account your weather client AEA is using (you need to first load your AEAs private key into MetaMask). Your private key is at `weather_client/eth_private_key.txt`. +Go to the MetaMask Faucet and request some test ETH for the account your weather client AEA is using (you need to first load your AEAs private key into MetaMask). Your private key is at `weather_client/eth_private_key.txt`. ### Run the AEAs -Run both AEAs, from their respective terminals +Run both AEAs, from their respective terminals. ``` bash aea run ``` - -You will see that the AEAs negotiate and then transact using the Fetch.ai test net. +You will see that the AEAs negotiate and then transact using the Ethereum `testnet`. ### Delete the AEAs From 13249956712e6c15a0a4cbdb71d244e6e1dc0ae2 Mon Sep 17 00:00:00 2001 From: David Minarsch Date: Tue, 5 Nov 2019 13:42:53 +0000 Subject: [PATCH 14/52] Add test in aea --- tests/data/aea-config.example_w_keys.yaml | 40 +++++++++++++++++++++++ tests/test_aea.py | 3 ++ 2 files changed, 43 insertions(+) create mode 100644 tests/data/aea-config.example_w_keys.yaml diff --git a/tests/data/aea-config.example_w_keys.yaml b/tests/data/aea-config.example_w_keys.yaml new file mode 100644 index 0000000000..d12c285049 --- /dev/null +++ b/tests/data/aea-config.example_w_keys.yaml @@ -0,0 +1,40 @@ +aea_version: 0.1.1 +agent_name: myagent +authors: Fetch.AI Limited +version: 0.1.0 +license: Apache 2.0 +url: "" +connections: +- default-oef +default_connection: default-oef +private_key_paths: +- private_key_path: + ledger: default + path: 'private.pem' +- private_key_path: + ledger: fetchai + path: 'fet_private_key.txt' +- private_key_path: + ledger: ethereum + path: 'eth_private_key.txt' +protocols: +- oef +- default +- tac +- fipa +skills: +- echo_skill +description: "An example of agent configuration file for testing purposes." +logging_config: + disable_existing_loggers: false + version: 1 +registry_path: ../../packages +ledger_apis: +- ledger_api: + ledger: fetchai + addr: example.fetch-ai.com + port: 8080 +- ledger_api: + ledger: ethereum + addr: example.ethereum.com + port: 8080 diff --git a/tests/test_aea.py b/tests/test_aea.py index bb82443dd7..45e4cdccb8 100644 --- a/tests/test_aea.py +++ b/tests/test_aea.py @@ -55,6 +55,9 @@ def test_initialise_AEA(): my_AEA.setup() assert my_AEA.resources is not None,\ "Resources must not be None after setup" + my_AEA.resources = Resources(str(Path(CUR_PATH, "aea"))) + assert my_AEA.resources is not None,\ + "Resources must not be None after set" def test_act(): From 7faa9c1e94a3fb4e40400434bdf18913240b9a69 Mon Sep 17 00:00:00 2001 From: Katharine Murphy Date: Tue, 5 Nov 2019 14:48:16 +0000 Subject: [PATCH 15/52] Edits from comments --- docs/hacking-an-agent.md | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/docs/hacking-an-agent.md b/docs/hacking-an-agent.md index f10b451bc1..7a49e4cb61 100644 --- a/docs/hacking-an-agent.md +++ b/docs/hacking-an-agent.md @@ -43,39 +43,41 @@ from .conftest import CUR_PATH, DummyConnection ## Initialise the agent -Create a node. - +Create a `Connection` and a `MailBox`. ``` python -node = LocalNode() +stub_connection = StubConnection(input_file_path='input.txt', output_file_path='output.txt') +mailbox = MailBox(stub_connection) ``` -Initialise a `MailBox` object with a public key and grab the private key and add it to a wallet. - +Create a wallet with a private key. ``` python -public_key_1 = "mailbox1" -mailbox1 = MailBox(OEFLocalConnection(public_key_1, node)) private_key_pem_path = os.path.join(CUR_PATH, "data", "priv.pem") wallet = Wallet({'default': private_key_pem_path}) ``` -Using a variable for accessing running ledgers, initialise the agent. +Get the public key from the wallet. You would use this to talk to an OEF or Fetch.ai ledger node. +``` python +public_key = wallet.public_keys['default'] +``` +For ledger APIs, we simply feed the agent a dictionary. ``` python ledger_apis = LedgerApis({}) -my_AEA = AEA("Agent0", mailbox1, wallet, ledger_apis, resources=Resources(str(Path(CUR_PATH, "aea")))) ``` +Create the resources. +``` python +resources = Resources(str(Path(CUR_PATH, "aea"))) +``` -## Run the agent - -Running the agent invokes the `act()` function in a `Behaviour` object. - -Initialise the agent as above and add a mailbox to it. - +Now we have everything we need for initialisation. ``` python -mailbox = MailBox(OEFLocalConnection(public_key, node)) +my_AEA = AEA("my_agent", mailbox, wallet, ledger_apis, resources) ``` + +## Run the agent + Create a thread and add the agent to it. ``` python @@ -99,9 +101,9 @@ assert behaviour[0].nb_act_called > 0, "Act() wasn't called" Finalise the agent thread. ``` python -finally: - agent.stop() - t.join() +agent.stop() +t.join() +t = None ``` From b48a04ab77008cf95a912d06f28ae2832fc93e0f Mon Sep 17 00:00:00 2001 From: Diarmid Campbell Date: Wed, 6 Nov 2019 11:06:50 +0000 Subject: [PATCH 16/52] tests for running agents --- aea/cli_gui/__init__.py | 21 +++++++++++++--- aea/cli_gui/aea_cli_rest.yaml | 8 ++++-- aea/cli_gui/static/css/home.css | 43 +++++++++++++++++++++++++++++++- aea/cli_gui/templates/home.html | 20 ++++++++++++--- aea/cli_gui/templates/home.js | 9 ++++--- tests/test_gui/test_run_agent.py | 21 +++++++++++++--- 6 files changed, 105 insertions(+), 17 deletions(-) diff --git a/aea/cli_gui/__init__.py b/aea/cli_gui/__init__.py index 82cf539676..668d5076af 100644 --- a/aea/cli_gui/__init__.py +++ b/aea/cli_gui/__init__.py @@ -264,7 +264,7 @@ def stop_oef_node(): return "All fine", 200 # 200 (OK) -def start_agent(agent_id): +def start_agent(agent_id, connection_id): """Start a local agent running.""" # Test if it is already running in some form if agent_id in flask.app.agent_processes: @@ -279,7 +279,20 @@ def start_agent(agent_id): return {"detail": "Agent {} is already running".format(agent_id)}, 400 # 400 Bad request agent_dir = os.path.join(flask.app.agents_dir, agent_id) - agent_process = _call_aea_async(["aea", "run"], agent_dir) + + if connection_id is not None and connection_id != "": + connections = get_local_items(agent_id, "connection") + has_named_connection = False + for element in connections: + if element["id"] == connection_id: + has_named_connection = True + if has_named_connection: + agent_process = _call_aea_async(["aea", "run", "--connection", connection_id], agent_dir) + else: + return {"detail": "Trying to run agent {} with non-existant connection: {}".format(agent_id, connection_id)}, 400 # 400 Bad request + else: + agent_process = _call_aea_async(["aea", "run"], agent_dir) + if agent_process is None: return {"detail": "Failed to run agent {}".format(agent_id)}, 400 # 400 Bad request else: @@ -412,7 +425,7 @@ def favicon(): return app def run(): - + """Run the GUI.""" _kill_running_oef_nodes() app = create_app() app.run(host='127.0.0.1', port=8080, debug=False) @@ -420,7 +433,7 @@ def run(): return app def run_test(): - + """Run the gui in the form where we can run tests against it.""" #_kill_running_oef_nodes() app = create_app() return app.app.test_client() diff --git a/aea/cli_gui/aea_cli_rest.yaml b/aea/cli_gui/aea_cli_rest.yaml index 0493cc06e3..308105bd77 100644 --- a/aea/cli_gui/aea_cli_rest.yaml +++ b/aea/cli_gui/aea_cli_rest.yaml @@ -292,17 +292,21 @@ paths: description: id of agent to run type: string required: True + - name: connection_id + in: body + description: id f the connection to activate when running + schema: + type: string + required: True responses: 201: description: Start the OEF Nodoe schema: type: string - 400: description: Cannot start agent schema: type: string - get: operationId: aea.cli_gui.get_agent_status tags: diff --git a/aea/cli_gui/static/css/home.css b/aea/cli_gui/static/css/home.css index 91facdc8e1..a8f7198dfd 100644 --- a/aea/cli_gui/static/css/home.css +++ b/aea/cli_gui/static/css/home.css @@ -155,5 +155,46 @@ td { background-color: black; color: red; font-family: "Courier New", Courier, monospace; +} + +/* Dropdown Button */ +.dropbtn { + background-color: #4CAF50; + color: white; + padding: 16px; + font-size: 16px; + border: none; +} + +/* The container
    - needed to position the dropdown content */ +.dropdown { + position: relative; + display: inline-block; +} + +/* Dropdown Content (Hidden by Default) */ +.dropdown-content { + display: none; + position: absolute; + background-color: #f1f1f1; + min-width: 160px; + box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2); + z-index: 1; +} + +/* Links inside the dropdown */ +.dropdown-content a { + color: black; + padding: 12px 16px; + text-decoration: none; + display: block; +} + +/* Change color of dropdown links on hover */ +.dropdown-content a:hover {background-color: #ddd;} + +/* Show the dropdown menu on hover */ +.dropdown:hover .dropdown-content {display: block;} -} \ No newline at end of file +/* Change the background color of the dropdown button when the dropdown content is shown */ +.dropdown:hover .dropbtn {background-color: #3e8e41;}w {display:block;} \ No newline at end of file diff --git a/aea/cli_gui/templates/home.html b/aea/cli_gui/templates/home.html index 7efcf6f887..1eb477c387 100644 --- a/aea/cli_gui/templates/home.html +++ b/aea/cli_gui/templates/home.html @@ -88,14 +88,24 @@

    Selected {{htmlElements[i][1]}}:

    Running "NONE" Agent

    + + + + + + + + + +

    Agent Status: NONE

    -
    -
    +

    -
    -
    +

    @@ -129,8 +139,10 @@

    Selected {{htmlElements[i][1]}}:
    {% endif %} {% endfor %} +

    OEF Node

    +

    OEF Node Status: NONE

    diff --git a/aea/cli_gui/templates/home.js b/aea/cli_gui/templates/home.js index 93c4242347..cc949c13c3 100644 --- a/aea/cli_gui/templates/home.js +++ b/aea/cli_gui/templates/home.js @@ -220,12 +220,14 @@ class Model{ self.$event_pump.trigger('model_error', [xhr, textStatus, errorThrown]); }) } - startAgent(agentId){ + startAgent(agentId, runConnectionId){ var ajax_options = { type: 'POST', url: 'api/agent/' + agentId + '/run', accepts: 'application/json', - contentType: 'plain/text' + contentType: 'application/json', + dataType: 'json', + data: JSON.stringify(runConnectionId) }; var self = this; $.ajax(ajax_options) @@ -525,8 +527,9 @@ class Controller{ $('#startAgent').click({el: element}, function(e) { e.preventDefault(); var agentId = $('#localAgentsSelectionId').html() + var connectionId = $('#runConnectionId').val() if (self.validateId(agentId)){ - self.model.startAgent(agentId) + self.model.startAgent(agentId, connectionId) } else{ alert('Error: Attempting to start agent with ID: ' + agentId); diff --git a/tests/test_gui/test_run_agent.py b/tests/test_gui/test_run_agent.py index e5997cae58..b2c31666ca 100644 --- a/tests/test_gui/test_run_agent.py +++ b/tests/test_gui/test_run_agent.py @@ -40,16 +40,18 @@ def test_create_and_run_agent(self): ) assert response_add.status_code == 201 - # run the agent + + # run the agent with local connection (as no OEF node is running) response_run = self.app.post( 'api/agent/' + agent_name + "/run", - data=None, content_type='application/json', + data=json.dumps("local") ) assert response_run.status_code == 201 time.sleep(2) + # Get the running status response_status = self.app.get( 'api/agent/' + agent_name + "/run", data=None, @@ -60,8 +62,8 @@ def test_create_and_run_agent(self): assert data["error"] == "" assert "RUNNING" in data["status"] - assert "do connected finished" in data["tty"] + # Stop the agent running response_stop = self.app.delete( 'api/agent/' + agent_name + "/run", data=None, @@ -69,4 +71,17 @@ def test_create_and_run_agent(self): ) assert response_stop.status_code == 200 + # Get the running status + response_status = self.app.get( + 'api/agent/' + agent_name + "/run", + data=None, + content_type='application/json', + ) + assert response_status.status_code == 200 + data = json.loads(response_status.get_data(as_text=True)) + + assert "process terminate" in data["error"] + assert "NOT_STARTED" in data["status"] + + From 7ac3893db97d7d3b2757fc1c52ea42396077b5d0 Mon Sep 17 00:00:00 2001 From: Diarmid Campbell Date: Wed, 6 Nov 2019 11:17:05 +0000 Subject: [PATCH 17/52] removed unused css and html code --- aea/cli_gui/static/css/home.css | 43 +-------------------------------- aea/cli_gui/templates/home.html | 9 +------ 2 files changed, 2 insertions(+), 50 deletions(-) diff --git a/aea/cli_gui/static/css/home.css b/aea/cli_gui/static/css/home.css index a8f7198dfd..ea32d223e4 100644 --- a/aea/cli_gui/static/css/home.css +++ b/aea/cli_gui/static/css/home.css @@ -156,45 +156,4 @@ td { color: red; font-family: "Courier New", Courier, monospace; } - -/* Dropdown Button */ -.dropbtn { - background-color: #4CAF50; - color: white; - padding: 16px; - font-size: 16px; - border: none; -} - -/* The container
    - needed to position the dropdown content */ -.dropdown { - position: relative; - display: inline-block; -} - -/* Dropdown Content (Hidden by Default) */ -.dropdown-content { - display: none; - position: absolute; - background-color: #f1f1f1; - min-width: 160px; - box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2); - z-index: 1; -} - -/* Links inside the dropdown */ -.dropdown-content a { - color: black; - padding: 12px 16px; - text-decoration: none; - display: block; -} - -/* Change color of dropdown links on hover */ -.dropdown-content a:hover {background-color: #ddd;} - -/* Show the dropdown menu on hover */ -.dropdown:hover .dropdown-content {display: block;} - -/* Change the background color of the dropdown button when the dropdown content is shown */ -.dropdown:hover .dropbtn {background-color: #3e8e41;}w {display:block;} \ No newline at end of file +own:hover .dropbtn {background-color: #3e8e41;}w {display:block;} \ No newline at end of file diff --git a/aea/cli_gui/templates/home.html b/aea/cli_gui/templates/home.html index 1eb477c387..c604fdb9e7 100644 --- a/aea/cli_gui/templates/home.html +++ b/aea/cli_gui/templates/home.html @@ -88,14 +88,7 @@

    Selected {{htmlElements[i][1]}}:

    Running "NONE" Agent

    - - - - - - - - +