Skip to content

Latest commit

 

History

History
99 lines (63 loc) · 4.57 KB

README.md

File metadata and controls

99 lines (63 loc) · 4.57 KB
Raw

Turing test EXtended Architecture (TEXA): A method for interacting, evaluating and assessing AI in a multi-context environment

CodeFactor Build Status

TEXA is a novel testing and benchmarking framework backed by a simple mathematical theory. It can be used to interact, evaluate and assess AIs that work in multi-context environments.

In simple terms, TEXA enhances the concept of Turing testing by allowing the tester to assign a binary score to each and every interaction with the machine. This will allow us to quantify the quality of our models and contrast them between multiple contexts.

Libraries used

The system uses a number of open source projects to work properly:

  • texalib - Math library for computing the scores in TEXA
  • texajson - Dedicated JSON interpreter library that serves as interface to encode and decode TEXA related data. Also used to publish the results to IPFS network.
  • store - Library used to interact with the Ethereum blockchain
  • ElizaBOT-JS - Javascript implementation of the ELIZA specification by Weizenbaum, 1966. Special thanks to Landsteiner!
  • go-ethereum/* - Ethereum library for interacting with contracts

Installation

Golang

TEXA requires Go Lang v1.10+ to run.

Check your version using the command:

go version

Redis

If go is installed, please make sure that you have a redis instance running on the default port. You can simply run one using the following command:

docker run --name redis-test-instance -p 6379:6379 -d redis

config.json

Once redis is running, make sure that you have created a config.json secret file on your local repository's root folder. Here is an example content:

{
    "ethereum_rpc_endpoint": "https://kovan.infura.io/v3/<YOUR-INFURA-TOKEN-GOES-HERE>",
    "wallet_privatekey": "<YOUR-ETH-WALLET-PRIVATE-KEY>",
    "storage_contract_address": "0x611ef1c4f63b5808535715120ace298a7de835b6"
}

We use a contract on the Kovan testnet to record each test session. So, make sure that enough balance is present in your Ethereum wallet. You can tap more balance using the Kovan Faucet.

IPFS

Since the results are being published to the public IPFS network, please make sure that you are running an IPFS daemon.

You can learn how to install IPFS from here.

If you already have IPFS installed, start it using the command:

ipfs daemon

IPFS cluster (optional)

If you plan to persist the TEXA interrogation and results data for immediate usage, you can make a redundant copy using collaborative IPFS clusters.

Learn how to setup/join collaborative IPFS clusters here.

Run Texa server

Once you have Installed the dependencies, you may start the server:

go run main.go

Now, read the Section 7 "SNAPSHOTS AND RESULTS" in the whitepaper to understand the usage of this system.

If you interact with the system as instructed in the whitepaper, the results of the interrogation session will be written to the public IPFS. The hash of the content on IPFS is used as the data to sign a new transaction on Ethereum Kovan blockchain. When the results are posted on IPFS and committed to the Ethereum blockchain, you'll receive 2 public links as shown below: texa-results-ipfs-blockchain These links are valid and remain useful until public IPFS and ethereum kovan testnet exists. This is a small demonstration of how AI and its testing process could be democratized using blockchain and other decentralized technologies.

TODO (Future Work)

  • Needs testing
  • Lacks complete support for non-Eliza AIs(non-JS reference implementations through APIs).
  • APIs can be exposed to build use-cases such as ranking apps etc.
  • Feel free to try new ideas!

License

Apache 2.0 on the demonstrated work. Derived work carry respective Licenses. Please refer the links.