Before proceeding, ensure that Bittensor is installed. Follow the instructions below:
Once bittensor
is installed, proceed with the following steps:
Start by installing the necessary dependencies to run a Substrate node.
Update your system packages:
sudo apt update
Install the required libraries and tools:
sudo apt install --assume-yes make build-essential git clang curl libssl-dev llvm libudev-dev protobuf-compiler
Rust is the programming language used for Substrate development, and Cargo is the Rust package manager.
To install Rust and Cargo:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Update your shell's source to include Cargo's path:
source "$HOME/.cargo/env"
Clone the Subtensor codebase to your local machine:
git clone https://github.com/toilaluan/subtensor.git
Navigate to the Subtensor directory and initialize the server:
cd subtensor
./scripts/init.sh
./scripts/localnet.sh
NOTE: Monitor the build and initialization outputs. The initial build may take some time, depending on your hardware.
Different roles within the subnet, such as subnet owner, subnet validator, and subnet miner, require separate wallets:
- The owner wallet creates and manages the subnet.
- The validator and miner must register with the subnet created by the owner to run their respective scripts.
Create a coldkey for the owner role:
btcli wallet new_coldkey --wallet.name owner
Set up the miner's wallets:
btcli wallet new_coldkey --wallet.name miner
btcli wallet new_hotkey --wallet.name miner --wallet.hotkey default
Set up the validator's wallets:
btcli wallet new_coldkey --wallet.name validator
btcli wallet new_hotkey --wallet.name validator --wallet.hotkey default
Tokens are required to initialize the incentive mechanism on the chain and for registering the subnet.
- The owner needs > τ1000 to create a subnet.
- Validators & Miners need > τ0 to register with the subnet.
Mint faucet tokens for the owner, validator, and miner as follows:
# For the owner
btcli wallet faucet --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946
# Repeat as needed
# For the validator
btcli wallet faucet --wallet.name validator --subtensor.chain_endpoint ws://127.0.0.1:9946
# For the miner
btcli wallet faucet --wallet.name miner --subtensor.chain_endpoint ws://127.0.0.1:9946
Establish a new subnet on the local chain using the following command. The cost is exactly τ1000.000000000 for the first subnet:
btcli subnet create --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946
NOTE: The local chain assigns a default netuid
of 1 for the first subnet. Subsequent registrations will increase the netuid
sequentially. There is a limit of 8 subnets.
Register your subnet validator and miner on the subnet to allocate unique slots. The subnet currently supports up to 128 slots.
Commands for registering the subnet miner and validator:
# For the subnet miner
btcli subnet register --wallet.name miner --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946
# For the subnet validator
btcli subnet register --wallet.name validator --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946
Bootstrap the incentives on your new subnet by adding stake to its incentive mechanism:
btcli stake add --wallet.name validator --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946
- Validators iteratively request prompts from an LLM Server, then send requests to Miners for image generation.
- Miners generate images and return them.
- Validators request rewards from the
Rewarding Server and update the weights.
Clone the necessary repositories and install dependencies:
git clone https://github.com/toilaluan/bittensor-fixed-imagenet.git
git submodule update --init --recursive
pip install -r requirements.txt
Install and run Redis
:
sudo apt install redis
sudo systemctl restart redis
sudo systemctl status redis
Initialize the Prompt and Reward APIs:
# For Prompt API
cd prompt_gen_api
pip install -r requirements.txt
python app.py
# For Reward API
cd reward_api
pip install -r requirements.txt
python app.py
Define enviroment parameters .env
, remember to change url & port based on your system:
PROMPT_ENDPOINT=http://127.0.0.1:15409/prompt_generate
REWARD_ENDPOINT=http://127.0.0.1:15410/verify
MINER_SD_ENDPOINT=http://127.0.0.1:15414/generate
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_LIST=bittensor
Execute the subnet miner and validator, ensuring the correct subnet parameters are specified:
# For the subnet miner
python sd_net/base_miner/miner.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name miner --wallet.hotkey default --logging.debug
# For the subnet validator
python sd_net/validators/validator.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name validator --wallet.hotkey default --logging.debug
We use streamlit for allowing user queue request (prompt) to redis
, validator dequeues request from redis
then do inference.
streamlit run sd_net/validators/webui.py