Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YAML] Refine ppss.yaml for closer 1:1 mapping to bots #451

Closed
16 tasks done
trentmc opened this issue Dec 13, 2023 · 0 comments
Closed
16 tasks done

[YAML] Refine ppss.yaml for closer 1:1 mapping to bots #451

trentmc opened this issue Dec 13, 2023 · 0 comments
Assignees
@trentmc
Labels
Type: Enhancement New feature or request

Comments

@trentmc
Copy link
Member

trentmc commented Dec 13, 2023 •
edited by calina-c
Loading

Background / motivation

Parent: #400 "Re-merge YAML & CLI: revive changes, fix bugs, test better, re-merge"

This helps via:

  • Clarify what ohlcv feeds lake will work with
  • Clarify what gql feeds lake will work with
  • Clarify - for each bot on its own terms - what predictoor feeds it works with. Predictoor, trader, trueval, etc
  • The 1:1 mapping makes it more clear how each bot uses each setup

Pre-requisites and TODOs

  • #446 [YAML] Rename/move files & dirs for proper separation among lake, AI models, analytics
    • refactor feedstr, timeframestr, pairstr, and exchangestr from tuples -> classes/objects
  • #462 Make pdr lake cli-driven command. Do this first!
  • Move {max_n_train, autoregressive_n} from data_ss into predictoor_ss.
  • Implement support for the rest of ppss.yaml (draft below)
  • Ensure: for trueval agent, that it's properly filtering out feeds according to trueval_ss. This will fix
  • Ensure: that sim_ss is using predictoor_ss and trader_ss
  • Is each bot / command properly filtering out feeds according to its corresponding ss?
    • base_predictoor_agent.py -> filters on predictoor_ss.predict_feed and predictoor_ss.ai_model_ss.input_feeds
    • base_trader_agent.py -> filters on trader_ss.feed
    • sim_agent.py -> no work, since it's using predictoor_ss and trader_ss
    • publisher_agent.py -> filters on publisher_ss.feeds -> moved to [YAML] Filter by feeds in publisher module #490
    • trueval_agent.py (?) -> filters on trueval_ss.feeds. Note: doing this will fix #449
    • dfbuyer_agent.py (?) -> filters on dfbuyer_ss.feeds
    • (For the above: should minimize DRY violations when the feeds are grabbed / printed / filtered / printed. Eg via a new helper module in subgraph/)
    • Any other pdr command for stats: what filtering is needed, if any?

Draft ppss.yaml

Here's the new ppss.yaml:

# (web3_pp / network settings is at bottom, because it's long)
lake_ss:
  parquet_dir: parquet_data
  feeds:
    - binance BTC/USDT
#    - binance BTC/USDT ETH/USDT BNB/USDT XRP/USDT ADA/USDT DOGE/USDT SOL/USDT LTC/USDT TRX/USDT DOT/USDT
#    - kraken BTC/USDT
  st_timestr: 2023-06-01_00:00 # starting date for data
  fin_timestr: now # ending date for data

predictoor_ss:
  predict_feed: binance BTC/USDT c 5m
  bot_only:
    s_until_epoch_end: 60 # in s. Start predicting if there's > this time left
    stake_amount: 1 # stake this amount with each prediction. In OCEAN

  aimodel_ss:
    input_feeds:
      - binance BTC/USDT c
      - binance ETH/USDT ohlcv
#      - binance BTC/USDT ETH/USDT BNB/USDT XRP/USDT ADA/USDT DOGE/USDT SOL/USDT LTC/USDT TRX/USDT DOT/USDT ohlcv
#      - kraken BTC/USDT c
    max_n_train: 5000 # no. epochs to train model on
    autoregressive_n : 10 # no. epochs that model looks back, to predict next
    approach: LIN

trader_ss:
  feed: binance BTC/USDT c 5m
  sim_only:
    fee_percent: 0.0 # simulated % fee
    init_holdings:
      - 100000 USDT
      - 0 BTC
    buy_amt: 10 USD # buy this amount in each epoch
  bot_only:
    min_buffer: 30 # in s. only trade if there's > this time left
    max_tries: 10 # max no. attempts to process a feed
    position_size: 3 # buy/sell this amount in each epoch

sim_ss: # sim only. Uses predictoor_ss and trader_ss here
  test_n : 200 # number of epochs to simulate
  do_plot: True
  log_dir: logs

# ------------------------------------------------------------------
# Bots run by OPF

publisher_ss:
  feeds:
    - 5m binance BTC/USDT
#    - 1h binance BTC/USDT
#    - 5m binance BTC/USDT ETH/USDT BNB/USDT XRP/USDT ADA/USDT DOGE/USDT SOL/USDT LTC/USDT TRX/USDT DOT/USDT

  sapphire-mainnet:
    fee_collector_address: 0x0000000000000000000000000000000000000000
  sapphire-testnet:
    fee_collector_address: 0x0000000000000000000000000000000000000000
  development:
    fee_collector_address: 0x0000000000000000000000000000000000000000

trueval_ss:
  feeds:
    - 5m binance BTC/USDT c
#    - 1h binance BTC/USDT c
#    - 5m binance BTC/USDT ETH/USDT BNB/USDT XRP/USDT ADA/USDT DOGE/USDT SOL/USDT LTC/USDT TRX/USDT DOT/USDT ohlcv

  batch_size: 30
  sleep_time: 30

dfbuyer_ss:
  feeds:
    - 5m binance BTC/USDT c
#    - 1h binance BTC/USDT c
#    - 5m binance BTC/USDT ETH/USDT BNB/USDT XRP/USDT ADA/USDT DOGE/USDT SOL/USDT LTC/USDT TRX/USDT DOT/USDT ohlcv

  batch_size: 20
  consume_interval_seconds: 86400
  weekly_spending_limit: 37000

payout_ss:
  batch_size: 250


# ------------------------------------------------------------------
# Network settings


web3_pp:
  
  sapphire-testnet:
    address_file: "~/.ocean/ocean-contracts/artifacts/address.json"
    rpc_url: https://testnet.sapphire.oasis.dev
    subgraph_url: https://v4.subgraph.sapphire-testnet.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph
    owner_addrs: "0xe02a421dfc549336d47efee85699bd0a3da7d6ff" # OPF deployer address
  
  sapphire-mainnet:
    address_file: "~/.ocean/ocean-contracts/artifacts/address.json"
    rpc_url: https://sapphire.oasis.io
    subgraph_url: https://v4.subgraph.sapphire-mainnet.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph
    owner_addrs: "0x4ac2e51f9b1b0ca9e000dfe6032b24639b172703" # OPF deployer address
    
  development:
    address_file: "~/.ocean/ocean-contracts/artifacts/address.json"
    rpc_url: http://localhost:8545
    subgraph_url: http://localhost:9000/subgraphs/name/oceanprotocol/ocean-subgraph
    owner_addrs: "0xe2DD09d719Da89e5a3D0F2549c7E24566e947260" # OPF deployer address. Taken from ocean.py setup-local.md FACTORY_DEPLOYER_PRIVATE_KEY

  barge-predictoor-bot:
    address_file: "~/barge-predictoor-bot.address.json"
    private_key: "0xc594c6e5def4bab63ac29eed19a134c130388f74f019bc74b8f4389df2837a58" # address is 0xe2DD...
    rpc_url: http://4.245.224.119:8545 # from VPS 
    subgraph_url: http://4.245.224.119:9000/subgraphs/name/oceanprotocol/ocean-subgraph # from VPS
    owner_addrs: "0xe2DD09d719Da89e5a3D0F2549c7E24566e947260" # OPF deployer address. Taken from ocean.py setup-local.md FACTORY_DEPLOYER_PRIVATE_KEY

  barge-pytest:
    address_file: "~/barge-pytest.address.json"
    private_key: "0xc594c6e5def4bab63ac29eed19a134c130388f74f019bc74b8f4389df2837a58"
    rpc_url: http://74.234.16.165:8545
    subgraph_url: http://74.234.16.165:9000/subgraphs/name/oceanprotocol/ocean-subgraph
    owner_addrs: "0xe2DD09d719Da89e5a3D0F2549c7E24566e947260" # OPF deployer address. Taken from ocean.py setup-local.md FACTORY_DEPLOYER_PRIVATE_KEY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@trentmc trentmc

Labels
Type: Enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@calina-c @trentmc