Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test 900 still failing sporadically #574

Open
barakman opened this issue Apr 25, 2024 · 0 comments
Open

Test 900 still failing sporadically #574

barakman opened this issue Apr 25, 2024 · 0 comments
Assignees
@sklbancor
Labels
bug Something isn't working

Comments

@barakman
Copy link
Collaborator

barakman commented Apr 25, 2024 •
edited
Loading

Bug Description

  • Test /fastlane-bot/resources/NBTest/NBTest_900_OptimizerDetailedSlow.py fails sporadically, even after removing all time-dependent assertions.
  • The previous analysis of this issue was of these time-dependent assertions being the only reason behind the sporadic failures.
  • But following the removal of these assertions from the test, it is possible that some critical decisions inside the optimizer module itself might be based on the randomness embedded in time.time()

Severity (High/Medium/Low)

  • High, due to the last bullet above

Expected Behavior

  • This test should always complete successfully

Actual Behavior

  • This test sometimes raises an exception before completion

Failure Example

2024-04-25T06:58:41.6635898Z _______________________ test_general_and_specific_tests ________________________
2024-04-25T06:58:41.6636543Z 
2024-04-25T06:58:41.6642473Z     def test_general_and_specific_tests():
2024-04-25T06:58:41.6643598Z     # ------------------------------------------------------------
2024-04-25T06:58:41.6644250Z     
2024-04-25T06:58:41.6644616Z         CA = CAm
2024-04-25T06:58:41.6645022Z     
2024-04-25T06:58:41.6645488Z         # ### General tests
2024-04-25T06:58:41.6645977Z     
2024-04-25T06:58:41.6646610Z         # #### General data integrity (should ALWAYS hold)
2024-04-25T06:58:41.6647276Z     
2024-04-25T06:58:41.6647546Z         assert len(pairs0) > 2500
2024-04-25T06:58:41.6648197Z         assert len(pairs) > 2500
2024-04-25T06:58:41.6648752Z         assert len(pairs0) > len(pairs)
2024-04-25T06:58:41.6649160Z         assert len(pairsc) > ***0
2024-04-25T06:58:41.6649536Z         assert len(CCm.tokens()) > 2000
2024-04-25T06:58:41.6649899Z         assert len(CCm)>4000
2024-04-25T06:58:41.6650381Z         assert len(CCm.filter_pairs(onein=f"{T.ETH}")) > ***900 # ETH pairs
2024-04-25T06:58:41.6651048Z         assert len(CCm.filter_pairs(onein=f"{T.USDC}")) > 300 # USDC pairs
2024-04-25T06:58:41.6651688Z         assert len(CCm.filter_pairs(onein=f"{T.USDT}")) > ***90 # USDT pairs
2024-04-25T06:58:41.6652326Z         assert len(CCm.filter_pairs(onein=f"{T.DAI}")) > 50 # DAI pairs
2024-04-25T06:58:41.6652949Z         assert len(CCm.filter_pairs(onein=f"{T.WBTC}")) > 30 # WBTC pairs
2024-04-25T06:58:41.6653404Z     
2024-04-25T06:58:41.6653744Z         xis0 = {c.cid: (c.x, c.y) for c in CCm if c.x==0}
2024-04-25T06:58:41.6654242Z         yis0 = {c.cid: (c.x, c.y) for c in CCm if c.y==0}
2024-04-25T06:58:41.6654814Z         assert len(xis0) == 0 # set loglevel debug to see removal of curves
2024-04-25T06:58:41.6655302Z         assert len(yis0) == 0
2024-04-25T06:58:41.6655781Z     
2024-04-25T06:58:41.6656130Z         # #### Data integrity
2024-04-25T06:58:41.6656631Z     
2024-04-25T06:58:41.6657068Z         assert len(CCm) == 4***55
2024-04-25T06:58:41.6657699Z         assert len(CCu3) == ***4***
2024-04-25T06:58:41.6658319Z         assert len(CCu2) == 2***77
2024-04-25T06:58:41.6659181Z         assert len(CCs2) == 236
2024-04-25T06:58:41.6659834Z         assert len(CCm.tokens()) == 2233
2024-04-25T06:58:41.6660510Z         assert len(CCm.pairs()) == 2834
2024-04-25T06:58:41.6661304Z         assert len(CCm.pairs(standardize=False)) == 2864
2024-04-25T06:58:41.6661961Z     
2024-04-25T06:58:41.6662313Z     
2024-04-25T06:58:41.6662864Z         assert CA.pairs()  == CCm.pairs(standardize=True)
2024-04-25T06:58:41.6664173Z         assert CA.pairsc() == {c.pairo.primary for c in CCm if c.P("exchange")=="carbon_v***"}
2024-04-25T06:58:41.6665232Z         assert CA.tokens() == CCm.tokens()
2024-04-25T06:58:41.6665962Z     
2024-04-25T06:58:41.6666374Z         # #### prices
2024-04-25T06:58:41.6666849Z     
2024-04-25T06:58:41.6667409Z         r*** = CCc***.prices(result=CCc***.PR_TUPLE)
2024-04-25T06:58:41.6668349Z         r2 = CCc***.prices(result=CCc***.PR_TUPLE, primary=False)
2024-04-25T06:58:41.6669430Z         r3 = CCc***.prices(result=CCc***.PR_TUPLE, primary=False, inclpair=False)
2024-04-25T06:58:41.6670350Z         assert isinstance(r***, tuple)
2024-04-25T06:58:41.6671013Z         assert isinstance(r2, tuple)
2024-04-25T06:58:41.6671651Z         assert isinstance(r3, tuple)
2024-04-25T06:58:41.6672288Z         assert len(r***) == len(r2)
2024-04-25T06:58:41.6672914Z         assert len(r***) == len(r3)
2024-04-25T06:58:41.6673528Z         assert len(r***[0]) == 3
2024-04-25T06:58:41.6674157Z         assert isinstance(r***[0][0], str)
2024-04-25T06:58:41.6674866Z         assert isinstance(r***[0][***], float)
2024-04-25T06:58:41.6675576Z         assert len(r***[0][2].split("/"))==2
2024-04-25T06:58:41.6676180Z     
2024-04-25T06:58:41.6676546Z         r2[:2]
2024-04-25T06:58:41.6676942Z     
2024-04-25T06:58:41.6677299Z         r3[:2]
2024-04-25T06:58:41.6677692Z     
2024-04-25T06:58:41.6678292Z         r***a = CCc***.prices(result=CCc***.PR_DICT)
2024-04-25T06:58:41.6679172Z         r2a = CCc***.prices(result=CCc***.PR_DICT, primary=False)
2024-04-25T06:58:41.6680246Z         r3a = CCc***.prices(result=CCc***.PR_DICT, primary=False, inclpair=False)
2024-04-25T06:58:41.6681157Z         assert isinstance(r***a, dict)
2024-04-25T06:58:41.6681812Z         assert isinstance(r2a, dict)
2024-04-25T06:58:41.6682453Z         assert isinstance(r3a, dict)
2024-04-25T06:58:41.6683094Z         assert len(r***a) == len(r***)
2024-04-25T06:58:41.6683730Z         assert len(r***a) == len(r2a)
2024-04-25T06:58:41.6684376Z         assert len(r***a) == len(r3a)
2024-04-25T06:58:41.6685381Z         assert list(r***a.keys()) == list(x[0] for x in r***)
2024-04-25T06:58:41.6686348Z         assert r***a.keys() == r2a.keys()
2024-04-25T06:58:41.6687026Z         assert r***a.keys() == r3a.keys()
2024-04-25T06:58:41.6688073Z         assert set(len(x) for x in r***a.values()) == {2},  "all records must be of of length 2"
2024-04-25T06:58:41.6689554Z         assert set(type(x[0]) for x in r***a.values()) == {float},  "all records must have first type float"
2024-04-25T06:58:41.6691080Z         assert set(type(x[***]) for x in r***a.values()) == {str},  "all records must have second type str"
2024-04-25T06:58:41.6692139Z         assert tuple(r3a.values()) == r3
2024-04-25T06:58:41.6692713Z     
2024-04-25T06:58:41.6693292Z         df =  CCc***.prices(result=CCc***.PR_DF, primary=False)
2024-04-25T06:58:41.6694083Z         assert len(df) == len(r***)
2024-04-25T06:58:41.6694829Z         assert tuple(df.index) == tuple(x[0] for x in r***)
2024-04-25T06:58:41.6695605Z         assert tuple(df["price"]) == r3
2024-04-25T06:58:41.6696186Z         df
2024-04-25T06:58:41.6696556Z     
2024-04-25T06:58:41.6696956Z         # #### more prices
2024-04-25T06:58:41.6697426Z     
2024-04-25T06:58:41.6697913Z         CCt = CCm.bypairs(f"{T.USDC}/{T.ETH}")
2024-04-25T06:58:41.6698531Z     
2024-04-25T06:58:41.6699197Z         r = CCt.prices(result=CCt.PR_TUPLE)
2024-04-25T06:58:41.6699892Z         assert isinstance(r, tuple)
2024-04-25T06:58:41.6700516Z         assert len(r) == len(CCt)
2024-04-25T06:58:41.6701890Z         assert r[0] == ('6c988ffdc9e74acd97ccfb***6dd65c***0', ***833.9007005259564, 'WETH-6Cc2/USDC-eB48')
2024-04-25T06:58:41.6703084Z         assert CCt.prices() == CCt.prices(result=CCt.PR_DICT)
2024-04-25T06:58:41.6703910Z         r = CCt.prices(result=CCt.PR_DICT)
2024-04-25T06:58:41.6704573Z         assert len(r) == len(CCt)
2024-04-25T06:58:41.6705187Z         assert isinstance(r, dict)
2024-04-25T06:58:41.6706406Z         assert r['6c988ffdc9e74acd97ccfb***6dd65c***0'] == (***833.9007005259564, 'WETH-6Cc2/USDC-eB48')
2024-04-25T06:58:41.6707482Z         df = CCt.prices(result=CCt.PR_DF)
2024-04-25T06:58:41.6708146Z         assert len(df) == len(CCt)
2024-04-25T06:58:41.6709551Z         assert tuple(df.loc["***70***4***8346046923***73***6873037***5884***057339-0"]) == (***799.9999997028303, 'WETH-6Cc2/USDC-eB48')
2024-04-25T06:58:41.6710617Z     
2024-04-25T06:58:41.6711005Z         # #### price_ranges
2024-04-25T06:58:41.6711489Z     
2024-04-25T06:58:41.6711970Z         CCt = CCm.bypairs(f"{T.USDC}/{T.ETH}")
2024-04-25T06:58:41.6712656Z         CAt = CPCAnalyzer(CCt)
2024-04-25T06:58:41.6713159Z     
2024-04-25T06:58:41.6713658Z         r = CAt.price_ranges(result=CAt.PR_TUPLE)
2024-04-25T06:58:41.6714362Z         assert len(r) == len(CCt)
2024-04-25T06:58:41.6714927Z         assert r[0] == (
2024-04-25T06:58:41.6715543Z             'WETH/USDC',        # pair
2024-04-25T06:58:41.6716267Z             '***6dd65c***0',       # cid
2024-04-25T06:58:41.6716993Z             'sushiswap_v2',     # exchange
2024-04-25T06:58:41.6717708Z             'b',                # buy
2024-04-25T06:58:41.6718380Z             's',                # sell
2024-04-25T06:58:41.6719026Z             0,                  # min_primary
2024-04-25T06:58:41.6719714Z             None,               # max_primary
2024-04-25T06:58:41.6720375Z             ***833.9007005259564  # pp
2024-04-25T06:58:41.6720924Z         )
2024-04-25T06:58:41.6721330Z         assert r[***] == (
2024-04-25T06:58:41.6721876Z             'WETH/USDC',
2024-04-25T06:58:41.6722409Z             '4***057334-0',
2024-04-25T06:58:41.6722965Z             'carbon_v***',
2024-04-25T06:58:41.6723477Z             'b',
2024-04-25T06:58:41.6723929Z             '',
2024-04-25T06:58:41.6724382Z             ***699.999829864358,
2024-04-25T06:58:41.6724955Z             ***700.000***6986434***,
2024-04-25T06:58:41.6725528Z             ***700.000***6986434***
2024-04-25T06:58:41.6726066Z         )
2024-04-25T06:58:41.6726669Z         r = CAt.price_ranges(result=CAt.PR_TUPLE, short=False)
2024-04-25T06:58:41.6727602Z         assert r[0] == (
2024-04-25T06:58:41.6728403Z             'WETH-6Cc2/USDC-eB48',
2024-04-25T06:58:41.6729198Z             '6c988ffdc9e74acd97ccfb***6dd65c***0',
2024-04-25T06:58:41.6729909Z             'sushiswap_v2',
2024-04-25T06:58:41.6730432Z             'b',
2024-04-25T06:58:41.6730894Z             's',
2024-04-25T06:58:41.6731328Z             0,
2024-04-25T06:58:41.6731779Z             None,
2024-04-25T06:58:41.6732274Z             ***833.9007005259564
2024-04-25T06:58:41.6732751Z         )
2024-04-25T06:58:41.6733071Z         r = CAt.price_ranges(result=CAt.PR_DICT)
2024-04-25T06:58:41.6733496Z         assert len(r) == len(CCt)
2024-04-25T06:58:41.6734021Z         assert r['6c988ffdc9e74acd97ccfb***6dd65c***0'] == (
2024-04-25T06:58:41.6734472Z             'WETH/USDC',
2024-04-25T06:58:41.6734806Z             '***6dd65c***0',
2024-04-25T06:58:41.6735135Z             'sushiswap_v2',
2024-04-25T06:58:41.6735448Z             'b',
2024-04-25T06:58:41.6735726Z             's',
2024-04-25T06:58:41.6735966Z             0,
2024-04-25T06:58:41.6736218Z             None,
2024-04-25T06:58:41.6736509Z             ***833.9007005259564
2024-04-25T06:58:41.6736808Z         )
2024-04-25T06:58:41.6737121Z         df = CAt.price_ranges(result=CAt.PR_DF)
2024-04-25T06:58:41.6737527Z         assert len(df) == len(CCt)
2024-04-25T06:58:41.6738034Z         assert tuple(df.index.names) == ('pair', 'exch', 'cid')
2024-04-25T06:58:41.6738680Z         assert tuple(df.columns) == ('b', 's', 'p_min', 'p_max', 'p_marg')
2024-04-25T06:58:41.6739754Z         assert set(df["p_marg"]) == set(x[-***] for x in CAt.price_ranges(result=CCt.PR_TUPLE))
2024-04-25T06:58:41.6740404Z         for p***, p2 in zip(df["p_marg"], df["p_marg"][***:]):
2024-04-25T06:58:41.6740829Z             assert p2 >= p***
2024-04-25T06:58:41.6741132Z         df
2024-04-25T06:58:41.6741360Z     
2024-04-25T06:58:41.6741598Z         # #### count_by_pairs
2024-04-25T06:58:41.6741882Z     
2024-04-25T06:58:41.6742227Z         assert len(CA.count_by_pairs()) == len(CA.pairs())
2024-04-25T06:58:41.6742744Z         assert sum(CA.count_by_pairs()["count"])==len(CA.CC)
2024-04-25T06:58:41.6743331Z         assert np.all(CA.count_by_pairs() == CA.count_by_pairs(asdf=True))
2024-04-25T06:58:41.6743997Z         assert len(CA.count_by_pairs()) == len(CA.count_by_pairs(asdf=False))
2024-04-25T06:58:41.6744599Z         assert type(CA.count_by_pairs()).__name__ == "DataFrame"
2024-04-25T06:58:41.6745173Z         assert type(CA.count_by_pairs(asdf=False)).__name__ == "list"
2024-04-25T06:58:41.6745786Z         assert type(CA.count_by_pairs(asdf=False)[0]).__name__ == "tuple"
2024-04-25T06:58:41.6746262Z         for i in range(***0):
2024-04-25T06:58:41.6746835Z             assert len(CA.count_by_pairs(minn=i)) >= len(CA.count_by_pairs(minn=i)), f"failed {i}"
2024-04-25T06:58:41.6747366Z     
2024-04-25T06:58:41.6747621Z         # #### count_by_tokens
2024-04-25T06:58:41.6747919Z     
2024-04-25T06:58:41.6748171Z         r = CA.count_by_tokens()
2024-04-25T06:58:41.6748547Z         assert len(r) == len(CA.tokens())
2024-04-25T06:58:41.6748968Z         assert sum(r["total"]) == 2*len(CA.CC)
2024-04-25T06:58:41.6749522Z         assert tuple(r["total"]) == tuple(x[***] for x in CA.CC.token_count())
2024-04-25T06:58:41.6750054Z         for ix, row in r[:***0].iterrows():
2024-04-25T06:58:41.6750570Z             assert row[0] >= sum(row[***:]), f"failed at {ix} {tuple(row)}"
2024-04-25T06:58:41.6751036Z         CA.count_by_tokens()
2024-04-25T06:58:41.6751325Z     
2024-04-25T06:58:41.6751596Z         # #### pool_arbitrage_statistics
2024-04-25T06:58:41.6751937Z     
2024-04-25T06:58:41.6752230Z         pas = CAm.pool_arbitrage_statistics()
2024-04-25T06:58:41.6752766Z         assert np.all(pas == CAm.pool_arbitrage_statistics(CAm.POS_DF))
2024-04-25T06:58:41.6753250Z         assert len(pas)==***65
2024-04-25T06:58:41.6753806Z         assert list(pas.columns) == ['price', 'vl', 'itm', 'b', 's', 'bsv']
2024-04-25T06:58:41.6754485Z         assert list(pas.index.names) == ['pair', 'exchange', 'cid0']
2024-04-25T06:58:41.6755323Z         assert {x[0] for x in pas.index} == {Pair.n(x) for x in CAm.pairsc()}
2024-04-25T06:58:41.6756387Z         assert {x[***] for x in pas.index} == {'bancor_v2', 'bancor_v3','carbon_v***','sushiswap_v2','uniswap_v2','uniswap_v3'}
2024-04-25T06:58:41.6757004Z         pas
2024-04-25T06:58:41.6757234Z     
2024-04-25T06:58:41.6757570Z         pasd = CAm.pool_arbitrage_statistics(CAm.POS_DICT)
2024-04-25T06:58:41.6758022Z         assert isinstance(pasd, dict)
2024-04-25T06:58:41.6758393Z         assert len(pasd) == 26
2024-04-25T06:58:41.6758842Z         assert len(pasd['WETH-6Cc2/DAI-***d0F']) == 7
2024-04-25T06:58:41.6759326Z         pd0 = pasd['WETH-6Cc2/DAI-***d0F'][0]
2024-04-25T06:58:41.6759855Z         assert pd0[:2] == ('WETH/DAI', 'WETH-6Cc2/DAI-***d0F')
2024-04-25T06:58:41.6760355Z         assert iseq(pd0[2], ***840.***2***649***367***3***)
2024-04-25T06:58:41.6760885Z         assert pd0[3:6] == ('594', '594', 'uniswap_v3')
2024-04-25T06:58:41.6761339Z         assert iseq(pd0[6], 8.466598820***98278)
2024-04-25T06:58:41.6761958Z         assert pd0[7:] == ('', 'b', 's', 'buy-sell-WETH @ ***840.***2 DAI per WETH')
2024-04-25T06:58:41.6762422Z         pd0
2024-04-25T06:58:41.6762648Z     
2024-04-25T06:58:41.6763016Z         pasl = CAm.pool_arbitrage_statistics(result = CAm.POS_LIST)
2024-04-25T06:58:41.6763500Z         assert isinstance(pasl, tuple)
2024-04-25T06:58:41.6763883Z         assert len(pasl) == len(pas)
2024-04-25T06:58:41.6764416Z         pd0 = [(ix, x) for ix, x in enumerate(pasl) if x[2]==***840.***2***649***367***3***]
2024-04-25T06:58:41.6764932Z         pd0 = pasl[pd0[0][0]]
2024-04-25T06:58:41.6765408Z         assert pd0[:2] == ('WETH/DAI', 'WETH-6Cc2/DAI-***d0F')
2024-04-25T06:58:41.6765903Z         assert iseq(pd0[2], ***840.***2***649***367***3***)
2024-04-25T06:58:41.6766432Z         assert pd0[3:6] == ('594', '594', 'uniswap_v3')
2024-04-25T06:58:41.6766884Z         assert iseq(pd0[6], 8.466598820***98278)
2024-04-25T06:58:41.6767488Z         assert pd0[7:] == ('', 'b', 's', 'buy-sell-WETH @ ***840.***2 DAI per WETH')
2024-04-25T06:58:41.6767963Z         pd0
2024-04-25T06:58:41.6768188Z     
2024-04-25T06:58:41.6768424Z         # ### MargP Optimizer
2024-04-25T06:58:41.6768715Z     
2024-04-25T06:58:41.6768960Z         # #### margp optimizer
2024-04-25T06:58:41.6769252Z     
2024-04-25T06:58:41.6769640Z         tokenlist = f"{T.ETH},{T.USDC},{T.USDT},{T.BNT},{T.DAI},{T.WBTC}"
2024-04-25T06:58:41.6770118Z         targettkn = f"{T.USDC}"
2024-04-25T06:58:41.6770616Z         O = MargPOptimizer(CCm.bypairs(CCm.filter_pairs(bothin=tokenlist)))
2024-04-25T06:58:41.6771294Z         r = O.margp_optimizer(targettkn, params=dict(verbose=False, debug=False))
2024-04-25T06:58:41.6771909Z         r.trade_instructions(ti_format=O.TIF_DFAGGR).fillna("")
2024-04-25T06:58:41.6772312Z     
2024-04-25T06:58:41.6772575Z         # #### MargpOptimizerResult
2024-04-25T06:58:41.6772905Z     
2024-04-25T06:58:41.6773271Z         assert type(r) == MargPOptimizer.MargpOptimizerResult
2024-04-25T06:58:41.6773830Z         assert iseq(r.result, -4606.0***0***57294979)
2024-04-25T06:58:41.6774243Z         # assert r.time > 0.00***
2024-04-25T06:58:41.6774587Z         # assert r.time < 0.***
2024-04-25T06:58:41.6774950Z         assert r.method == O.METHOD_MARGP
2024-04-25T06:58:41.6775346Z         assert r.targettkn == targettkn
2024-04-25T06:58:41.6776047Z         assert set(r.tokens_t)==set(['USDT-***ec7', 'WETH-6Cc2', 'WBTC-C599', 'DAI-***d0F', 'BNT-FF***C'])
2024-04-25T06:58:41.6776767Z         p_opt_d0 = {t:x for x, t in zip(r.p_optimal_t, r.tokens_t)}
2024-04-25T06:58:41.6777357Z         p_opt_d = {t:round(x,6) for x, t in zip(r.p_optimal_t, r.tokens_t)}
2024-04-25T06:58:41.6777887Z         print("optimal p", p_opt_d)
2024-04-25T06:58:41.6778475Z         assert p_opt_d == {'WETH-6Cc2': ***842.67228, 'WBTC-C599': 27604.***43472,
2024-04-25T06:58:41.6779430Z                         'BNT-FF***C': 0.429078, 'USDT-***ec7': ***.00058, 'DAI-***d0F': ***.000***79}
2024-04-25T06:58:41.6780174Z         assert r.p_optimal[r.targettkn] == ***
2024-04-25T06:58:41.6780837Z         po = [(k,v) for k,v in r.p_optimal.items()][:-***]
2024-04-25T06:58:41.6781296Z         assert len(po)==len(r.p_optimal_t)
2024-04-25T06:58:41.6781659Z         for k,v in po:
2024-04-25T06:58:41.6782088Z             assert p_opt_d0[k] == v, f"error at {k}, {v}, {p_opt_d0[k]}"
2024-04-25T06:58:41.6782514Z     
2024-04-25T06:58:41.6782769Z         # #### TradeInstructions
2024-04-25T06:58:41.6783075Z     
2024-04-25T06:58:41.6783529Z         assert r.trade_instructions() == r.trade_instructions(ti_format=O.TIF_OBJECTS)
2024-04-25T06:58:41.6784148Z         ti = r.trade_instructions(ti_format=O.TIF_OBJECTS)
2024-04-25T06:58:41.6784599Z         cids = tuple(ti_.cid for ti_ in ti)
2024-04-25T06:58:41.6784998Z         assert isinstance(ti, tuple)
2024-04-25T06:58:41.6785344Z         assert len(ti) == 86
2024-04-25T06:58:41.6785700Z         ti0=[x for x in ti if x.cid=="357"]
2024-04-25T06:58:41.6786076Z         assert len(ti0)==***
2024-04-25T06:58:41.6786367Z         ti0=ti0[0]
2024-04-25T06:58:41.6786677Z         assert ti0.cid == ti0.curve.cid
2024-04-25T06:58:41.6787118Z         assert type(ti0).__name__ == "TradeInstruction"
2024-04-25T06:58:41.6787639Z         assert type(ti[0]) == MargPOptimizer.TradeInstruction
2024-04-25T06:58:41.6788101Z         assert ti0.tknin == f"{T.USDT}"
2024-04-25T06:58:41.6788493Z         assert ti0.tknout == f"{T.USDC}"
2024-04-25T06:58:41.6788936Z         assert round(ti0.amtin, 8)  == ***2***4.45596849
2024-04-25T06:58:41.6789473Z         assert round(ti0.amtout, 8) == -***2***6.4***933959
2024-04-25T06:58:41.6789899Z         assert ti0.error is None
2024-04-25T06:58:41.6790206Z         ti[:2]
2024-04-25T06:58:41.6790447Z     
2024-04-25T06:58:41.6790774Z         tid = r.trade_instructions(ti_format=O.TIF_DICTS)
2024-04-25T06:58:41.6791214Z         assert isinstance(tid, tuple)
2024-04-25T06:58:41.6791577Z         assert len(tid) == len(ti)
2024-04-25T06:58:41.6791968Z         tid0=[x for x in tid if x["cid"]=="357"]
2024-04-25T06:58:41.6792351Z         assert len(tid0)==***
2024-04-25T06:58:41.6792659Z         tid0=tid0[0]
2024-04-25T06:58:41.6792953Z         assert type(tid0)==dict
2024-04-25T06:58:41.6793328Z         assert tid0["tknin"] == f"{T.USDT}"
2024-04-25T06:58:41.6793735Z         assert tid0["tknout"] == f"{T.USDC}"
2024-04-25T06:58:41.6794186Z         assert round(tid0["amtin"], 8)  == ***2***4.45596849
2024-04-25T06:58:41.6794745Z         assert round(tid0["amtout"], 8) == -***2***6.4***933959
2024-04-25T06:58:41.6795183Z         assert tid0["error"] is None
2024-04-25T06:58:41.6795504Z         tid[:2]
2024-04-25T06:58:41.6795738Z     
2024-04-25T06:58:41.6796087Z         df = r.trade_instructions(ti_format=O.TIF_DF).fillna("")
2024-04-25T06:58:41.6796548Z         assert tuple(df.index) == cids
2024-04-25T06:58:41.6797099Z         assert np.all(r.trade_instructions(ti_format=O.TIF_DFRAW).fillna("")==df)
2024-04-25T06:58:41.6797634Z         assert len(df) == len(ti)
2024-04-25T06:58:41.6798187Z         assert list(df.columns)[:4] == ['pair', 'pairp', 'tknin', 'tknout']
2024-04-25T06:58:41.6798768Z         assert len(df.columns) == 4 + len(r.tokens_t) + ***
2024-04-25T06:58:41.6799204Z         tif0 = dict(df.loc["357"])
2024-04-25T06:58:41.6799673Z         assert tif0["pair"] == "USDC-eB48/USDT-***ec7"
2024-04-25T06:58:41.6800117Z         assert tif0["pairp"] == "USDC/USDT"
2024-04-25T06:58:41.6800524Z         assert tif0["tknin"] == tid0["tknin"]
2024-04-25T06:58:41.6800952Z         assert tif0[tif0["tknin"]] == tid0["amtin"]
2024-04-25T06:58:41.6801406Z         assert tif0[tif0["tknout"]] == tid0["amtout"]
2024-04-25T06:58:41.6801782Z         df[:2]
2024-04-25T06:58:41.6802016Z     
2024-04-25T06:58:41.6802396Z         dfa = r.trade_instructions(ti_format=O.TIF_DFAGGR).fillna("")
2024-04-25T06:58:41.6802950Z         assert tuple(dfa.index)[:-4] == cids
2024-04-25T06:58:41.6803356Z         assert len(dfa) == len(df)+4
2024-04-25T06:58:41.6803938Z         assert len(dfa.columns) == len(r.tokens_t) + ***
2024-04-25T06:58:41.6804513Z         assert set(dfa.columns) == set(r.tokens_t).union(set([r.targettkn]))
2024-04-25T06:58:41.6805372Z         assert list(dfa.index)[-4:] == ['PRICE', 'AMMIn', 'AMMOut', 'TOTAL NET']
2024-04-25T06:58:41.6805856Z         dfa[:***0]
2024-04-25T06:58:41.6806104Z     
2024-04-25T06:58:41.6806436Z         dfpg = r.trade_instructions(ti_format=O.TIF_DFPG)
2024-04-25T06:58:41.6806947Z         assert set(x[***] for x in dfpg.index) == set(cids)
2024-04-25T06:58:41.6807406Z         assert np.all(dfpg["gain_tknq"]>=0)
2024-04-25T06:58:41.6807805Z         assert np.all(dfpg["gain_r"]>=0)
2024-04-25T06:58:41.6808260Z         assert round(np.max(dfpg["gain_r"]),8) == 0.04739068
2024-04-25T06:58:41.6808829Z         assert round(np.min(dfpg["gain_r"]),8) == ***.772e-05
2024-04-25T06:58:41.6809270Z         assert len(dfpg) == len(ti)
2024-04-25T06:58:41.6809736Z         for p, t in zip(tuple(dfpg["pair"]), tuple(dfpg["tknq"])):
2024-04-25T06:58:41.6810290Z             assert p.split("/")[***] == t, f"error in {p} [{t}]"
2024-04-25T06:58:41.6811056Z         print(f"total gains: {sum(dfpg['gain_ttkn']):,.2f} {r.targettkn} [result={-r.result:,.2f}]")
2024-04-25T06:58:41.6811725Z         assert abs(sum(dfpg["gain_ttkn"])/r.result+***)<0.0***
2024-04-25T06:58:41.6812138Z         dfpg[:***0]
2024-04-25T06:58:41.6812391Z     
2024-04-25T06:58:41.6812641Z         # ### Convex Optimizer
2024-04-25T06:58:41.6812932Z     
2024-04-25T06:58:41.6813274Z         tokens = f"{T.DAI},{T.USDT},{T.HEX},{T.WETH},{T.LINK}"
2024-04-25T06:58:41.6813809Z         CCo  = CCu2.bypairs(CCu2.filter_pairs(bothin=tokens))
2024-04-25T06:58:41.6814330Z         CCo += CCs2.bypairs(CCu2.filter_pairs(bothin=tokens))
2024-04-25T06:58:41.6814767Z         CA   = CPCAnalyzer(CCo)
2024-04-25T06:58:41.6815120Z         O = ConvexOptimizer(CCo)
2024-04-25T06:58:41.6815490Z         #ArbGraph.from_cc(CCo).plot()._
2024-04-25T06:58:41.6815828Z     
2024-04-25T06:58:41.6816066Z         CA.count_by_tokens()
2024-04-25T06:58:41.6816349Z     
2024-04-25T06:58:41.6816560Z         # +
2024-04-25T06:58:41.6816802Z         #CCo.plot()
2024-04-25T06:58:41.6817057Z         # -
2024-04-25T06:58:41.6817275Z     
2024-04-25T06:58:41.6817530Z         # #### convex optimizer
2024-04-25T06:58:41.6817829Z     
2024-04-25T06:58:41.6818058Z         targettkn = T.USDT
2024-04-25T06:58:41.6818555Z         # r = O.margp_optimizer(targettkn, params=dict(verbose=True, debug=False))
2024-04-25T06:58:41.6819411Z         # r
2024-04-25T06:58:41.6819652Z     
2024-04-25T06:58:41.6819948Z         SFC = O.SFC(**{targettkn:O.AMMPays})
2024-04-25T06:58:41.6820497Z         r = O.convex_optimizer(SFC, verbose=False, solver=O.SOLVER_SCS)
2024-04-25T06:58:41.6820945Z         r
2024-04-25T06:58:41.6821167Z     
2024-04-25T06:58:41.6821429Z         # #### NofeesOptimizerResult
2024-04-25T06:58:41.6821758Z     
2024-04-25T06:58:41.6822030Z         round(r.result,-5)
2024-04-25T06:58:41.6822310Z     
2024-04-25T06:58:41.6822687Z         assert type(r) == ConvexOptimizer.NofeesOptimizerResult
2024-04-25T06:58:41.6823380Z         # assert round(r.result,-5) <= -***500000.0
2024-04-25T06:58:41.6823878Z         # assert round(r.result,-5) >= -2500000.0
2024-04-25T06:58:41.6824255Z         # assert r.time < 8
2024-04-25T06:58:41.6824590Z         assert r.method == "convex"
2024-04-25T06:58:41.6825360Z         assert set(r.token_table.keys()) == set(['USDT-***ec7', 'WETH-6Cc2', 'LINK-86CA', 'DAI-***d0F', 'HEX-eb39'])
2024-04-25T06:58:41.6826025Z         assert len(r.token_table[T.USDT].x)==0
2024-04-25T06:58:41.6826465Z         assert len(r.token_table[T.USDT].y)==***0
2024-04-25T06:58:41.6826985Z         lx = list(it.chain(*[rr.x for rr in r.token_table.values()]))
2024-04-25T06:58:41.6827432Z         lx.sort()
2024-04-25T06:58:41.6827832Z         ly = list(it.chain(*[rr.y for rr in r.token_table.values()]))
2024-04-25T06:58:41.6828261Z         ly.sort()
2024-04-25T06:58:41.6828576Z         assert lx == [_ for _ in range(2***)]
2024-04-25T06:58:41.6829122Z         assert ly == lx
2024-04-25T06:58:41.6829397Z     
2024-04-25T06:58:41.6829786Z         # #### trade instructions
2024-04-25T06:58:41.6830093Z     
2024-04-25T06:58:41.6830350Z         ti = r.trade_instructions()
2024-04-25T06:58:41.6830822Z         assert type(ti[0]) == ConvexOptimizer.TradeInstruction
2024-04-25T06:58:41.6831239Z     
2024-04-25T06:58:41.6831687Z         assert r.trade_instructions() == r.trade_instructions(ti_format=O.TIF_OBJECTS)
2024-04-25T06:58:41.6832312Z         ti = r.trade_instructions(ti_format=O.TIF_OBJECTS)
2024-04-25T06:58:41.6832766Z         cids = tuple(ti_.cid for ti_ in ti)
2024-04-25T06:58:41.6833163Z         assert isinstance(ti, tuple)
2024-04-25T06:58:41.6833526Z         assert len(ti) == 2***
2024-04-25T06:58:41.6833893Z         ti0=[x for x in ti if x.cid=="***75"]
2024-04-25T06:58:41.6834268Z         assert len(ti0)==***
2024-04-25T06:58:41.6834566Z         ti0=ti0[0]
2024-04-25T06:58:41.6834866Z         assert ti0.cid == ti0.curve.cid
2024-04-25T06:58:41.6835317Z         assert type(ti0).__name__ == "TradeInstruction"
2024-04-25T06:58:41.6835843Z         assert type(ti[0]) == ConvexOptimizer.TradeInstruction
2024-04-25T06:58:41.6836365Z >       assert ti0.tknin == f"{T.LINK}"
2024-04-25T06:58:41.6836930Z E       AssertionError: assert 'DAI-***d0F' == 'LINK-86CA'
2024-04-25T06:58:41.6837392Z E         - LINK-86CA
2024-04-25T06:58:41.6837715Z E         + DAI-***d0F
2024-04-25T06:58:41.6837904Z 
2024-04-25T06:58:41.6838304Z fastlane_bot/tests/nbtest/test_900_OptimizerDetailedSlow.py:566: AssertionError
@barakman barakman added the bug Something isn't working label Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sklbancor sklbancor

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@barakman @sklbancor