Feature request: Full UserProvidedMarketData Example

[thayes75]

Specifications

• Problem: Feature request
• OS: Linux
• Cvxportfolio version: 1. 3.2
• Python version: 3.9
• Pandas version: 2.2.0
• Data: User-provided

Description

I keep running into issues with setting up an optimization where I have all of the data already prepared. For example a risk model (from vendor), Mean Daily Volumes (in ccy), my own alpha forecasts, etc. However, as the market data input has changed from the older (deprecated) version, I'm running into issues understanding the full set up to pass in my own data. In particular, it still keeps trying to estimate the HistoricalMeanVolumes (see output below).

As such, I was hoping that at some point a more complete example of UserProvidedMarketData would be able to show me where I'm missing something in the set up for passing in my own data.

Snippet of my set up

cvx_mktdat = cvx.UserProvidedMarketData(
    cvx_rets,
    volumes      = cvx_mdv.ffill(),
    min_history  = pd.Timedelta(1,'D'),
    online_usage = False,
    cash_key     = 'cash', 
    universe_selection_in_time = cvx_univ,    # This is full True/False universe by date
)

# Scalars being passed at the moment
hcost_model = cvx.HoldingCost( short_fees=BORROW_CST )
tcost_model = cvx.TransactionCost( a=SPREAD/2., b=MRKT_IMPCT, volume_hat=cvx_mdv.ffill() )

# Risk model is by date/asset from vendor
risk_model  = cvx.FactorModelCovariance( F=cvx_f, d=cvx_d, Sigma_F=cvx_c )


# Set Objective Function  
# Gammas are scalars
# cvx_alps is date/asset level DF of return forecasts
my_obj = cvx.ReturnsForecast( r_hat=cvx_alps ) \
    - gamma_rsk * risk_model  \       
    - gamma_trd * tcost_model \
    - gamma_brw * hcost_model

# Set Constraint List
my_cons  = [ cvx.DollarNeutral(), cvx.MarketNeutral(), cvx.LeverageLimit(max_lev) ]
my_costs = [ tcost_model, hcost_model ]

# Set Policy 
policy   = cvx.SinglePeriodOptimization( 
    objective   = my_obj, 
    constraints = my_cons,
    solver      = 'ECOS',
    ignore_dpp  = ignore_dpp,
    include_cash_return=False,
)

# Set up Simulator and Backtest
simulator = cvx.MarketSimulator( costs=my_costs,market_data=cvx_mktdat )
result = simulator.backtest( policy,start_time=start_date,end_time=end_date,initial_value=1E9)

Additional information

It doesn't matter which start date I chose, it always spits out this error below but I've checked the MDV data coverage with the universe and it's clean.

Output

cvxportfolio.errors.ForecastError: HistoricalMeanVolume can not compute the forecast at time 2015-02-02 00:00:00-05:00 because there are no observation for either some asset or some pair of assets (in the case of covariance).

[enzbus]

Thanks for the detailed post. I think it is a bug, looks like it's ignoring your volume_hat and defaulting to forecasting internally. Let me check, if that is the case I'll fix quickly. Regarding full example request, you are right, hopefully I can put together something in a short time. The code that has been translated for the legacy paper examples is working, in any case, so you can have a look around there.

[enzbus]

Actually, it's probably due to MarketNeutral.

Try this:

cvx.MarketNeutral(benchmark=cvx.MarketBenchmark(mean_volume_forecast=cvx_mdv.ffill()))

If this fixes it we're good with this script. I'll make two separate feature requests for this.

[thayes75]

Thanks, I was able to get things to run if I managed to change the volumes data to cvx_mdv.ffill().bfill() but I also had to filter returns explicitly to my universe_selection_in_time . It looks like the trading_calendar uses the dates from returns as its base frame (i.e., valid dates) and universe_selection_in_time as its security base frame (valid securities).

Perhaps having the universe_selection_in_time act the base frame for dates and securities would be a fix. Either way, I have things running now.

Thanks!

[thayes75]

Will consider this closed.