Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some questions on parameterizing the subproblem and numerical issues #753

Closed
WuSiren opened this issue Jun 26, 2024 · 7 comments
Closed

Some questions on parameterizing the subproblem and numerical issues #753

WuSiren opened this issue Jun 26, 2024 · 7 comments

Comments

@WuSiren
Copy link

WuSiren commented Jun 26, 2024 •
edited
Loading

Hello, Prof. @odow ! I have had some new questions during learning and using SDDP recently. I beg your help and hope you can give me some advice!

  1. The doc says:

Random variables are local to a node $i$, and they can differ between nodes. For example, some nodes may have random variables, and some nodes may not.

but I didn't find where it introduce how to add a random variable (or random process) that has a stage-varying distribution, could you please do me a favor to point me to the link?

  1. Here shows an example about parameterizing the subproblem when the uncertainty is in the objective function. Now I want to confirm if it's legal to parameterize the subproblem in the following way (to invoke the @stageobjective macro outside of the parameterize function) when the uncertainty is implicitly involved in the objective function:
...
@variable(subproblem, x)
@variable(subproblem, u)
@expression(subproblem, obj, x + u)
Ω = [...]
P = [...]
SDDP.parameterize(subproblem, Ω, P) do ω
    JuMP.fix(u, ω)
end
@stageobjective(
    subproblem,
    obj
)
  1. It's about the numerical issue. The Numerical stability report reminds me that
Numerical stability report
  Non-zero Matrix range     [5e-05, 3e+03]
  Non-zero Objective range  [1e+00, 3e+04]
  Non-zero Bounds range     [0e+00, 0e+00]
  Non-zero RHS range        [1e+00, 1e+04]
WARNING: numerical stability issues detected
  - Matrix range contains small coefficients
Very large or small absolute values of coefficients
can cause numerical stability issues. Consider
reformulating the model.

I checked my model carefully and found maybe there are indeed parameters whose values are 0.00005 and some number greater than 1e3, but they are meaningful. So I ignored it (should I?) and tried to train the model with iteration_limit = 200. The following is the output log. Do you think this solution process is normal? Note that there is a Warning: Attempting to recover from serious numerical issues... after Iter. 69. Is the result reliable under this situation? Should I, or can I, get better results by increasing the iteration_limit value, given the Bound seem not to increase any more?

Thank you for your kind help all the time!

------------------------------------------------------------------------------
          SDDP.jl (c) Oscar Dowson and SDDP.jl contributors, 2017-23

Problem
  Nodes           : 12
  State variables : 299
  Scenarios       : 4.09600e+03
  Existing cuts   : false
  Subproblem structure                      : (min, max)
    Variables                               : (994, 994)
    VariableRef in MOI.LessThan{Float64}    : (1, 1)
    VariableRef in MOI.ZeroOne              : (320, 320)
    AffExpr in MOI.LessThan{Float64}        : (407, 407)
    AffExpr in MOI.GreaterThan{Float64}     : (272, 272)
    AffExpr in MOI.EqualTo{Float64}         : (354, 354)
    VariableRef in MOI.GreaterThan{Float64} : (360, 360)
Options
  Solver          : serial mode
  Risk measure    : SDDP.Expectation()
  Sampling scheme : SDDP.InSampleMonteCarlo

Numerical stability report
  Non-zero Matrix range     [5e-05, 3e+03]
  Non-zero Objective range  [1e+00, 3e+04]
  Non-zero Bounds range     [0e+00, 0e+00]
  Non-zero RHS range        [1e+00, 1e+04]
WARNING: numerical stability issues detected
  - Matrix range contains small coefficients
Very large or small absolute values of coefficients
can cause numerical stability issues. Consider
reformulating the model.

 Iteration    Simulation       Bound         Time (s)    Proc. ID   # Solves
        1    2.367443e+07   1.876826e+06   7.160001e-01          1         36
        2    2.381984e+07   2.098343e+06   1.104000e+00          1         72
        3    2.376045e+07   2.100168e+06   1.449000e+00          1        108
        4    2.378467e+07   3.871412e+06   1.795000e+00          1        144
        5    2.387718e+07   3.871412e+06   2.255000e+00          1        180
        6    2.379044e+07   3.873307e+06   2.801000e+00          1        216
        7    2.385231e+07   4.491386e+06   3.286000e+00          1        252
        8    2.377664e+07   4.561269e+06   3.932000e+00          1        288
        9    2.383376e+07   4.561269e+06   4.480000e+00          1        324
       10    2.377919e+07   4.570083e+06   5.004000e+00          1        360
       11    2.386374e+07   4.590386e+06   5.751000e+00          1        396
       12    2.385051e+07   4.594672e+06   6.499000e+00          1        432
       13    2.392658e+07   4.597482e+06   7.382000e+00          1        468
       14    2.376412e+07   4.605299e+06   8.365000e+00          1        504
       15    2.384647e+07   4.605173e+06   9.456000e+00          1        540
       16    2.383210e+07   4.605173e+06   1.062900e+01          1        576
       17    2.382061e+07   4.605173e+06   1.186500e+01          1        612
       18    2.388906e+07   4.605173e+06   1.296700e+01          1        648
       19    2.390420e+07   4.605173e+06   1.420500e+01          1        684
       20    2.390720e+07   4.605173e+06   1.536700e+01          1        720
       21    2.378914e+07   4.605173e+06   1.682600e+01          1        756
       22    2.385449e+07   4.605173e+06   1.832600e+01          1        792
       23    2.379554e+07   4.605173e+06   1.984200e+01          1        828
       24    2.385318e+07   4.605173e+06   2.114000e+01          1        864
       25    2.391281e+07   4.605173e+06   2.234000e+01          1        900
       26    2.386491e+07   4.605425e+06   2.375900e+01          1        936
       27    2.381921e+07   4.605551e+06   2.496400e+01          1        972
       28    2.379814e+07   4.605551e+06   2.663400e+01          1       1008
       29    2.377354e+07   4.605551e+06   2.840500e+01          1       1044
       30    2.378761e+07   4.605551e+06   2.990000e+01          1       1080
       31    2.383779e+07   4.605425e+06   3.212400e+01          1       1116
       32    2.383663e+07   4.608178e+06   3.324900e+01          1       1152
       33    2.383788e+07   8.625664e+06   3.370600e+01          1       1188
       34    2.367122e+07   8.625664e+06   3.425700e+01          1       1224
       35    2.373745e+07   1.063798e+07   3.470400e+01          1       1260
       36    2.387293e+07   1.063798e+07   3.518700e+01          1       1296
       37    2.382405e+07   1.305982e+07   3.579300e+01          1       1332
       38    2.385441e+07   1.305982e+07   3.652400e+01          1       1368
       39    2.382440e+07   1.490478e+07   3.713000e+01          1       1404
       40    2.382554e+07   1.490478e+07   3.772000e+01          1       1440
       41    2.386414e+07   1.490793e+07   3.830800e+01          1       1476
       42    2.376802e+07   1.490793e+07   3.885500e+01          1       1512
       43    2.378664e+07   1.494698e+07   3.945100e+01          1       1548
       44    2.379070e+07   1.555755e+07   4.082600e+01          1       1584
       45    2.375289e+07   1.555755e+07   4.228300e+01          1       1620
       46    2.380519e+07   1.567278e+07   4.308600e+01          1       1656
       47    2.381055e+07   1.567382e+07   4.375200e+01          1       1692
       48    2.378320e+07   1.577178e+07   4.419600e+01          1       1728
       49    2.383064e+07   1.577178e+07   4.473000e+01          1       1764
       50    2.387173e+07   1.580460e+07   4.524600e+01          1       1800
       51    2.384779e+07   1.580460e+07   4.582000e+01          1       1836
       52    2.379365e+07   1.642198e+07   4.652900e+01          1       1872
       53    2.381700e+07   1.684142e+07   4.715300e+01          1       1908
       54    2.385437e+07   1.724181e+07   4.758900e+01          1       1944
       55    2.387841e+07   1.734626e+07   4.830600e+01          1       1980
       56    2.382590e+07   1.748264e+07   4.920200e+01          1       2016
       57    2.383400e+07   1.753408e+07   4.993500e+01          1       2052
       58    2.378086e+07   1.753408e+07   5.038300e+01          1       2088
       59    2.372966e+07   1.781618e+07   5.108000e+01          1       2124
       60    2.379078e+07   1.823780e+07   5.168400e+01          1       2160
       61    2.382505e+07   1.844682e+07   5.247400e+01          1       2196
       62    2.379164e+07   1.845926e+07   5.342400e+01          1       2232
       63    2.375224e+07   1.902534e+07   5.398700e+01          1       2268
       64    2.380357e+07   1.902534e+07   5.502600e+01          1       2304
       65    2.377373e+07   1.903392e+07   5.557200e+01          1       2340
       66    2.374735e+07   1.903392e+07   5.616500e+01          1       2376
       67    2.384913e+07   1.903728e+07   5.688500e+01          1       2412
       68    2.387609e+07   1.905237e+07   5.782200e+01          1       2448
       69    2.378701e+07   1.905237e+07   5.887400e+01          1       2484
┌ Warning: Attempting to recover from serious numerical issues...
└ @ SDDP C:\Users\Name\.julia\packages\SDDP\2Jypx\src\algorithm.jl:261
       70    2.377627e+07   1.916283e+07   5.976100e+01          1       2520
       71    2.377944e+07   1.916283e+07   6.047800e+01          1       2556
       72    2.375634e+07   1.916597e+07   6.116400e+01          1       2592
       73    2.376534e+07   1.918088e+07   6.176400e+01          1       2628
       74    2.379649e+07   1.920615e+07   6.301800e+01          1       2664
       75    2.367808e+07   1.920735e+07   6.458200e+01          1       2700
       76    2.382994e+07   1.921550e+07   6.571100e+01          1       2736
       77    2.385098e+07   1.921987e+07   6.684300e+01          1       2772
       78    2.382292e+07   1.922295e+07   6.790000e+01          1       2808
       79    2.386738e+07   1.922295e+07   6.890600e+01          1       2844
       80    2.386020e+07   1.922295e+07   7.000000e+01          1       2880
       81    2.384622e+07   1.922295e+07   7.097200e+01          1       2916
       82    2.379939e+07   1.922295e+07   7.205700e+01          1       2952
       83    2.373278e+07   1.922342e+07   7.348400e+01          1       2988
       84    2.385879e+07   1.922342e+07   7.514500e+01          1       3024
       85    2.375325e+07   1.922342e+07   7.694700e+01          1       3060
       86    2.372863e+07   1.922342e+07   7.868300e+01          1       3096
       87    2.372149e+07   1.922342e+07   8.027300e+01          1       3132
       88    2.378760e+07   1.922342e+07   8.174500e+01          1       3168
       89    2.378955e+07   1.922342e+07   8.308300e+01          1       3204
       90    2.372502e+07   1.922342e+07   8.456400e+01          1       3240
       91    2.373915e+07   1.922342e+07   8.628200e+01          1       3276
       92    2.379069e+07   1.922342e+07   8.788600e+01          1       3312
       93    2.371390e+07   1.922342e+07   8.949700e+01          1       3348
       94    2.376073e+07   1.922342e+07   9.115000e+01          1       3384
       95    2.375346e+07   1.922342e+07   9.289300e+01          1       3420
       96    2.382331e+07   1.922342e+07   9.467800e+01          1       3456
       97    2.378381e+07   1.922342e+07   9.660400e+01          1       3492
       98    2.375754e+07   1.922342e+07   9.847500e+01          1       3528
       99    2.369469e+07   1.922342e+07   1.003270e+02          1       3564
      100    2.375547e+07   1.922342e+07   1.023800e+02          1       3600
      101    2.375027e+07   1.922342e+07   1.044110e+02          1       3636
      102    2.373522e+07   1.922342e+07   1.063690e+02          1       3672
      103    2.377170e+07   1.922342e+07   1.084570e+02          1       3708
      104    2.377237e+07   1.922342e+07   1.104880e+02          1       3744
      105    2.373138e+07   1.922342e+07   1.126480e+02          1       3780
      106    2.376941e+07   1.922342e+07   1.143600e+02          1       3816
      107    2.377290e+07   1.922342e+07   1.161670e+02          1       3852
      108    2.375295e+07   1.922342e+07   1.180750e+02          1       3888
      109    2.367533e+07   1.922342e+07   1.203500e+02          1       3924
      110    2.374732e+07   1.922342e+07   1.226000e+02          1       3960
      111    2.374357e+07   1.922342e+07   1.248590e+02          1       3996
      112    2.373807e+07   1.922342e+07   1.271090e+02          1       4032
      113    2.374698e+07   1.922342e+07   1.290360e+02          1       4068
      114    2.376057e+07   1.922342e+07   1.311840e+02          1       4104
      115    2.373118e+07   1.922342e+07   1.332340e+02          1       4140
      116    2.375652e+07   1.922342e+07   1.354800e+02          1       4176
      117    2.373076e+07   1.922342e+07   1.377500e+02          1       4212
      118    2.376347e+07   1.922342e+07   1.402300e+02          1       4248
      119    2.373451e+07   1.922342e+07   1.425990e+02          1       4284
      120    2.376065e+07   1.922342e+07   1.447790e+02          1       4320
      121    2.371670e+07   1.922342e+07   1.468060e+02          1       4356
      122    2.371951e+07   1.922342e+07   1.487300e+02          1       4392
      123    2.374190e+07   1.922342e+07   1.507150e+02          1       4428
      124    2.374187e+07   1.922342e+07   1.524000e+02          1       4464
      125    2.373138e+07   1.922342e+07   1.540940e+02          1       4500
      126    2.376232e+07   1.922342e+07   1.558370e+02          1       4536
      127    2.376534e+07   1.922342e+07   1.578070e+02          1       4572
      128    2.374170e+07   1.922342e+07   1.593700e+02          1       4608
      129    2.376245e+07   1.922342e+07   1.619670e+02          1       4644
      130    2.374472e+07   1.922342e+07   1.641130e+02          1       4680
      131    2.376722e+07   1.922342e+07   1.660690e+02          1       4716
      132    2.372951e+07   1.922342e+07   1.678710e+02          1       4752
      133    2.374263e+07   1.922342e+07   1.698930e+02          1       4788
      134    2.370568e+07   1.922342e+07   1.716610e+02          1       4824
      135    2.375617e+07   1.922342e+07   1.733340e+02          1       4860
      136    2.374378e+07   1.922342e+07   1.752230e+02          1       4896
      137    2.372232e+07   1.922342e+07   1.770410e+02          1       4932
      138    2.374378e+07   1.922342e+07   1.789260e+02          1       4968
      139    2.370933e+07   1.922342e+07   1.806000e+02          1       5004
      140    2.373795e+07   1.922342e+07   1.824200e+02          1       5040
      141    2.377745e+07   1.922342e+07   1.842090e+02          1       5076
      142    2.372670e+07   1.922342e+07   1.862240e+02          1       5112
      143    2.374451e+07   1.922342e+07   1.881330e+02          1       5148
      144    2.376253e+07   1.922342e+07   1.900690e+02          1       5184
      145    2.374003e+07   1.922342e+07   1.918620e+02          1       5220
      146    2.371138e+07   1.922342e+07   1.938020e+02          1       5256
      147    2.372138e+07   1.922342e+07   1.957200e+02          1       5292
      148    2.371659e+07   1.922342e+07   1.978100e+02          1       5328
      149    2.373701e+07   1.922342e+07   1.997490e+02          1       5364
      150    2.372670e+07   1.922342e+07   2.016780e+02          1       5400
      151    2.375034e+07   1.922342e+07   2.035280e+02          1       5436
      152    2.372422e+07   1.922342e+07   2.051320e+02          1       5472
      153    2.374463e+07   1.922342e+07   2.067870e+02          1       5508
      154    2.371484e+07   1.922342e+07   2.084480e+02          1       5544
      155    2.378682e+07   1.922342e+07   2.105070e+02          1       5580
      156    2.376440e+07   1.922342e+07   2.123900e+02          1       5616
      157    2.375597e+07   1.922342e+07   2.142590e+02          1       5652
      158    2.375307e+07   1.922342e+07   2.163280e+02          1       5688
      159    2.375034e+07   1.922342e+07   2.184170e+02          1       5724
      160    2.375315e+07   1.922342e+07   2.204330e+02          1       5760
      161    2.371659e+07   1.922342e+07   2.223360e+02          1       5796
      162    2.377527e+07   1.922342e+07   2.241510e+02          1       5832
      163    2.375558e+07   1.922342e+07   2.260100e+02          1       5868
      164    2.372891e+07   1.922342e+07   2.277260e+02          1       5904
      165    2.373815e+07   1.922342e+07   2.297320e+02          1       5940
      166    2.376807e+07   1.922342e+07   2.316090e+02          1       5976
      167    2.376159e+07   1.922342e+07   2.335280e+02          1       6012
      168    2.373088e+07   1.922342e+07   2.353260e+02          1       6048
      169    2.373463e+07   1.922342e+07   2.369820e+02          1       6084
      170    2.377336e+07   1.922342e+07   2.387780e+02          1       6120
      171    2.364993e+07   1.922342e+07   2.404300e+02          1       6156
      172    2.374545e+07   1.922342e+07   2.422480e+02          1       6192
      173    2.371109e+07   1.922342e+07   2.439060e+02          1       6228
      174    2.371859e+07   1.922342e+07   2.454920e+02          1       6264
      175    2.376440e+07   1.922342e+07   2.474290e+02          1       6300
      176    2.370225e+07   1.922342e+07   2.492160e+02          1       6336
      177    2.377545e+07   1.922342e+07   2.509990e+02          1       6372
      178    2.373982e+07   1.922342e+07   2.528630e+02          1       6408
      179    2.374097e+07   1.922342e+07   2.547000e+02          1       6444
      180    2.376982e+07   1.922342e+07   2.565330e+02          1       6480
      181    2.371670e+07   1.922342e+07   2.584060e+02          1       6516
      182    2.373170e+07   1.922342e+07   2.602030e+02          1       6552
      183    2.375597e+07   1.922342e+07   2.621050e+02          1       6588
      184    2.375409e+07   1.922342e+07   2.642670e+02          1       6624
      185    2.373982e+07   1.922342e+07   2.670500e+02          1       6660
      186    2.378120e+07   1.922342e+07   2.687830e+02          1       6696
      187    2.372795e+07   1.922342e+07   2.710070e+02          1       6732
      188    2.373253e+07   1.922342e+07   2.728900e+02          1       6768
      189    2.372013e+07   1.922342e+07   2.749370e+02          1       6804
      190    2.373513e+07   1.922342e+07   2.767960e+02          1       6840
      191    2.369865e+07   1.922342e+07   2.786180e+02          1       6876
      192    2.375597e+07   1.922342e+07   2.803500e+02          1       6912
      193    2.369671e+07   1.922342e+07   2.822520e+02          1       6948
      194    2.374753e+07   1.922342e+07   2.839390e+02          1       6984
      195    2.375315e+07   1.922342e+07   2.858020e+02          1       7020
      196    2.375222e+07   1.922342e+07   2.875080e+02          1       7056
      197    2.374472e+07   1.922342e+07   2.893620e+02          1       7092
      198    2.369859e+07   1.922342e+07   2.910870e+02          1       7128
      199    2.369656e+07   1.922342e+07   2.929720e+02          1       7164
      200    2.374097e+07   1.922342e+07   2.948530e+02          1       7200

Terminating training
  Status         : iteration_limit
  Total time (s) : 2.948530e+02
  Total solves   : 7200
  Best bound     :  1.922342e+07
  Simulation CI  :  2.377260e+07 ± 7.222065e+03
------------------------------------------------------------------------------
@WuSiren
Copy link
Author

WuSiren commented Jun 26, 2024

  1. I trained it again with iteration_limit = 250, but the output log and result are quite confusing: there is no Warning: Attempting to recover from serious numerical issues... this time, and the resulting Bound is less than that with iteration_limit = 200, how come?
------------------------------------------------------------------------------
          SDDP.jl (c) Oscar Dowson and SDDP.jl contributors, 2017-23

Problem
  Nodes           : 12
  State variables : 299
  Scenarios       : 4.09600e+03
  Existing cuts   : false
  Subproblem structure                      : (min, max)
    Variables                               : (994, 994)
    VariableRef in MOI.LessThan{Float64}    : (1, 1)
    VariableRef in MOI.ZeroOne              : (320, 320)
    AffExpr in MOI.LessThan{Float64}        : (407, 407)
    AffExpr in MOI.GreaterThan{Float64}     : (272, 272)
    AffExpr in MOI.EqualTo{Float64}         : (354, 354)
    VariableRef in MOI.GreaterThan{Float64} : (360, 360)
Options
  Solver          : serial mode
  Risk measure    : SDDP.Expectation()
  Sampling scheme : SDDP.InSampleMonteCarlo

Numerical stability report
  Non-zero Matrix range     [5e-05, 3e+03]
  Non-zero Objective range  [1e+00, 3e+04]
  Non-zero Bounds range     [0e+00, 0e+00]
  Non-zero RHS range        [1e+00, 1e+04]
WARNING: numerical stability issues detected       
  - Matrix range contains small coefficients       
Very large or small absolute values of coefficients
can cause numerical stability issues. Consider     
reformulating the model.

 Iteration    Simulation       Bound         Time (s)    Proc. ID   # Solves
        1    2.369787e+07   1.876826e+06   7.319999e-01          1         36
        2    2.379377e+07   2.098343e+06   1.142000e+00          1         72
        3    2.379640e+07   2.100168e+06   1.471000e+00          1        108
        4    2.382735e+07   3.871412e+06   1.796000e+00          1        144
        5    2.390147e+07   3.871412e+06   2.274000e+00          1        180
        6    2.377783e+07   3.873307e+06   2.757000e+00          1        216
        7    2.388537e+07   4.329638e+06   3.359000e+00          1        252
        8    2.372340e+07   4.329638e+06   4.078000e+00          1        288
        9    2.382303e+07   4.394544e+06   4.616000e+00          1        324
       10    2.389730e+07   4.394544e+06   5.022000e+00          1        360
       11    2.386338e+07   4.406783e+06   5.437000e+00          1        396
       12    2.390824e+07   4.417413e+06   6.102000e+00          1        432
       13    2.382103e+07   4.434491e+06   6.882000e+00          1        468
       14    2.390602e+07   4.437003e+06   7.642000e+00          1        504
       15    2.381047e+07   4.443378e+06   8.783000e+00          1        540
       16    2.386565e+07   4.445118e+06   9.566000e+00          1        576
       17    2.386576e+07   4.448028e+06   1.074300e+01          1        612
       18    2.390563e+07   4.452878e+06   1.192700e+01          1        648
       19    2.381704e+07   4.452984e+06   1.376200e+01          1        684
       20    2.388876e+07   4.452984e+06   1.519900e+01          1        720
       21    2.388382e+07   4.452984e+06   1.670900e+01          1        756
       22    2.390334e+07   4.452984e+06   1.831500e+01          1        792
       23    2.383347e+07   4.452984e+06   1.999300e+01          1        828
       24    2.391432e+07   4.452984e+06   2.169400e+01          1        864
       25    2.384185e+07   4.452984e+06   2.345500e+01          1        900
       26    2.379932e+07   4.452984e+06   2.523500e+01          1        936
       27    2.385432e+07   4.452984e+06   2.695400e+01          1        972
       28    2.379137e+07   4.452984e+06   2.905500e+01          1       1008
       29    2.378066e+07   4.452984e+06   3.115000e+01          1       1044
       30    2.380021e+07   4.452984e+06   3.309400e+01          1       1080
       31    2.376432e+07   4.452984e+06   3.521000e+01          1       1116
       32    2.378494e+07   4.452984e+06   3.726500e+01          1       1152
       33    2.379901e+07   4.452984e+06   3.950600e+01          1       1188
       34    2.377463e+07   4.452984e+06   4.154200e+01          1       1224
       35    2.382057e+07   4.452984e+06   4.364400e+01          1       1260
       36    2.377932e+07   4.452984e+06   4.569000e+01          1       1296
       37    2.379901e+07   4.452984e+06   4.771200e+01          1       1332
       38    2.379713e+07   4.452984e+06   4.992600e+01          1       1368
       39    2.377838e+07   4.452984e+06   5.207400e+01          1       1404
       40    2.379901e+07   4.452984e+06   5.474100e+01          1       1440
       41    2.375307e+07   4.452984e+06   5.680000e+01          1       1476
       42    2.378119e+07   4.452984e+06   5.891700e+01          1       1512
       43    2.375682e+07   4.452984e+06   6.109500e+01          1       1548
       44    2.373016e+07   4.452984e+06   6.308300e+01          1       1584
       45    2.377276e+07   4.452984e+06   6.509300e+01          1       1620
       46    2.376151e+07   4.452984e+06   6.727400e+01          1       1656
       47    2.381494e+07   4.452984e+06   6.939500e+01          1       1692
       48    2.379338e+07   4.452984e+06   7.147600e+01          1       1728
       49    2.380932e+07   4.452984e+06   7.369100e+01          1       1764
       50    2.375119e+07   4.452984e+06   7.590100e+01          1       1800
       51    2.379151e+07   4.452984e+06   7.799300e+01          1       1836
       52    2.382151e+07   4.452984e+06   8.018100e+01          1       1872
       53    2.376526e+07   4.452984e+06   8.221600e+01          1       1908
       54    2.376807e+07   4.452984e+06   8.441600e+01          1       1944
       55    2.374932e+07   4.452984e+06   8.649800e+01          1       1980
       56    2.377557e+07   4.452984e+06   8.841400e+01          1       2016
       57    2.376526e+07   4.452984e+06   9.047800e+01          1       2052
       58    2.377838e+07   4.452984e+06   9.260800e+01          1       2088
       59    2.376338e+07   4.452984e+06   9.506100e+01          1       2124
       60    2.377182e+07   4.452984e+06   9.737500e+01          1       2160
       61    2.381026e+07   4.452984e+06   9.964200e+01          1       2196
       62    2.376713e+07   4.452984e+06   1.030800e+02          1       2232
       63    2.379526e+07   4.452984e+06   1.059680e+02          1       2268
       64    2.378119e+07   4.452984e+06   1.085710e+02          1       2304
       65    2.380088e+07   4.452984e+06   1.111690e+02          1       2340
       66    2.378588e+07   4.452984e+06   1.137830e+02          1       2376
       67    2.376244e+07   4.452984e+06   1.161290e+02          1       2412
       68    2.375776e+07   4.452984e+06   1.186120e+02          1       2448
       69    2.379619e+07   4.452984e+06   1.209730e+02          1       2484
       70    2.376994e+07   4.452984e+06   1.233290e+02          1       2520
       71    2.379151e+07   4.452984e+06   1.257700e+02          1       2556
       72    2.374276e+07   4.452984e+06   1.281480e+02          1       2592
       73    2.380838e+07   4.452984e+06   1.304790e+02          1       2628
       74    2.375682e+07   4.452984e+06   1.322880e+02          1       2664
       75    2.378588e+07   4.452984e+06   1.343340e+02          1       2700
       76    2.380276e+07   4.452984e+06   1.363550e+02          1       2736
       77    2.377838e+07   4.452984e+06   1.380870e+02          1       2772
       78    2.375682e+07   4.452984e+06   1.399790e+02          1       2808
       79    2.380463e+07   4.452984e+06   1.421050e+02          1       2844
       80    2.378869e+07   4.452984e+06   1.440720e+02          1       2880
       81    2.376807e+07   4.452984e+06   1.461430e+02          1       2916
       82    2.380088e+07   4.452984e+06   1.481140e+02          1       2952
       83    2.380088e+07   4.452984e+06   1.501580e+02          1       2988
       84    2.374651e+07   4.452984e+06   1.522530e+02          1       3024
       85    2.381869e+07   4.452984e+06   1.545020e+02          1       3060
       86    2.376901e+07   4.452984e+06   1.563910e+02          1       3096
       87    2.380619e+07   4.452984e+06   1.585480e+02          1       3132
       88    2.376901e+07   4.452984e+06   1.608610e+02          1       3168
       89    2.372637e+07   4.452984e+06   1.627510e+02          1       3204
       90    2.373574e+07   4.452984e+06   1.644520e+02          1       3240
       91    2.379901e+07   4.452984e+06   1.665710e+02          1       3276
       92    2.379151e+07   4.452984e+06   1.687620e+02          1       3312
       93    2.377557e+07   4.452984e+06   1.709910e+02          1       3348
       94    2.378026e+07   4.452984e+06   1.731310e+02          1       3384
       95    2.381307e+07   4.452984e+06   1.751680e+02          1       3420
       96    2.376151e+07   4.452984e+06   1.778440e+02          1       3456
       97    2.377088e+07   4.452984e+06   1.809620e+02          1       3492
       98    2.377463e+07   4.452984e+06   1.829470e+02          1       3528
       99    2.375119e+07   4.452984e+06   1.850930e+02          1       3564
      100    2.373901e+07   4.452984e+06   1.870780e+02          1       3600
      101    2.381119e+07   4.452984e+06   1.890530e+02          1       3636
      102    2.377932e+07   4.452984e+06   1.911410e+02          1       3672
      103    2.380369e+07   4.452984e+06   1.931900e+02          1       3708
      104    2.376151e+07   4.452984e+06   1.951260e+02          1       3744
      105    2.375682e+07   4.452984e+06   1.970500e+02          1       3780
      106    2.374651e+07   4.452984e+06   1.989940e+02          1       3816
      107    2.380737e+07   4.452984e+06   2.010660e+02          1       3852
      108    2.376057e+07   4.452984e+06   2.029790e+02          1       3888
      109    2.380182e+07   4.452984e+06   2.048830e+02          1       3924
      110    2.374369e+07   4.452984e+06   2.069070e+02          1       3960
      111    2.376619e+07   4.452984e+06   2.089280e+02          1       3996
      112    2.379338e+07   4.452984e+06   2.111120e+02          1       4032
      113    2.381026e+07   4.452984e+06   2.133710e+02          1       4068
      114    2.375307e+07   4.452984e+06   2.153910e+02          1       4104
      115    2.374838e+07   4.452984e+06   2.173510e+02          1       4140
      116    2.382151e+07   4.452984e+06   2.195500e+02          1       4176
      117    2.380838e+07   4.452984e+06   2.218220e+02          1       4212
      118    2.378869e+07   4.452984e+06   2.239920e+02          1       4248
      119    2.378682e+07   4.452984e+06   2.258730e+02          1       4284
      120    2.375776e+07   4.452984e+06   2.279980e+02          1       4320
      121    2.378776e+07   4.452984e+06   2.300910e+02          1       4356
      122    2.378869e+07   4.452984e+06   2.322880e+02          1       4392
      123    2.378119e+07   4.452984e+06   2.345310e+02          1       4428
      124    2.378026e+07   4.452984e+06   2.366490e+02          1       4464
      125    2.381401e+07   4.452984e+06   2.388910e+02          1       4500
      126    2.375588e+07   4.452984e+06   2.409240e+02          1       4536
      127    2.375682e+07   4.452984e+06   2.428120e+02          1       4572
      128    2.378682e+07   4.452984e+06   2.448350e+02          1       4608
      129    2.378963e+07   4.452984e+06   2.468080e+02          1       4644
      130    2.377838e+07   4.452984e+06   2.487290e+02          1       4680
      131    2.382331e+07   4.452984e+06   2.503470e+02          1       4716
      132    2.378581e+07   4.452984e+06   2.520430e+02          1       4752
      133    2.377268e+07   4.452984e+06   2.537030e+02          1       4788
      134    2.379901e+07   4.452984e+06   2.556780e+02          1       4824
      135    2.376799e+07   4.452984e+06   2.575130e+02          1       4860
      136    2.378768e+07   4.452984e+06   2.593420e+02          1       4896
      137    2.373893e+07   4.452984e+06   2.612500e+02          1       4932
      138    2.376612e+07   4.452984e+06   2.631060e+02          1       4968
      139    2.377362e+07   4.452984e+06   2.648490e+02          1       5004
      140    2.376987e+07   4.452984e+06   2.667090e+02          1       5040
      141    2.374456e+07   4.452984e+06   2.686440e+02          1       5076
      142    2.378206e+07   4.452984e+06   2.705360e+02          1       5112
      143    2.379518e+07   4.452984e+06   2.724490e+02          1       5148
      144    2.379612e+07   4.452984e+06   2.743310e+02          1       5184
      145    2.379612e+07   4.452984e+06   2.761670e+02          1       5220
      146    2.379143e+07   4.452984e+06   2.783910e+02          1       5256
      147    2.380831e+07   4.452984e+06   2.806920e+02          1       5292
      148    2.377456e+07   4.452984e+06   2.827930e+02          1       5328
      149    2.378299e+07   4.452984e+06   2.847130e+02          1       5364
      150    2.377174e+07   4.452984e+06   2.866880e+02          1       5400
      151    2.376432e+07   4.452984e+06   2.889590e+02          1       5436
      152    2.380362e+07   4.452984e+06   2.908740e+02          1       5472
      153    2.377643e+07   4.452984e+06   2.927800e+02          1       5508
      154    2.378018e+07   4.452984e+06   2.946370e+02          1       5544
      155    2.372236e+07   4.452984e+06   2.964630e+02          1       5580
      156    2.377737e+07   4.452984e+06   2.982070e+02          1       5616
      157    2.378768e+07   4.452984e+06   3.000260e+02          1       5652
      158    2.376049e+07   4.452984e+06   3.018050e+02          1       5688
      159    2.373927e+07   4.452984e+06   3.037880e+02          1       5724
      160    2.378206e+07   4.452984e+06   3.055250e+02          1       5760
      161    2.380549e+07   4.452984e+06   3.072750e+02          1       5796
      162    2.379237e+07   4.452984e+06   3.090890e+02          1       5832
      163    2.379987e+07   4.452984e+06   3.110360e+02          1       5868
      164    2.382424e+07   4.452984e+06   3.129230e+02          1       5904
      165    2.377643e+07   4.452984e+06   3.148640e+02          1       5940
      166    2.377549e+07   4.452984e+06   3.168860e+02          1       5976
      167    2.380276e+07   4.452984e+06   3.189340e+02          1       6012
      168    2.378018e+07   4.452984e+06   3.209080e+02          1       6048
      169    2.380643e+07   4.452984e+06   3.227530e+02          1       6084
      170    2.377174e+07   4.452984e+06   3.246110e+02          1       6120
      171    2.376049e+07   4.452984e+06   3.268860e+02          1       6156
      172    2.375206e+07   4.452984e+06   3.285570e+02          1       6192
      173    2.378018e+07   4.452984e+06   3.304490e+02          1       6228
      174    2.376987e+07   4.452984e+06   3.322020e+02          1       6264
      175    2.380268e+07   4.452984e+06   3.339360e+02          1       6300
      176    2.378768e+07   4.452984e+06   3.357250e+02          1       6336
      177    2.375862e+07   4.452984e+06   3.375990e+02          1       6372
      178    2.381768e+07   4.452984e+06   3.394200e+02          1       6408
      179    2.376331e+07   4.452984e+06   3.412640e+02          1       6444
      180    2.376049e+07   4.452984e+06   3.431570e+02          1       6480
      181    2.382151e+07   4.452984e+06   3.450270e+02          1       6516
      182    2.380557e+07   4.452984e+06   3.478820e+02          1       6552
      183    2.377369e+07   4.452984e+06   3.498570e+02          1       6588
      184    2.374924e+07   4.452984e+06   3.516580e+02          1       6624
      185    2.377088e+07   4.452984e+06   3.536170e+02          1       6660
      186    2.378018e+07   4.452984e+06   3.555170e+02          1       6696
      187    2.379057e+07   4.452984e+06   3.574500e+02          1       6732
      188    2.381956e+07   4.452984e+06   3.593050e+02          1       6768
      189    2.382424e+07   4.452984e+06   3.611180e+02          1       6804
      190    2.377456e+07   4.452984e+06   3.630090e+02          1       6840
      191    2.380369e+07   4.452984e+06   3.649170e+02          1       6876
      192    2.379706e+07   4.452984e+06   3.668020e+02          1       6912
      193    2.378588e+07   4.452984e+06   3.686650e+02          1       6948
      194    2.376799e+07   4.452984e+06   3.703680e+02          1       6984
      195    2.381299e+07   4.452984e+06   3.721670e+02          1       7020
      196    2.378674e+07   4.452984e+06   3.738970e+02          1       7056
      197    2.379893e+07   4.452984e+06   3.757200e+02          1       7092
      198    2.381862e+07   4.452984e+06   3.775930e+02          1       7128
      199    2.375112e+07   4.452984e+06   3.794940e+02          1       7164
      200    2.381299e+07   4.452984e+06   3.815080e+02          1       7200
      201    2.378862e+07   4.452984e+06   3.834440e+02          1       7236
      202    2.374362e+07   4.452984e+06   3.852670e+02          1       7272
      203    2.376901e+07   4.452984e+06   3.873270e+02          1       7308
      204    2.377268e+07   4.452984e+06   3.890710e+02          1       7344
      205    2.378299e+07   4.452984e+06   3.909840e+02          1       7380
      206    2.371703e+07   4.452984e+06   3.928960e+02          1       7416
      207    2.380924e+07   4.452984e+06   3.946760e+02          1       7452
      208    2.378768e+07   4.452984e+06   3.965790e+02          1       7488
      209    2.380831e+07   4.452984e+06   3.986590e+02          1       7524
      210    2.379526e+07   4.452984e+06   4.005400e+02          1       7560
      211    2.381682e+07   4.452984e+06   4.025520e+02          1       7596
      212    2.373807e+07   4.452984e+06   4.044770e+02          1       7632
      213    2.377737e+07   4.452984e+06   4.062890e+02          1       7668
      214    2.381112e+07   4.452984e+06   4.082820e+02          1       7704
      215    2.377831e+07   4.452984e+06   4.101250e+02          1       7740
      216    2.377549e+07   4.452984e+06   4.121200e+02          1       7776
      217    2.376518e+07   4.452984e+06   4.139790e+02          1       7812
      218    2.378112e+07   4.452984e+06   4.158620e+02          1       7848
      219    2.380838e+07   4.452984e+06   4.178840e+02          1       7884
      220    2.378487e+07   4.452984e+06   4.198890e+02          1       7920
      221    2.378487e+07   4.452984e+06   4.217020e+02          1       7956
      222    2.378213e+07   4.452984e+06   4.236990e+02          1       7992
      223    2.380276e+07   4.452984e+06   4.256580e+02          1       8028
      224    2.374234e+07   4.452984e+06   4.276140e+02          1       8064
      225    2.380932e+07   4.452984e+06   4.296180e+02          1       8100
      226    2.379244e+07   4.452984e+06   4.317130e+02          1       8136
      227    2.376143e+07   4.452984e+06   4.335540e+02          1       8172
      228    2.377088e+07   4.452984e+06   4.355240e+02          1       8208
      229    2.378487e+07   4.452984e+06   4.373640e+02          1       8244
      230    2.377081e+07   4.452984e+06   4.393480e+02          1       8280
      231    2.378682e+07   4.452984e+06   4.413320e+02          1       8316
      232    2.378206e+07   4.452984e+06   4.432710e+02          1       8352
      233    2.374182e+07   4.452984e+06   4.452340e+02          1       8388
      234    2.378206e+07   4.452984e+06   4.470540e+02          1       8424
      235    2.374643e+07   4.452984e+06   4.488870e+02          1       8460
      236    2.379237e+07   4.452984e+06   4.507490e+02          1       8496
      237    2.380831e+07   4.452984e+06   4.527240e+02          1       8532
      238    2.376893e+07   4.452984e+06   4.547580e+02          1       8568
      239    2.379049e+07   4.452984e+06   4.567510e+02          1       8604
      240    2.382619e+07   4.452984e+06   4.587780e+02          1       8640
      241    2.379143e+07   4.452984e+06   4.606410e+02          1       8676
      242    2.378393e+07   4.452984e+06   4.627240e+02          1       8712
      243    2.379237e+07   4.452984e+06   4.647980e+02          1       8748
      244    2.379237e+07   4.452984e+06   4.667840e+02          1       8784
      245    2.375674e+07   4.452984e+06   4.690750e+02          1       8820
      246    2.379338e+07   4.452984e+06   4.712540e+02          1       8856
      247    2.378018e+07   4.452984e+06   4.732150e+02          1       8892
      248    2.381768e+07   4.452984e+06   4.754760e+02          1       8928
      249    2.380088e+07   4.452984e+06   4.773950e+02          1       8964
      250    2.377737e+07   4.452984e+06   4.795650e+02          1       9000

Terminating training
  Status         : iteration_limit
  Total time (s) : 4.795650e+02
  Total solves   : 9000
  Best bound     :  4.452984e+06
  Simulation CI  :  2.378839e+07 ± 4.238939e+03
------------------------------------------------------------------------------

@odow
Copy link
Owner

odow commented Jun 26, 2024

  1. See https://sddp.dev/stable/tutorial/markov_uncertainty/ You can use the index of the node to chooses a different sample space or probability vector, or in the SDDP.parameterize function you can have if t == 1 #=do something=# else #=do something else=# end
  2. Yes. But as with all code, you should always debug your model on small instances, plot the solution with a spaghetti plot, etc. You should always assume that there is the risk of a bug (in your code, SDDP.jl, or the solver), and never assume that just because you used a library that it is "correct".
  3. So I ignored it sure, you can ignore it. It is a warning, not an error, but it often means that there is a "better" way to reformulate your problem, or that you expect too much precision. I like to think of 5e-5 as "does a millionaire know when they spend $50?"
  4. It looks like you used a different random seed between the runs. Since there are VariableRef in MOI.ZeroOne constraints, you have a mixed-integer problem. SDDP.jl is not guaranteed to find a global solution. It may find only a locally feasible policy.

You should also try updating to the latest version of SDDP.jl. I improved a few numerical things, so the warning shouldn't appear like that in the most recent version.

@WuSiren
Copy link
Author

WuSiren commented Jun 27, 2024 •
edited
Loading

Thank you, Professor! Every time I see your reply, I realize how stupid my questions are! Please forgive me!

BTW, now my SDDP is v1.1.2. Is it the latest version? I found the latest version here is v1.7.0 but I was not able to update to it throgh ] up SDDP.

@odow
Copy link
Owner

odow commented Jun 27, 2024

Try ] add [email protected]. You probably have some other packages that are holding you back.

(We've changed 106 things since 1.1.2! v1.1.2...v1.7.0)

@WuSiren
Copy link
Author

WuSiren commented Jun 27, 2024 •
edited
Loading

But there even isn't an update symbol ^, why?

(test) pkg> st
Status `D:\test\Project.toml`
  [87dc4568] HiGHS v1.5.0
  [4076af6c] JuMP v1.9.0
  [f4570300] SDDP v1.1.2

This is the result after I run ] add [email protected]:

(test) pkg>  add SDDP@1.7
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package SDDP [f4570300]:
 SDDP [f4570300] log:
 ├─possible versions are: 0.1.0-1.1.2 or uninstalled
 └─restricted to versions 1.7 by an explicit requirement — no versions left

@odow
Copy link
Owner

odow commented Jun 27, 2024

] update? How did you install Julia? It doesn't appear that you have access to the latest version of the General registry.

@WuSiren
Copy link
Author

WuSiren commented Jun 27, 2024

It's solved! Thank you! 🤝

@WuSiren WuSiren closed this as completed Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@odow @WuSiren