Error when trying to train a policy

[SerenaZwww]

I think I have a model now, because when I run:

model = SDDP.LinearPolicyGraph(
    stages = 3,
    sense = :Min,
    lower_bound = 0.0,
    optimizer = HiGHS.Optimizer,
) do subproblem, node
...

I get:

A policy graph with 3 nodes.
 Node indices: 1, 2, 3

Now I'm trying to train a policy. There's an error message saying "i not defined". I don't know where the error is.

------------------------------------------------------------------------------
                      SDDP.jl (c) Oscar Dowson, 2017-21

Problem
  Nodes           : 3
  State variables : 126
  Scenarios       : 8.00000e+03
  Existing cuts   : false
  Subproblem structure                      : (min, max)
    Variables                               : (883, 883)
    VariableRef in MOI.LessThan{Float64}    : (378, 379)
    VariableRef in MOI.ZeroOne              : (252, 252)
    AffExpr in MOI.LessThan{Float64}        : (414, 414)
    QuadExpr in MOI.EqualTo{Float64}        : (126, 126)
    AffExpr in MOI.EqualTo{Float64}         : (182, 182)
    VariableRef in MOI.GreaterThan{Float64} : (379, 379)
Options
  Solver          : serial mode
  Risk measure    : SDDP.Expectation()
  Sampling scheme : SDDP.InSampleMonteCarlo

UndefVarError: i not defined

Stacktrace:
 [1] (::var"#186#205"{JuMP.Containers.DenseAxisArray{VariableRef, 4, NTuple{4, Vector{Int64}}, NTuple{4, JuMP.Containers._AxisLookup{Dict{Int64, Int64}}}}})(ω::JuMP.Containers.DenseAxisArray{DiscreteNonParametric{Int64, Float64, UnitRange{Int64}, Vector{Float64}}, 3, Tuple{Vector{Int64}, Vector{Int64}, Vector{Int64}}, Tuple{JuMP.Containers._AxisLookup{Dict{Int64, Int64}}, JuMP.Containers._AxisLookup{Dict{Int64, Int64}}, JuMP.Containers._AxisLookup{Dict{Int64, Int64}}}})
   @ Main .\In[13]:74
 [2] parameterize(node::SDDP.Node{Int64}, noise::JuMP.Containers.DenseAxisArray{DiscreteNonParametric{Int64, Float64, UnitRange{Int64}, Vector{Float64}}, 3, Tuple{Vector{Int64}, Vector{Int64}, Vector{Int64}}, Tuple{JuMP.Containers._AxisLookup{Dict{Int64, Int64}}, JuMP.Containers._AxisLookup{Dict{Int64, Int64}}, JuMP.Containers._AxisLookup{Dict{Int64, Int64}}}})
   @ SDDP C:\Users\Serena\.julia\packages\SDDP\VpZOu\src\algorithm.jl:255
 [3] numerical_stability_report(io::IOBuffer, model::SDDP.PolicyGraph{Int64}; by_node::Bool, print::Bool, warn::Bool)
   @ SDDP C:\Users\Serena\.julia\packages\SDDP\VpZOu\src\print.jl:366
 [4] (::SDDP.var"#70#74"{Int64, SDDP.PolicyGraph{Int64}})(io::IOBuffer)
   @ SDDP C:\Users\Serena\.julia\packages\SDDP\VpZOu\src\algorithm.jl:973
 [5] sprint(::Function; context::Nothing, sizehint::Int64)
   @ Base .\strings\io.jl:114
 [6] sprint
   @ .\strings\io.jl:108 [inlined]
 [7] train(model::SDDP.PolicyGraph{Int64}; iteration_limit::Int64, time_limit::Nothing, print_level::Int64, log_file::String, log_frequency::Int64, run_numerical_stability_report::Bool, stopping_rules::Vector{SDDP.AbstractStoppingRule}, risk_measure::SDDP.Expectation, sampling_scheme::SDDP.InSampleMonteCarlo, cut_type::SDDP.CutType, cycle_discretization_delta::Float64, refine_at_similar_nodes::Bool, cut_deletion_minimum::Int64, backward_sampling_scheme::SDDP.CompleteSampler, dashboard::Bool, parallel_scheme::SDDP.Serial, forward_pass::SDDP.DefaultForwardPass, forward_pass_resampling_probability::Nothing, add_to_existing_cuts::Bool, duality_handler::SDDP.ContinuousConicDuality, forward_pass_callback::SDDP.var"#73#77")
   @ SDDP C:\Users\Serena\.julia\packages\SDDP\VpZOu\src\algorithm.jl:972
 [8] top-level scope
   @ In[14]:1

[odow]

There is a bug in your parameterize call. It is on the 74th line. But that's not very helpful. So save this as a script and run include("my_script.jl") and it will give you a proper line number.

[SerenaZwww]

Thanks for your reply! I have modified something, and now I think there is a bug in JuMP.fix:

MethodError: no method matching fix(::JuMP.Containers.DenseAxisArray{VariableRef, 3, Tuple{Base.OneTo{Int64}, UnitRange{Int64}, UnitRange{Int64}}, Tuple{JuMP.Containers._AxisLookup{Base.OneTo{Int64}}, JuMP.Containers._AxisLookup{Tuple{Int64, Int64}}, JuMP.Containers._AxisLookup{Tuple{Int64, Int64}}}}, ::JuMP.Containers.DenseAxisArray{DiscreteNonParametric{Int64, Float64, UnitRange{Int64}, Vector{Float64}}, 3, Tuple{Base.OneTo{Int64}, UnitRange{Int64}, UnitRange{Int64}}, Tuple{JuMP.Containers._AxisLookup{Base.OneTo{Int64}}, JuMP.Containers._AxisLookup{Tuple{Int64, Int64}}, JuMP.Containers._AxisLookup{Tuple{Int64, Int64}}}})

And my code is:

prob = Vector{Vector{Vector{Float64}}}([
    [
        [0.7, 0.2, 0.1], 
        [0.9, 0.1], 
        [1.0]
    ],
    [
        [0.5, 0.4, 0.1], 
        [0.7, 0.3], 
        [1.0]
    ],
    [
        [0.1, 0.7, 0.2], 
        [0.5, 0.5], 
        [1.0]
    ]
])
@variable(subproblem, d[i in 1:14, b in 0:2, l in 0:2])
@variable(subproblem, u[i in 1:14, b in 0:2, l in 0:2, l_ in 0:2])
N = 20
Ω = [
    Containers.@container([i in 1:14, b in 0:2, l in 0:2], DiscreteNonParametric(l:2, prob[b + 1][l + 1]))
    for _ in 1:N
]
P = fill(1 / N, N)
SDDP.parameterize(subproblem, Ω, P) do ω
    JuMP.fix(d, ω)  # Note the fix.( 
    for i in 1:14, b in 0:2, l in 0:2, l_ in 0:2
        u[i, b, l, l_] =  d[i, b, l] == l_ ? 1 : 0
    end
    return
end

Note that the distribution which $d$ follows is not related to $i$. I don't know if the error happened here.

[odow]

The MethodError is a different error.

JuMP.fix(d, ω) # Note the fix.(

You need JuMP.fix.(d, ω)

See https://jump.dev/JuMP.jl/stable/tutorials/getting_started/getting_started_with_julia/#Broadcasting

[SerenaZwww]

Sorry, I didn't notice the comment. Now I change it to fix.( and another error occurred:

MethodError: no method matching fix(::VariableRef, ::DiscreteNonParametric{Int64, Float64, UnitRange{Int64}, Vector{Float64}})

[odow]

Now you are trying to fix a variable to a value of type DiscreteNonParametric{Int64, Float64, UnitRange{Int64}, Vector{Float64}}.

You must instead fix the variable to a Float64.

You probably want (I didn't test) a call to rand in:

Ω = [
    Containers.@container(
        [i in 1:14, b in 0:2, l in 0:2],
        rand(DiscreteNonParametric(l:2, prob[b + 1][l + 1]))
    ) for _ in 1:N
]

[SerenaZwww]

Thanks for your reply! A call to rand works! But now I have a new problem:

MethodError: Cannot `convert` an object of type Float64 to an object of type VariableRef

So I change

@variable(subproblem, u[i in 1:14, b in 0:2, l in 0:2, l_ in 0:2])

to

u = Array{Float64}(undef, 14, 3, 3, 3)

And the above error disappears. Yet there's some NaN in $u$, leading to error Invalid coefficient NaN on variable sa[1,0,0].. I think each element in $u$ is well-defined, and I can't figure out why. I've attached the complete code below.
code_20240217.zip

[odow]

I think here u[i, b, l, l_] = d[i, b, l] == l_ ? 1 : 0 you are also trying to fix a decision variable to a value?

You need instead fix(u[i, b, l, l_], d[i, b, l] == l_ ? 1 : 0)

[odow]

Closing because the discussion seems to have come to an end. Please comment below if you have other questions and I will re-open.