Trouble with (Named)Tuples

[cscherrer]

I'm getting some strange behavior with NamedTuples.

Because gradient returns a NamedTuple, it seems more convenient to work consistently in those terms than to constantly convert back and forth with a struct. So I have a loss(m,df) that takes a NamedTuple and a DataFrame, and return the loss.

Since this is a high-dimensional problem, I'm starting with something simple and (hopefully) cheap - a Newton-Raphson step along the gradient at each iteration.

Here's what I now have working:

g = gradient(m -> loss(m, X), m)[1]
line(t) = map((m,g) -> m - t*g, m, g)
ℓ(t) = loss(line(t), X)
dℓ(t) = (gradient(t) do t
    Zygote.forwarddiff(ℓ,t)
end)[1]
ddℓ(t) = (gradient(t) do t
    Zygote.forwarddiff(dℓ,t)[1]
end)[1]
m = line(-dℓ(0.0)/ddℓ(0.0))

But if I take out the forwarddiff, I get

julia> dℓ(t) = (gradient(t) do t
           ℓ(t) # Zygote.forwarddiff(ℓ,t)
       end)[1]
dℓ (generic function with 1 method)

julia> dℓ(0.0)
ERROR: Non-differentiable function getfield
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at /home/chad/.julia/packages/Zygote/VeaFW/src/compiler/interface2.jl:0 [inlined]
 [3] (::typeof(∂(getfield)))(::Array{Float64,1}) at /home/chad/.julia/packages/Zygote/VeaFW/src/compiler/interface2.jl:19
 [4] macro expansion at ./namedtuple.jl:182 [inlined]
 [5] map at ./namedtuple.jl:177 [inlined]
 [6] (::typeof(∂(map)))(::NamedTuple{(:μ, :employer, :title, :city),NTuple{4,Array{Float64,1}}}) at /home/chad/.julia/packages/Zygote/VeaFW/src/compiler/interface2.jl:0

(and a few more lines)

[MikeInnes]

Hopefully something simple, but we'll likely need the definition of loss to see what's happening here. If you can reduce the loss function to a single line of code or so that causes this, that'd be a big help.

FWIW, you can make the gradient of a struct be another struct if you want. We don't do this by default just because in general, the gradient of a struct is not necessarily the same struct (e.g. colour types).