Cannot take Jacobian of a function that returns a tuple

[dpsanders]

julia v0.5> h(xx) = ((x, y)=xx; (x^2 - y^2 - 1, y - x))
h (generic function with 1 method)

julia v0.5> ForwardDiff.jacobian(x->h(x), [1,2])
------ MethodError --------------------- Stacktrace (most recent call last)

 [1] — jacobian(::##21#22, ::Array{Int64,1}) at jacobian.jl:6

 [2] — jacobian at jacobian.jl:7 [inlined]

 [3] — vector_mode_jacobian(::##21#22, ::Array{Int64,1}, ::ForwardDiff.JacobianConfig{2,Int64,Array{ForwardDiff.Dual{2,Int64},1}}) at jacobian.jl:76

MethodError: no method matching similar(::Tuple{ForwardDiff.Dual{2,Int64},ForwardDiff.Dual{2,Int64}}, ::Type{Int64}, ::Int64, ::Int64)
Closest candidates are:
  similar{T}(::AbstractArray{T,N}, ::Type{T}, ::Union{AbstractUnitRange{T},Integer}...) at abstractarray.jl:436
  similar(::Any, ::Union{AbstractUnitRange{T},Integer}...) at abstractarray.jl:478
  similar(::BitArray{N}, ::Int64...) at bitarray.jl:377

[jrevels]

It's documented in the API that the output of the target function passed to ForwardDiff.jacobian must be <:AbstractArray. Here, we can just use StaticArrays to achieve this:

julia> using ForwardDiff, StaticArrays

julia> h(xx) = ((x, y)=xx; SVector(x^2 - y^2 - 1, y - x))
h (generic function with 1 method)

julia> h([1,2])
2-element StaticArrays.SVector{2,Int64}:
 -4
  1

julia> ForwardDiff.jacobian(h, [1, 2])
2×2 Array{Int64,2}:
  2  -4
 -1   1

[dpsanders]

Thanks, I seem to have missed that in the docs.
Is there any way to get jacobian to return an SMatrix instead?

[jrevels]

Unless you give ForwardDiff a buffer to write the results to, it will construct the output buffer via similar. In the case of this example, here's the result of the similar call:

julia> ydual = out
2-element StaticArrays.SVector{2,ForwardDiff.Dual{2,Int64}}:
 Dual(-4,2,-4)
  Dual(1,-1,1)

julia> similar(ydual, Int, 2, 2)
2×2 Array{Int64,2}:
 4445979120  4445980464
 4450466064  4447128496

ForwardDiff could use map or something instead of similar + loading in the partials manually, but that's only viable for vector-mode differentiation. That would be okay except that it would cause a type instability (since chunk-mode would produce a normal Matrix while vector-mode would produce an SMatrix).

[KristofferC]

When using chunk mode, you could store the already computed partials somewhere and when all chunks are done, create the output SMatrix in one swoop with a generated function.

[jrevels]

When using chunk mode, you could store the already computed partials somewhere and when all chunks are done, create the output SMatrix in one swoop with a generated function.

You're referring here to introducing special codepaths for StaticArray types right? I agree that a lot of desirable behaviors could be introduced by creating StaticArrays-specific methods, but I'm not yet sure that ForwardDiff should take on StaticArrays as a dependency...

[KristofferC]

I was just saying it is theoretically possible. Providing functionality for AD with statically sized arrays that is completely allocation free and does not require a preallocated output matrix would be nice though. However, for array sizes when chunk mode starts to become good is when statically sized array become less good so I would say that the combination static array + chunk mode is not very important.