Inherit fields with default values when using @agent

[mastrof]

julia> @agent Foo1 GridAgent{2} begin
           a = 1
       end

julia> @agent Foo2 Foo1 begin
           b = 2
       end

julia> Foo1(id=1, pos=(0,0))
Foo1(1, (0, 0), 1)

julia> Foo2(id=1, pos=(0,0))
ERROR: UndefKeywordError: keyword argument `a` not assigned
Stacktrace:
 [1] top-level scope
   @ REPL[19]:1

I think it would be nice to have Foo2 inherit the default value of a from Foo1, is it problematic to implement?

(Sorry for the various requests concerning the @agent macro but I'm a bit out of my depth here and really appreciate all the work you guys put into this package)

[Tortar]

this is something I was thinking about too when I added default values to the macro, then I forgot about it :-), I think it should be possible!

[Tortar]

bad news: https://discourse.julialang.org/t/how-to-check-if-field-in-a-structure-has-defined-default-value/77449

maybe we can get around this limitation somehow though

[mastrof]

Maybe stupid idea: could it be an option to have a different macro, say @baseagent, which behaves exactly as @agent but adds an extra field _constructor where we store the expr to generate the fields? Something like _constructor = :( $$(Meta.quot([base_fields, additional_fields])) )
Then @agent would need an if clause and if _constructor exists it could use the original expression to create other types.

julia> @baseagent BA Agent0 AbstractAgent begin
           id::Int = 1
       end
expr = quote
    #= /home/riccardo/Playground/StructMacro/mymacro.jl:94 =#
    #= /home/riccardo/Playground/StructMacro/mymacro.jl:94 =# @kwdef mutable struct BA <: AbstractAgent
            #= /home/riccardo/Playground/StructMacro/mymacro.jl:95 =#
            #= /home/riccardo/Playground/StructMacro/mymacro.jl:96 =#
            id::Int = 1
            #= /home/riccardo/Playground/StructMacro/mymacro.jl:97 =#
            _constructor = $(Expr(:quote, :($(Expr(:$, :($(Expr(:quote, Vector{Any}[[], [:(id::Int = 1)]]))))))))
        end
end

julia> BA()
BA(1, Vector{Any}[[], [:(id::Int = 1)]])

where Agent0 here is just a dummy with no fields.
I still have to give a shot at redefining agent to extract _constructor and re-use it to define the new struct, but I mean the expr is there, it should be possible I guess

EDIT: nvm I didn't realize that the we still have a problem to access this field from the other macro 🙃

[Tortar]

I have another idea that should work: save the additional fields in a dictionary so that we save also the default value given since we actually save expressions, hopefully this should work without any change

edit: no, it doesn't work :D

[Tortar]

it seems like neither @with_kw can do it yet, this is the open issue about it: mauro3/Parameters.jl#113

[Tortar]

We could introspect with code_lowered probably to find the values of the defaults:

> @kwdef struct A
           x::Vector{Int} = [1,2,5]
           y::Int
       end

> code_lowered(A, Tuple{})
1-element Vector{Core.CodeInfo}:
 CodeInfo(
1 ─ %1 = Base.vect(1, 2, 5)
│        x = %1
│   %3 = Core.UndefKeywordError(:y)
│   %4 = Core.throw(%3)
│        y = %4
│   %6 = Main.:(var"#C#9")(x, y, #self#)
└──      return %6
)

but it seems rather difficult to trust a parsing of something like this to me :D

[mastrof]

I just stumbled upon Expronicon, I wonder if it can be of any help in reworking the macros? I only had a superficial look but maybe it's useful
E.g. it can retrieve all the relevant information from an expression and allow super easy access to it

julia> struct_expr = :(
           mutable struct Foo{D,T}
               a::Int
               b::T = zero(T)
               c::SVector{D} = fill(3.0, SVector{D})
           end
       )
:(mutable struct Foo{D, T}
      #= REPL[73]:3 =#
      a::Int
      #= REPL[73]:4 =#
      b::T = zero(T)
      #= REPL[73]:5 =#
      c::SVector{D} = fill(3.0, SVector{D})
  end)

julia> kws = JLKwStruct(struct_expr)
mutable struct Foo{D, T}
    a::Int
    b::T
    c::SVector{D}
end
function (Foo{D, T})(; a, b = zero(T), c = fill(3.0, SVector{D})) where {D, T}
    (Foo{D, T})(a, b, c)
end
function Foo(; a, b = zero(T), c = fill(3.0, SVector{D}))
    Foo(a, b, c)
end
nothing

julia> kws.fields
3-element Vector{JLKwField}:
 a::Int
 b::T
 c::SVector{D}

julia> kws.fields[1].default
no_default

julia> kws.fields[2].default
:(zero(T))

julia> kws.fields[3].default
:(fill(3.0, SVector{D}))

julia> kws.fields[3]
c::SVector{D}

Which means we could possibly define JLKwStructs behind the scenes and use them to generate actual agent types:

julia> kws = @expr JLKwStruct mutable struct Foo
           a::Int = 1
       end
mutable struct Foo
    a::Int
end
function Foo(; a = 1)
    Foo(a)
end
nothing
nothing

julia> codegen_ast(kws)
quote
    #= /home/riccardo/.julia/packages/Expronicon/7EBrJ/src/codegen.jl:102 =#
    mutable struct Foo
        #= REPL[2]:2 =#
        a::Int
    end
    #= /home/riccardo/.julia/packages/Expronicon/7EBrJ/src/codegen.jl:103 =#
    begin
        #= /home/riccardo/.julia/packages/Expronicon/7EBrJ/src/codegen.jl:158 =#
        function Foo(; a = 1)
            Foo(a)
        end
        #= /home/riccardo/.julia/packages/Expronicon/7EBrJ/src/codegen.jl:159 =#
        nothing
    end
    #= /home/riccardo/.julia/packages/Expronicon/7EBrJ/src/codegen.jl:104 =#
    nothing
end

julia> eval(codegen_ast(kws))

julia> Foo()
Foo(1)

[Tortar]

This seems like a great finding to me!

But if I imagine correctly what should be done is to have some sort of internal structure with all of these JLKwStruct(new agents) and then when the user creates another new agent, we pick the JLKwStruct(base agent) from the internal structure to build the new agent (which means we can probably use a dict)

[Tortar]

Is this correct in your eyes?

[mastrof]

Yes as a naive approach I was also thinking about having a Dict to store all the parsed expressions.
I don't know if we could consider it "good practice" to do such a thing but it doesn't seem like a big deal, especially because it only affects the stage of agent creation

[mastrof]

Ok I have a very rough yet working version, it has to be significantly improved but at least we know it can work.
Here I first create Bob from NoSpaceAgent and then Mark inherits const and kw fields from Bob

julia> @newagent Bob{D} NoSpaceAgent begin
           a::SVector{D,Int}
           b = 35
           const c = 2
       end

julia> Bob{3}(a = SVector{3}(1,2,3))
ERROR: UndefKeywordError: keyword argument `id` not assigned
Stacktrace:
 [1] top-level scope
   @ REPL[8]:1

julia> Bob{3}(a = SVector{3}(1,2,3); id=1)
Bob{3}(1, [1, 2, 3], 35, 2)

julia> @newagent Mark{D} Bob{D} where {D} begin
           d = randn()
       end

julia> Mark{1}(; id=27, a = SVector{1}(2))
Mark{1}(27, [2], 35, 2, -1.3877457562589872)

julia> m = Mark{1}(; id=27, a = SVector{1}(2))
Mark{1}(27, [2], 35, 2, -1.1504274174939895)

julia> m.c = 3
ERROR: setfield!: const field .c of type Mark cannot be changed
Stacktrace:
 [1] setproperty!(x::Mark{1}, f::Symbol, v::Int64)
   @ Base ./Base.jl:38
 [2] top-level scope
   @ REPL[13]:1

julia> m.d = 0
0

Here the code I'm using (which it's just a quick hack on top of the current @agent and atm requires re-export of the ExproniconLite functions used in the inner quote):

__AGENT_GENERATOR__ = Dict{Symbol,JLKwStruct}()

mutable struct NoSpaceAgent
    const id::Int
end
NoSpaceAgent_JLKwS = @expr JLKwStruct mutable struct NoSpaceAgent <: AbstractAgent
    const id::Int
end
__AGENT_GENERATOR__[NoSpaceAgent_JLKwS.name] = NoSpaceAgent_JLKwS

macro newagent(new_name, base_type, super_type, extra_fields)
    quote
        base_T = $(esc(base_type))
        BaseAgent = Agents.__AGENT_GENERATOR__[Symbol(base_T)]
        additional_fields = $(QuoteNode(extra_fields.args))
        # here, we mutate any const fields defined by the consts variable in the macro
        additional_fields = filter(f -> typeof(f) != LineNumberNode, additional_fields)
        args_names = map(f -> f isa Expr ? f.args[1] : f, additional_fields)
        index_consts = findfirst(f -> f == :constants, args_names)
        if index_consts != nothing
            consts_args = eval(splice!(additional_fields, index_consts))
            for arg in consts_args
                i = findfirst(a -> a == arg, args_names)
                additional_fields[i] = Expr(:const, additional_fields[i])
            end
        end
        name = $(QuoteNode(new_name))
        expr = quote
            # create struct with additional_fields using Expronicon
            S = @expr JLKwStruct mutable struct $name <: $$(QuoteNode(super_type))
                $(additional_fields...)
            end
            # add fields from base type
            S.fields = vcat($BaseAgent.fields, S.fields)
            # evaluate generated code
            Core.eval($$(__module__), eval(codegen_ast(S)))
            # add generator to dictionary
            Agents.__AGENT_GENERATOR__[S.name] = S
        end
        # evaluate macro in the calling module
        Core.eval($(__module__), expr)
        # allow attaching docstrings to the new struct, issue #715
        Core.@__doc__($(esc(Docs.namify(new_name))))
        nothing
    end
end

macro newagent(new_name, base_type, extra_fields)
    esc(quote
        Agents.@newagent($new_name, $base_type, Agents.AbstractAgent, $extra_fields)
    end)
end

[Tortar]

Very cool!

But wait a second, how did you manage to make this work?

@newagent Bob{D} NoSpaceAgent begin
           a::SVector{D,Int}
           b = 35
           const c = 2 #caused by this
       end

Are you using an old julia release? this should be a parsing error, @Datseris and I agreed upon this new version of the macro (to better support struct syntax)

@agent GridAgent{2} struct NewAgent <: Foo
    x::Int
    const y::Int
   z::Real = 0.5
end

which I will finishing implementing in #870 .

After that, implementing this feature would be really cool

[Tortar]

ok, understood now:

@newagent Bob{D} NoSpaceAgent begin
           a::SVector{D,Int}
           b = 35
           const c::Int # no assignment
       end

this doesn't work

[mastrof]

It's just due to the dirty implementation on my side; Expronicon allows generating structs with unassigned const values:

julia> s = JLKwStruct(;
           name = :Wilson,
           ismutable = true,
           fields = [
               JLKwField(; name = :a, type = Int, isconst = true)
           ]
       )
mutable struct Wilson
    const a::Int64
end
function Wilson(; a)
    Wilson(a)
end
nothing
nothing

julia> eval(codegen_ast(s))

julia> w = Wilson(a = 12)
Wilson(12)

julia> w.a = 444
ERROR: setfield!: const field .a of type Wilson cannot be changed
Stacktrace:
 [1] setproperty!(x::Wilson, f::Symbol, v::Int64)
   @ Base ./Base.jl:38
 [2] top-level scope
   @ REPL[19]:1

I think that just proper usage of their JLKwStruct and JLKwField constructors will make life much easier; I'll try to make a cleaner version of the macro

[mastrof]

Ah sorry I now see what you mean, it's Julia that does not allow you to simply write const c::Int inside of a begin block, got it

[Tortar]

Hi @mastrof I have finished #870 so now the agent macro is updated, in the end we slightly improved syntax to

@agent struct NewAgent(GridAgent{2}) <: Foo
    x::Int
    const y::Int
    z::Real = 0.5
end

and I'd really like to improve the new version even more with what we discussed here, if you open up a draft PR, I will try to help you as much as I can :-)

[Datseris]

I guess using this global dictionary __AGENT_GENERATOR__ = Dict{Symbol,Any}() isn't a bad idea after all.

[mastrof]

I'll be rather short on time in the next 2 weeks, but I'll draft the PR later today so we get moving.
The issue I'm facing is how to define a non-parametric type from a parametric one, e.g. @agent struct Foo(GridAgent{2}). The JLKwStruct stores the information about the expression that generates GridAgent, so I guess what we should do is evaluate the expressions associated to the default types of GridAgent{D} after manually replacing D -> 2. And in particular, it should be done in a way that automagically extends to however many parameters a user will specify..

(Btw Tortar if you think you can do this faster, which is probably the case, you don't have to wait for me)