Expectations on observation type

[zsunberg]

I'm thinking about making a bridge from POMDPs.jl to this package. Are there any expectations/requirements on what the output of observe is? Should it be an AbstractVector for example?

[zsunberg]

Oh, is it supposed to be an Observation? If so, are there any expectations on the type of Observation.state

[zsunberg]

Ah, I see that there actually is already some integration with POMDPs! Cool! I may make a PR with some updates to support more features of POMDPs.jl

[findmyway]

Great!

There's no constraint of Observation. It's just a very thin wrapper of NameTuple to make things easier for multiple dispatch.

I remember that POMDP related environments are written by @jbrea . He may provide help with your PR.

[findmyway]

Hi @zsunberg,

The observation can be an arbitrary type now, as long as if the interfaces are implemented.

Inspired by RLInterface.jl, I add the support for POMDP models here.

I'm not very familiar with the code in POMDP. So currently, for a (PO)MDPModel, I tried to implement the get_action_space interface like this here:

RLBase.get_action_space(m::Union{<:POMDP,<:MDP}) = convert(AbstractSpace, actions(m))

But it won't work for (Multi)ContinuousSpace (because I didn't implement the convert functions for them). Is there any methods in POMDP to detect whether a model is continuous or discrete?

Besides, the get_observation_space is also not implemented. One can manually specify it when it is useful. (For example, like this)

[zsunberg]

Nice, very cool! Where did you get the @interface macro? I am very interested in what you are using for that.

Also, is it necessary for all spaces to be AbstractSpaces? I have found it very convenient to let the action space be a Vector or Set.

There is not currently an interface to determine whether spaces continuous or discrete. We have not taken the time to develop any types to represent bounded continuous spaces since we found that most problems are hybrid anyways and it is actually convenient in many solvers to not make the distinction between continuous and discrete.

[findmyway]

Where did you get the @interface macro? I am very interested in what you are using for that.

Inspired by

• https://github.com/dalum/InlineExports.jl/blob/master/src/InlineExports.jl
• https://github.com/QuantumBFS/YaoBase.jl/blob/master/src/utils/interface.jl

I just use it for in-line export. And when the function body is missing, an error expression will be inserted.

is it necessary for all spaces to be AbstractSpaces?

Good question. If the action space is just Vector or Set. Then interfaces of AbstractSpace are implemented by default. So there shouldn't be any problem at least for now.

it is actually convenient in many solvers to not make the distinction between continuous and discrete.

I see. Thanks for the message.