-
Notifications
You must be signed in to change notification settings - Fork 1
Add Unity Scenes
We provide our entire Unity source code in the folder GRASP. In this wiki, we explain how to create a new test for the benchmark in Unity (we assume some prior knowledge about Unity).
We use the ML-Agents Unity plugin to generate videos through Python, i.e. every video is represented by an episode with multiple steps (all our scenes have 500 steps). New scenes can be added as follows:
New scenes should be created in Assets/Scenes
. The Room
prefab in Assets/Resources/Prefabs
can be used to start with a blank room in the scene.
The next step should be to create a GameObject in the scene that contains all the elements necessary for the setup of the new test. In our scenes, this GameObject is always referred to as Apparatus
. You can copy one of these objects from existing scenes into your new scene to start with. Dynamic objects in the experiment should be attached with either the ExperimentComponent
or MovingExperimentComponent
scripts that can be found in Assets/Scripts
: ExperimentComponent
is any dynamic object that is moving passively, e.g. due to gravity, while MovingExperimentComponent
is any dynamic object that is actively being moved to a target.
The logic for the newly added test should be implemented in a C# script in Assets/Scripts
. The logic is implemented in its own class that should inherit from the Experiment
class. Attach this script to the GameObject that was created in step 2 and populate the variables of the attached script in Unity. The default variables of the experiment class are:
-
Min Delay
andMax Delay
: A start delay before any actions are triggered is sampled from [Min Delay
,Max Delay
]. -
Components
: These are all the dynamic objects from step 2 that have been attached with either theExperimentComponent
orMovingExperimentComponent
script. These objects are being reset to their original positions after each episode. -
Randomization Objs
: These are all objects that have randomization scripts attached to them (see step 5 below), so they can be randomized every episode.
Finally, to tie everything together and be able to generate videos, an Observer
object that represents the camera has to be added to the scene. You can use the Observer
prefab in Assets/Resources/Prefabs
for this. This prefab has the ObservingAgent
script attached to it which inherits from the ML-Agents Agent
class. After attaching the apparatus from step 2 to the Experiment
variable of the Observer, the logic from the experiment will automatically be triggered every episode. Additionally, the ObservingAgent
script contains some logic for the randomization of the camera position.
We provide scripts to randomize the texture and color of objects in the scene. Simply attach the RandomizeColor
or RandomizeTextures
(or both) to an objects and all its children will have these properties randomized.