Skip to content

Add Unity Scenes

serwansj edited this page Jun 7, 2024 · 3 revisions

We provide our entire Unity source code in the folder GRASP. In this wiki, we explain how to create a new test for the benchmark in Unity (we assume some prior knowledge about Unity).

We use the ML-Agents Unity plugin to generate videos through Python, i.e. every video is represented by an episode with multiple steps (all our scenes have 500 steps). New scenes can be added as follows:

1. New scene

New scenes should be created in Assets/Scenes. The Room prefab in Assets/Resources/Prefabs can be used to start with a blank room in the scene.

2. Test Design

The next step should be to create a GameObject in the scene that contains all the elements necessary for the setup of the new test. In our scenes, this GameObject is always referred to as Apparatus. You can copy one of these objects from existing scenes into your new scene to start with. Dynamic objects in the experiment should be attached with either the ExperimentComponent or MovingExperimentComponent scripts that can be found in Assets/Scripts: ExperimentComponent is any dynamic object that is moving passively, e.g. due to gravity, while MovingExperimentComponent is any dynamic object that is actively being moved to a target.

3. Test Logic

The logic for the newly added test should be implemented in a C# script in Assets/Scripts. The logic is implemented in its own class that should inherit from the Experiment class. Attach this script to the GameObject that was created in step 2 and populate the variables of the attached script in Unity. The default variables of the experiment class are:

  • Min Delay and Max Delay: A start delay before any actions are triggered is sampled from [Min Delay, Max Delay].
  • Components: These are all the dynamic objects from step 2 that have been attached with either the ExperimentComponent or MovingExperimentComponent script. These objects are being reset to their original positions after each episode.
  • Randomization Objs: These are all objects that have randomization scripts attached to them (see step 5 below), so they can be randomized every episode.

4. Observer

Finally, to tie everything together and be able to generate videos, an Observer object that represents the camera has to be added to the scene. You can use the Observer prefab in Assets/Resources/Prefabs for this. This prefab has the ObservingAgent script attached to it which inherits from the ML-Agents Agent class. After attaching the apparatus from step 2 to the Experiment variable of the Observer, the logic from the experiment will automatically be triggered every episode. Additionally, the ObservingAgent script contains some logic for the randomization of the camera position.

5.(Optional) Randomization

We provide scripts to randomize the texture and color of objects in the scene. Simply attach the RandomizeColor or RandomizeTextures (or both) to an objects and all its children will have these properties randomized.

Clone this wiki locally