[Question] How can I give dummy steps?

[PARKBONG]

Question

Hello, I want to add dummy steps to the environment. For example, it would look like this:

Dummy action: The gripper moves 1 unit along the x-axis for 10 steps due to the dummy action, then closes to grab the object.
Action: The robot is driven by the agent. The robot performs a specific task (e.g., rotation).
Dummy action: After the rotation is performed, the gripper moves 1 unit along the y-axis due to the dummy action.
In this case, the entire environment has early termination and a multi-environment.

The important point here is that the rollout should not be contaminated. That is, when the env_0 dummy action is performed, env_0 does not provide any information to the rollout, and env_1 still collects rollouts. the env_0 and env_1 are updated together through sim.step().

Is there already completed work for this? I'm asking because it's challenging to code from scratch due to early termination and multi-environment.

Thank you.

[Mayankm96]

Hi,

Unfortunately, asynchronous updates of environment instances are not possible within Isaac Sim / Isaac Gym. Collecting large-scale data collection in this manner isn't possible directly.

As a workaround, what you could do is wait for all the environments to reach the objects and perform the dummy actions simultaneously for all of them. Alternatively, you could add a boolean flag to your extras dictionary that provides information on whether an agent action or dummy action is being executed for that environment instance (kind of like masking). Then on the data record/agent side, you can read the extras dictionary and decide to add the transitions or wait for the dummy action to complete.

[PARKBONG]

I don't know how the rollout is handled after it gets into the buffer yet, but maybe I can find the right way to add dummy steps to the environment.

Thank you for the advice to use extras dictionary.