Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pusher environment with Spring pipeline #477

Closed
wladekpal opened this issue Apr 17, 2024 · 3 comments
Closed

Pusher environment with Spring pipeline #477

wladekpal opened this issue Apr 17, 2024 · 3 comments

Comments

@wladekpal
Copy link

Hi,

I've been trying to experiment with Pusher env using Spring as a pipeline, and had significant trouble getting anything to train. I investigated a bit and found that there seems to be something wrong with Spring-Pusher combination. For example in the training notebook training goes well for Positional and Generalized pipelines, but for Spring it fails. The graphs below are generated using said notebook:

spring positional generalized
Spring Positional Generalized

I tried to find out what the problem may be and after looking at renders it seems to me that in Spring pipeline there is no friction (or it is very small) between the object and the surface. I also include the video generated with Spring pipeline, that shows this behavior.

pusher.-.Trim.mp4
@btaba
Copy link
Collaborator

btaba commented Apr 23, 2024

Hi @wladekpal

The pusher model doesn't have contact friction. The object rests on a plane using joint constraints

<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>

So I'm a bit surprised that the positional pipeline works, but spring doesn't due to "damping" of the velocity.

With that being said, the spring pipeline dampens velocity in the integrator here:

vel=jp.exp(sys.vel_damping * sys.dt) * xd_i.vel,
ang=jp.exp(sys.ang_damping * sys.dt) * xd_i.ang)

So maybe you can add vel_damping here to see if that helps?

<numeric data="0" name="ang_damping"/>

@wladekpal
Copy link
Author

Alright, thanks for the response. Generalized backend works fine for my purpose, I just wanted to know if that discrapency in backends is expected. Since it is I will close the issue.

@AwesomeLemon
Copy link

The issue should be reopened, as it is a real bug that breaks pusher when the spring backend is used. I encountered exactly the same behaviour as @wladekpal

Adding vel_damping as suggested by @btaba solved the problem, though I had to set its value to -1, despite it being set to either 0 or 1 in the Brax codebase.

Looking at the integrator.py code shows that having vel_damping > 0 leads to anti-damping, as xd_i.vel is multiplied by exp(positive number) > 1. Someone should probably check if vel_damping should indeed be set to 1 (and not -1) in walker2d.xml and hopper.xml

P.S. Hilariously, the hyperparameter optimization procedure that I ran was able to work around the infinite sliding by nudging the object very slightly, so that it would spend the most of the episode next to the goal

policy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants