Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reuse space-ros/simulation repository for IsaacSim #18

Open
franklinselva opened this issue Aug 15, 2024 · 8 comments · May be fixed by #21
Open

Reuse space-ros/simulation repository for IsaacSim #18

franklinselva opened this issue Aug 15, 2024 · 8 comments · May be fixed by #21

Comments

@franklinselva
Copy link

franklinselva commented Aug 15, 2024 •
edited
Loading

Since the contents of this repository are moved to space-ros/demos based on space-ros/space-ros#178, I believe this repository can be reused exclusively for Isaac Sim which leverages photorealistic simulations.

Feature Proposal

The proposal is to repurpose the current repository to manage the simualtion assets and extensions for Space ROS simulation with Nvidia's Isaac SIM. Eventually the repository will be repurposed as the extension pack of Space ROS provided for Nvidia's Isaac SIM. The assets are made photo-realistic with a minimal sim2real gap with the information provided in the internet resources. The assets are a compilation of large set of use cases related to Space ROS applications which is summarized below. In addition to the assets, new extensions are created for space ros demos which serve as an example for building an ecosystem of simualtion extensions which could grow over time. A detailed documentation of all the assets and extensions are provided to help with the usage of the extension module.

Environment Assets

  1. International Space Station Model. Useful for Canadarm2 related maintainence and docking applications. international-space-station
    canadarm2-iss

  2. Interior of International Space Station. Useful for Astrobee and Robonaut2 for assistance related applications. interior-of-international-space-station

  3. Mars Terrain. 800 sq.m area of realistic mars terrain data acquired from NASA's Mars Exploration Data. mars-environment-curiosity-rover

  4. Moon Terrain. 800 sq.m area of realistic moon terrain data acquired from NASA's Moon Exploraiton Data. moon-environment

  5. Simple Chess Envrionment for Robonaut2. Useful in manipulation and control related tasks for Robonaut2. robonaut2-chessboard-manipulation

  6. Simple Mars Replica for Ingenuity Helicopter. Useful for Visual Odometry, Search and Mission Planning related applidations. ingenuity-helicopter-simple

Robot Assets

  1. Canadarm2. The gazebosim equivalent of canadarm2 robot model. canadarm2
  2. Curiosity Mars Rover. The gazebosim equivalent of curiosity Mars rover. curiosity-rover
  3. Perseverance Mars Rover. The high fidelity Robot Model with advanced sensors for Mars exploration related tasks. perseverance-rover
  4. Robonaut2 Upper body. Manipulation tasks and planning for robonaut2 upper body. robonaut-upperbody
  5. Robonaut2 Full body. Useful for applications with Interior of ISS. robonaut
  6. Ingenuity Helicopter. Aerial robot for search applications in mars terrain. ingenuity-helicopter
  7. Canadarm2 Destro. The docking and maintainence robot for ISS. canadarm2-17dof

IsaacSim Extensions

  1. An SpaceROS specific Omnigraph extensions for controlling rovers with rocker-bogie mechanism
  2. An example omnigraph extensions for building new extension specific to Space ROS

Documentation

  1. Detailed description and information on Environment assets
  2. Detailed description and usage instructions on robot assets
  3. Detailed description and usage instructions on extensions

The assets are widely tested for its functionality and are mostly simulation ready at the moment. The demo packages are already being built for direct integration between gazebosim and isaac sim. These assets should help cover wide range of subdomains and features exptected to be built on space ROS project.

The potential directions moving forward from this feature is to create a standalone simulator built on top of Nvidia's Omniverse for better control in packaging and distributing the simulation assets and version controls of extensions.

MetaInfo

Title: Simulation Assets and Extensions for Nvidia's Isaac Sim
Freelancer ID: @selvakumarhs
NASA Space ROS SIM Challenge
This was referenced Aug 23, 2024
Open
Open
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Aug 31, 2024
@franklinselva
Remove Simulation assets of gazebo enviroment and urdf files
1db6977 
Related to space-ros/space-ros#178
Related to space-ros#18

 - Moved the simulation assets to space-ros/demos#31
 - Repurpose the repository for the simulation assets of Nvidia's
Isaac Sim
@franklinselva franklinselva linked a pull request Aug 31, 2024 that will close this issue
Open
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 1, 2024
@franklinselva
Update README.md and documentation for Isaac Sim Extension Module
3c3a2ba 
Related to space-ros#18
 - Add documentation for Isaac Sim Extension Module
 - Add images for Isaac Sim Extension Module documentation
 - Update README.md with information about Isaac Sim Extension Module
 - Add .gitattributes file to handle large files
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 2, 2024
@franklinselva
Update README.md and documentation for Isaac Sim Extension Module
f9699c4 
Related to space-ros#18
 - Add documentation for Isaac Sim Extension Module
 - Add images for Isaac Sim Extension Module documentation
 - Update README.md with information about Isaac Sim Extension Module
 - Add .gitattributes file to handle large files
 - Add current list of features available in robots
@franklinselva
Copy link
Author

@mkhansenbot @EzraBrooks I am trying to push the work on simulation assets for Nvidia's Isaac Sim. I am using git-lfs for pushing the large files to the repository but couln't succeed in pushing the changes to the fork.

I am thrown with,

batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation                                                                                                                                                                                                                                                                        
batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation
error: failed to push some refs to 'https://github.com/franklinselva/spaceros-simulation.git'

Upon looking on the issue related to this, the commit on git-lfs mentions the repository should already have lfs setup on the original repository or the contributor should have the push permissions for the original repository i.e. write access to space-ros/simulation. Can you please help me resolve this issue?

franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 8, 2024
@franklinselva
Add documentation for the repository
30b02dc 
Related to space-ros#18

 - Add documentation and description for environment and robot assets
 - Add documentation on extensions and usage
 - Add documentation on setting up ROS2 interface and ROS2 action graphs
 - Update README.md for system requirements, getting started guide and overview
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 8, 2024
@franklinselva
Add extensions for IssacSim targetting Space ROS
63384df 
Related to space-ros#18

 - Add new extension - Rover Simple Controller
 - Add example omnigraph extension - Negate Number
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 8, 2024
@franklinselva
Minor fixes to extensions docs
de036ee 
Related to space-ros#18
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Update robot documentations for required extensions
07f00be 
Related to space-ros#18
@EzraBrooks
Copy link
Member

I'm not 100% sold on the idea that Isaac Sim, a proprietary simulator, should live in the same repository as non-proprietary simulators. I don't want Space ROS to trend toward being vendor-locked by NVIDIA. I don't even have a computer with an NVIDIA GPU 😉

@franklinselva
Copy link
Author

franklinselva commented Sep 9, 2024 •
edited
Loading

One of the questions on the SpaceROS SIM Webinar was to choose Nvidia's Isaac Sim / Unreal engine for improved visual fidelity. The response to the question was we could go for one of the simulators in this case.

The Q&A of the webinar can be found here. Rows 3 and 27 are related to this issue.

I completely agree with the setup of requiring NVidia GPU for such simulators. For this purpose, I have already managed to write a workaround for this. More information can be found here.

Moving further, IsaacSim is not a definitive simulator but a reference application built on top of Nvidia Omniverse. This feature intends to come up with a dedicated application as I referred to in the description of this issue.

@EzraBrooks
Copy link
Member

Gotcha. I was more just saying "maybe it doesn't live in the same place as the Gz simulation", not that it doesn't have a place at all. I totally understand the desire for higher-fidelity perception simulation.

I'll take a look at setting up LFS here, since that's a good improvement anyway.

@franklinselva
Copy link
Author

Thanks a lot. I am currently setting up the LFS template for an easier entry point. I am stuck with pushing the assets to the repository at the moment. Is it something, I could overcome soon? The due date for the challenge is right around the corner.

@EzraBrooks
Copy link
Member

See #26, #27

@EzraBrooks
Copy link
Member

LFS should be enabled on the repo now. Let me know if it still gives you issues - maybe it's a permissions thing.

If it is, I'm not sure what the right route forward here is - giving upstream write access to everyone who wants to contribute doesn't seem sustainable.

franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Remove Simulation assets of gazebo enviroment and urdf files
c7078b5 
Related to space-ros/space-ros#178
Related to space-ros#18

 - Moved the simulation assets to space-ros/demos#31
 - Repurpose the repository for the simulation assets of Nvidia's
Isaac Sim
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Add documentation for the repository
4e29501 
Related to space-ros#18

 - Add documentation and description for environment and robot assets
 - Add documentation on extensions and usage
 - Add documentation on setting up ROS2 interface and ROS2 action graphs
 - Update README.md for system requirements, getting started guide and overview
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Add extensions for IssacSim targetting Space ROS
ff93198 
Related to space-ros#18

 - Add new extension - Rover Simple Controller
 - Add example omnigraph extension - Negate Number
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Minor fixes to extensions docs
40bf065 
Related to space-ros#18
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Update robot documentations for required extensions
c0877e9 
Related to space-ros#18
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Move asset images to git lfs
7736e7a 
Related to space-ros#18
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Add simulation assets for Nvidia's Isaac Sim
be47b29 
Related to space-ros#18

 - Add environment scenes
	- Add ISS and interior of ISS
	- Add Mars Terrain
	- Add Moon Terrain
	- Add Earth Model
 - Add robot scenes
	- Add curiosity Rover model
	- Add Ingenuity Helicopter model
	- Add Canadarm model group
	- Add Robonaut2 model group
	- Add JPL Athelte model
 - Add relevant scenes based on the models above
 - Add mars clear sky scene
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 9, 2024
@franklinselva
Update README.md for git-lfs instructions
d91836f 
Related to space-ros#18
@franklinselva
Copy link
Author

LFS setup works good. I have already pushed the change. Thanks for the help!

franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 11, 2024
@franklinselva
Improve suspension arm PD gains of curiosity rover for torque control
4029461 
Related to space-ros#18
franklinselva added a commit to franklinselva/spaceros-simulation that referenced this issue Sep 11, 2024
@franklinselva
Add Perseverance Rover Robot Model
1c48e83 
Related to space-ros#18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@EzraBrooks @franklinselva