RFVision

Introduction

RFVision is an open-source toolbox for robot vision. It is a part of the RobotFlow project.

The project takes a lot of design ideas from the great project mmcv, MMDetection, MMDetection3D, but with significant modifications to adapt to robot vision tasks. For pure 2D or 3D vision tasks, we recommend to use MMDetection/3D as they support more baselines.

But for robot vision where real-time visual servo, multi-modal perception are of more interest, RFVision is the project for you.

License

This project is released under the Apache 2.0 license.

Model Zoo

1. 2D Object Detection

   Most bag of freebies are supported

   • Mask RCNN
   • PANet
   • HTC
   • DCT-Mask
   • FCOS
   • SOLOv2
   • YOLOv3/3-tiny/4/4-tiny/4-large
   • USD-Seg
   • ContourRender
2. 3D Object Detection
   • VoteNet
   • ImVoteNet
3. Object Pose Estimation
   • DenseFusion
   • SkeletonMerger
   • ArticualtedPoseEstimation
   • OMAD
4. Human Analyzers
   • HMR
   • SimpleBaseline3D
   • InterNet
   • HandTailor
   • CPF (inference only, for training material, you may contact to get rfvision_restricted project)
   • Artiboost (rfvision_restricted project)
5. Multi-modality
   • 3D vision and touch for shape reconstruction

Installation

Please refer to get_started.md for installation

Contributing

We appreciate all contributions to improve RFVision. Please refer to CONTRIBUTING.md for the contributing guideline.

Ackowledgement

We borrow many design from projects in OpenMMLab, we appreciate their efforts on lowering the research bar of computer vision.

When the era of robot learning comes, we wish our toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new methods.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

Projects in RobotFlow

Software

• RFLib: RobotFlow foundational library for Robot Vision, Planning and Control.
• RFVision: RobotFlow vision-related toolbox and benchmark.
• RFMove: RobotFlow planning toolbox.
• ReinForce: RobotFlow reinforcement learning toolbox.
• RFController: RobotFlow controller toolbox.
• rFUniverse: A Unity-based Multi-purpose Simulation Environment.
• RFBulletT: A Pybullet-based Multi-purpose Simulation Environment.
• RF_ROS: ROS integration. Both ROS1 and ROS2 are supported.
• RobotFlow: The barebone of the whole system. It organizes all the functionalities.

Hardware

• RFDigit: A Customized Digit Tactile Sensor.

Open Ecosystem

• N-D Pose Annotator: support both rigid and articulated object pose annotation.
• model format converter: URDF and related model format converter.