Similarly to uHumans, we use a photo-realistic Unity-based simulator to test Kimera and its ability to reconstruct a DSG. The simulator provides the 2D semantic segmentation for Kimera. Humans are simulated using standard graphics assets, and in particular the realistic 3D models provided by the SMPL project. A ROS service enables us to spawn objects and agents into the scene on-demand. The simulator provides ground-truth poses of humans and objects, which we use for benchmarking. Using this setup, we create several large visual-inertial datasets which feature scenes with and without dynamic agents, as well as a large variety of environments (indoors and outdoors, small and large).
where xx
in *_xxh.bag
indicates the number of humans in the scene.
We provide a convenient python script to download and decompress the full dataset from google drive:
If you encounter any problem, you can always download the rosbags manually by clicking the links above, and you can optionally decompress the rosbag by running rosbag decompress
on it.
The reason why we decompress the rosbags is that it will make any subsequent code run faster.
/tesse/left_cam
, /tesse/right_cam
)/tesse/depth_cam
)/tesse/seg_cam
)/tesse/imu
)/tesse/odom
)/tesse/front_lidar/scan
)/tf
)/tf_static
)Here we provide a list of the topics in a rosbag:
types: geometry_msgs/Vector3Stamped [7b324c7325e683bf02a9b14b01090ec7]
nav_msgs/Odometry [cd5e73d190d741a2f92e81eda573aca7]
rosgraph_msgs/Clock [a9c97c1d230cfc112e270351a944ee47]
sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
sensor_msgs/Imu [6a62c6daae103f4ff57a132d6f95cec2]
sensor_msgs/LaserScan [90c7ef2dc6895d81024acba2ac42f369]
tf2_msgs/TFMessage [94810edda583a504dfda3829e70d7eec]
topics: /clock 10151 msgs : rosgraph_msgs/Clock
/tesse/depth_cam/camera_info 8307 msgs : sensor_msgs/CameraInfo
/tesse/depth_cam/mono/image_raw 8307 msgs : sensor_msgs/Image
/tesse/front_lidar/scan 16962 msgs : sensor_msgs/LaserScan
/tesse/imu/clean/imu 101213 msgs : sensor_msgs/Imu
/tesse/imu/noisy/biases/accel 101150 msgs : geometry_msgs/Vector3Stamped
/tesse/imu/noisy/biases/gyro 101150 msgs : geometry_msgs/Vector3Stamped
/tesse/imu/noisy/imu 101212 msgs : sensor_msgs/Imu
/tesse/left_cam/camera_info 8307 msgs : sensor_msgs/CameraInfo
/tesse/left_cam/mono/image_raw 8307 msgs : sensor_msgs/Image
/tesse/left_cam/rgb/image_raw 8307 msgs : sensor_msgs/Image
/tesse/odom 101213 msgs : nav_msgs/Odometry
/tesse/rear_lidar/scan 16962 msgs : sensor_msgs/LaserScan
/tesse/right_cam/camera_info 8307 msgs : sensor_msgs/CameraInfo
/tesse/right_cam/mono/image_raw 8307 msgs : sensor_msgs/Image
/tesse/right_cam/rgb/image_raw 8307 msgs : sensor_msgs/Image
/tesse/seg_cam/camera_info 8307 msgs : sensor_msgs/CameraInfo
/tesse/seg_cam/rgb/image_raw 8307 msgs : sensor_msgs/Image
/tf 113606 msgs : tf2_msgs/TFMessage
/tf_static 1 msg : tf2_msgs/TFMessage
uHumans2
?VO/VIO/SLAM in static and dynamic scenes: estimate localization errors from your Visual (mono/stereo) inertial odometry, or SLAM system, by using the ground-truth pose information in the /tesse/odom
topic.
3D geometric and semantic reconstruction: evaluate mapping errors. We cannot provide the ground-truth 3D meshes with semantic annotations, but depending on the number of requests we may provide an automated server for submission and evaluation. Feel free to contact us: arosinol@mit.edu
.
3D Human pose estimation: we provide the pose information of humans in the \tf
topic.
3D Object pose estimation: we provide the pose information of objects in the \tf_static
topic.
@InProceedings{Rosinol21arxiv-Kimera,
title = {Kimera: from {SLAM} to Spatial Perception with {3D} Dynamic Scene Graphs},
author = {A. Rosinol, A. Violette, M. Abate, N. Hughes, Y. Chang, J. Shi, A. Gupta, L. Carlone},
year = {2021},
booktitle = {arxiv},
pdf = {https://arxiv.org/pdf/2101.06894.pdf}
}
[2] A. Rosinol, A. Gupta, M. Abate, J. Shi, L. Carlone. 3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans. Robotics: Science and Systems (RSS), 2020. arXiv:2002.06289.
[3] A. Rosinol, T. Sattler, M. Pollefeys, L. Carlone. Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities. IEEE Int. Conf. on Robotics and Automation (ICRA), 2019. arXiv:1903.01067
[4] A. Rosinol, M. Abate, Y. Chang, L. Carlone, Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2020. arXiv:1910.02490.
Feel free to reach out with comments, feedback, or feature requests at: arosinol@mit.edu
.