TROSD: A New RGB-D Dataset for Transparent and Reflective

Object Segmentation in Practice

Abstract

The TROSD is a large-scale RGB-D dataset aiming to address the challenging object segmentation problem. This dataset contains 11,060 RGB-D images with three semantic classes in terms of reflective objects, transparent objects and background.
As shown in the figure above, the green and red pixels in the semantic map represent the reflective objects and the transparent objects respectively.

Dataset Collection

We implement an iPad 2018 equipped with a Structure Sensor to capture these RGB-D images. Structure Sensor is an infrared structured light sensor. It has one infrared sender and one infrared receiver. The sensor projects an IR speckle pattern to the target object. This pattern is then reflected back to the sensor, and the depth is calculated based on the time between sending and receiving. The depth image is sent back to iPad and processed to match the RGB image pixel-by-pixel.
For target objects, we consider common types of them (mainly including glass products, mirrors, plastic and metal) with different size, shape and color. For scenes, the TROSD dataset contains 14 unique scenes (e.g. livingroom, bathroom, office, etc). Finally, we use Labelme to annotate these images manually.

Download Link:

Google Drive: https://drive.google.com/file/d/1rXwsbpKEBH4dV38aiVk5jxJWASkjTbjQ/view?usp=sharing

Notice!

1. The TROSD dataset contains 11,060 RGB-D images (7,421 for training and 3,639 RGB-D images for test).

2. There is some noise in the depth images due to the working mechanism of RGB-D camera we implement. The depth is saved as .png image.

3. In the semantic mask, 0, 1, 2 represent the background, transparent objects and reflective objects respectively.

License

This TROSD dataset is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation.

1. Reference to the TROSD Dataset is needed in any work that makes use of the dataset.

2. That you may not use the dataset or any derivative work for commercial purposes as, for example, licensing or selling the data, or using the data with a purpose to procure a commercial gain.

3. That all rights not expressly granted to you are reserved by us (Department of Electronic Engineering, Tsinghua University).