Incorporating depth information into few-shot semantic segmentation - Université d'Évry Access content directly
Conference Papers Year : 2021

Incorporating depth information into few-shot semantic segmentation


Few-shot segmentation presents a significant challenge for semantic scene understanding under limited supervision. Namely, this task targets at generalizing the segmentation ability of the model to new categories given a few samples. In order to obtain complete scene information, we extend the RGB-centric methods to take advantage of complementary depth information. In this paper, we propose a two-stream deep neural network based on metric learning. Our method, known as RDNet, learns class-specific prototype representations within RGB and depth embedding spaces, respectively. The learned prototypes provide effective semantic guidance on the corresponding RGB and depth query image, leading to more accurate performance. Moreover, we build a novel outdoor scene dataset, known as Cityscapes-3i, using labeled RGB images and depth images from the Cityscapes dataset. We also perform ablation studies to explore the effective use of depth information in few-shot segmentation tasks. Experiments on Cityscapes-3i show that our method achieves excellent results with visual and complementary geometric cues from only a few labeled examples.
Fichier principal
Vignette du fichier
ICPR_2020_YZ_DS_OM_FM.pdf (6.84 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02887063 , version 1 (01-07-2020)


  • HAL Id : hal-02887063 , version 1


Yifei Zhang, Désiré Sidibé, Olivier Morel, Fabrice Meriaudeau. Incorporating depth information into few-shot semantic segmentation. 25th International Conference on Pattern Recognition (ICPR 2020), Jan 2021, Milan, Italy. pp.3582--3588. ⟨hal-02887063⟩
401 View
391 Download


Gmail Facebook X LinkedIn More