%0 Journal Article %T Improving Image Description with Auxiliary Modality for Visual Localization in Challenging Conditions %+ Equipe VIBOT - VIsion pour la roBOTique [ImViA EA7535 - ERL CNRS 6000] (VIBOT) %+ Laboratoire sciences et technologies de l'information géographique (LaSTIG) %+ Informatique, BioInformatique, Systèmes Complexes (IBISC) %A Piasco, Nathan %A Sidibé, Désiré %A Gouet-Brunet, Valérie %A Demonceaux, Cédric %< avec comité de lecture %@ 0920-5691 %J International Journal of Computer Vision %I Springer Verlag %V 129 %N 1 %P 185–202 %8 2021-01 %D 2021 %R 10.1007/s11263-020-01363-6 %K Global image descriptor %K Image retrieval %K Localization %K Side modality learning %K Depth from monocular %Z Computer Science [cs]/Artificial Intelligence [cs.AI] %Z Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] %Z Computer Science [cs]/Machine Learning [cs.LG] %Z Computer Science [cs]/Multimedia [cs.MM] %Z Computer Science [cs]/Robotics [cs.RO]Journal articles %X Image indexing for lifelong localization is a key component for a large panel of applications, including robot navigation, autonomous driving or cultural heritage valorization. The principal difficulty in long-term localization arises from the dynamic changes that affect outdoor environments. In this work, we propose a new approach for outdoor large scale image-based localization that can deal with challenging scenarios like cross-season, cross-weather and day/night localization. The key component of our method is a new learned global image descriptor, that can effectively benefit from scene geometry information during training. At test time, our system is capable of inferring the depth map related to the query image and use it to increase localization accuracy. We show through extensive evaluation that our method can improve localization performances, especially in challenging scenarios when the visual appearance of the scene has changed. Our method is able to leverage both visual and geometric clues from monocular images to create discriminative descriptors for cross-season localization and effective matching of images acquired at different time periods. Our method can also use weakly annotated data to localize night images across a reference dataset of daytime images. Finally we extended our method to reflectance modality and we compare multi-modal descriptors respectively based on geometry, material reflectance and a combination of both. %G English %2 https://hal.science/hal-02912239/document %2 https://hal.science/hal-02912239/file/journal_paper_last.pdf %L hal-02912239 %U https://hal.science/hal-02912239 %~ UNIV-BOURGOGNE %~ CNRS %~ UNIV-EVRY %~ IBISC %~ UNIV-PARIS-SACLAY %~ UNIV-EVRY-SACLAY %~ IMVIA %~ VIBOT %~ IBISC-SIAM %~ TEST-HALCNRS %~ UNIVERSITE-PARIS-SACLAY %~ GS-ENGINEERING %~ GS-COMPUTER-SCIENCE %~ GS-LIFE-SCIENCES-HEALTH %~ IGN-ENSG %~ UNIV-EIFFEL %~ U-EIFFEL