PhD Position: Vision Physical and Geometric Foundation Models for Autonomous Navigation

Mission:

This PhD project, co-funded by CNES and the Occitanie Region, focuses on developing advanced autonomous navigation systems for planetary or underground environments. The project addresses challenges such as the absence of GNSS, low visibility, complex topologies, and embedded computing constraints. The goal is to create a hybrid navigation pipeline that combines visual/inertial SLAM, Vision Foundation Models (VFM), and 3D reconstruction techniques to enable robust and predictive navigation.

Context:

Autonomous robotics in extreme environments requires robust localization, accurate state estimation, and adaptive planning. Recent advancements in computer vision, such as Vision Foundation Models (VFM) and 3D reconstruction methods (NeRF, 3D Gaussian Splatting), offer promising solutions. However, these models often lack the ability to reason about physics and geometry. This thesis aims to bridge this gap by integrating geometric and physical constraints into VFMs, enhancing their applicability in real-world scenarios.

Thesis Objectives:

• Develop a hybrid navigation pipeline combining:

  • Visual/inertial SLAM enriched with deep learning modules.

  • Vision Foundation Models anchored in physics and geometry (e.g., VGGT).

  • Advanced 3D reconstruction techniques (NeRF, 3D Gaussian Splatting) for dense, semantic mapping.

• Validate the pipeline using multi-sensor data (cameras, lidars, neuromorphic cameras) and multi-environment testing (simulators and real robotic platforms).

Methodological Approach:

• Year 1: Evaluate existing VFMs and create a simulated benchmark using NeRF/3DGS to test their physical and geometric capabilities.

• Year 2: Develop hybrid SLAM + VFM modules with explicit learning of physical and geometric constraints, and integrate them into a real-time navigation pipeline.

• Year 3: Extend the pipeline to complex environments and validate it on real robotic platforms (rovers, drones). Explore multi-sensor and multi-agent scenarios.

Expected Outcomes:

• Hybrid SLAM methods integrating depth vision, physics, and geometry.

• Continuous 3D representations for navigation and robust planning.

• Publications in top-tier conferences (CVPR, ICRA, RSS, ICCV, ECCV, NeurIPS).

• Demonstrators applicable to space exploration and CNES missions.

Requirements

Experience: No preference

Education Level: Master’s degree (MBA or equivalent in robotics, computer vision, or related fields)

Skills/Qualifications:

• Strong background in robotics, computer vision, or machine learning.

• Experience with SLAM, deep learning, and 3D reconstruction techniques (NeRF, 3D Gaussian Splatting) is a plus.

• Proficiency in programming (Python, C++) and familiarity with robotic platforms.

• Ability to work independently and collaboratively in a multidisciplinary team.

• Excellent written and verbal communication skills in English.