Imagine a robot that not only navigates your office but also remembers everything it has seen and can answer your questions about its experiences. Researchers are working on exactly that, giving robots a long-term memory to understand complex spaces over extended periods. This isn't about simple mapping; it's about building a rich history of events, objects, and even the timing of occurrences. The challenge is immense. How can a robot efficiently store and retrieve information from hours of continuous operation? The team at NVIDIA has developed a system called ReMEmbR (Retrieval-augmented Memory for Embodied Robots) that tackles this. ReMEmbR is designed for long-horizon video question answering in robot navigation. It works in two phases: memory building and querying. During memory building, ReMEmbR uses video captioning to describe short segments of what the robot sees and then stores these captions, along with location and time data, in a searchable database. When you ask a question, ReMEmbR uses an AI agent to understand your query and retrieve relevant information from its memory. It then formulates an answer, including specific location coordinates and time references, which are essential for robots to act on the retrieved information. To test ReMEmbR, the researchers created a new dataset called NaVQA (Navigation Video Question Answering). This dataset contains various types of questions related to location, time, and descriptions of events, like "Where did you see my phone?", "When did you see the boxes fall?", or "Was the sidewalk busy today?". Tests on NaVQA show that ReMEmbR can effectively answer complex questions, even across long video sequences up to 20 minutes. The system was also tested on a real robot navigating an office space. It successfully handled diverse instructions, including ambiguous queries like "Take me somewhere with a nice view." Interestingly, these robots can learn to interpret “nice view” by searching for open spaces, large windows, and plants. This research isn't just a cool demo; it represents a significant step towards more intelligent, interactive robots. Imagine robots that can give detailed reports of their operations, understand contextual information, and perform complex, time-sensitive tasks in ever-changing environments. Future work includes integrating additional data sources like semantic maps and addressing how to handle situations with multiple valid answers to a question. The team is also exploring ways to make the memory-building process even more efficient by focusing on aggregating only crucial information, which becomes even more important as the robot’s operational time extends. This kind of innovation is bringing us closer to the day when robots truly understand and interact with the world around them, not just as navigators, but as active participants with a memory of their own.