AnswersPublished18 sources
Can AI Agents Remember What They Saw? Zhejiang University’s EMem-Bench Puts Them to the Test
Zhejiang University’s OmniAI team created EMem Bench, a 2,554 episode benchmark for testing whether embodied agents can retain and use past interactions. The benchmark tests four abilities: recalling earlier observations, tracking changes, remembering interaction feedback and applying past experience to new tasks.
Published byEdited with GPT-6 LunaImages generated with GPT Image 2
Research answer
Key takeaways
- Zhejiang University’s OmniAI team created EMem Bench, a 2,554 episode benchmark for testing whether embodied agents can retain and use past interactions.
- The benchmark tests four abilities: recalling earlier observations, tracking changes, remembering interaction feedback and applying past experience to new tasks.
- In a review of 100 selected failure trajectories, memory related problems accounted for 91% of cases.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: What is Zhejiang University’s OmniAI Group’s EmbodiedMemory-Bench (EMem-Bench), reported on October 18, 2026, when was its paper posted to a. Article summary: EmbodiedMemory-Bench (EMem-Bench) is Zhejiang University’s OmniAI Group’s benchmark for testing whether embodied agents can build and update memories from past interactions, then use those memories to complete later task. Topic tags: general, academic, general web, government, documentation. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, water
An AI agent working in a home-like environment needs more than a good view of what is in front of it. It also has to remember what it saw earlier, keep track of what changed after an action and use that history to decide what to do next.
That is the challenge behind EmbodiedMemory-Bench, or EMem-Bench, from Zhejiang University’s OmniAI team. The benchmark contains 2,554 executable interaction episodes and tests whether an agent can build or update memories from past experience, then use them to complete a later task by acting in its environment. The paper was submitted to arXiv on September 23, 2026.1
17
Why memory matters for household robots
A robot might spot an object on a counter but fail to remember where it was when asked to retrieve it later. Or it might act on an outdated assumption after the object has been moved. In longer tasks, it also needs to remember what an earlier action revealed—and carry useful lessons over to a new situation.
To examine these kinds of problems, the team manually reviewed 100 failed trajectories from Gemini-3-Flash and Qwen3-VL-32B on EmbodiedBench’s EB-ALF and EB-Hab suites. The review grouped the failures into four memory-related categories:
- Forgotten visual details: 35%. An object had appeared in view, but the agent did not retain its location and later searched for it repeatedly.
- Missed changes in the environment: 18%. An object had been moved or handled, yet the agent continued to act on its previous state.
- Forgotten interaction feedback: 31%. The agent did not retain information revealed by an action’s outcome, such as whether a grasp succeeded.
- Failure to transfer experience: 7%. The agent had received a demonstration or correction but repeated the same mistake with another object.
Together, these categories account for 91% of the reviewed failures. That figure describes a selected set of failed trajectories, not the overall failure rate of either model—and the sample was not presented as a random set of runs.
Four ways to test an agent’s memory
EMem-Bench gives an agent a chronological, multimodal interaction history and then asks it to complete a later task. The agent has to use the history to act in the environment, so success depends on more than simply answering a question about the past.1
11
The benchmark’s four task families test whether an agent can:
- Recall earlier observations, including visual details such as where an object was seen.
- Track changes, updating its memory when an object’s location or state changes.
- Remember interaction outcomes, including information revealed by an earlier action.
- Apply past experience to a different object or scene.
11
16
The point is to test memory through what the agent does next. If it has retained useful information, it should be able to act on it in the follow-up task.1
EMem adds an external memory system
Alongside the benchmark, the team introduced Embodied-Memorizer (EMem), an external memory system that organizes experience into scene, spatial and event memories. It also trained EMem-8B, a model designed to manage and use those memories.1
6
15
In a comparison using GPT-5.4-mini as the shared underlying model, the reported average success rate was 35.3% without EMem and 58.9% with it—a gain of 23.6 percentage points. TeleMem scored 37.8% in the same comparison.6
Those figures describe performance in a particular benchmark setup. They do not establish that EMem will deliver the same gains across every model, task or real-world deployment.
A preprint, media coverage and reported downloads
The paper appeared on arXiv on September 23, 2026.17 Chinese media coverage followed on October 18, and the OmniAI team’s page also lists coverage of the work.
6
13
An October report said that EMem-Bench had received nearly 30,000 downloads on Hugging Face roughly three weeks after its release. That is a historical figure reported at the time, not a verified current download total.6
What simulation results can—and can’t—show
EMem-Bench evaluates interactive memory tasks in simulated environments.1
4 Its results offer evidence about how systems perform on those tests, but simulation scores alone cannot show whether a robot will be equally reliable amid real-world sensor noise or execution errors. That would require validation with physical robots.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "Can AI Agents Remember What They Saw? Zhejiang University’s EMem-Bench Puts Them to the Test"?
Zhejiang University’s OmniAI team created EMem Bench, a 2,554 episode benchmark for testing whether embodied agents can retain and use past interactions.
What are the key points to validate first?
Zhejiang University’s OmniAI team created EMem Bench, a 2,554 episode benchmark for testing whether embodied agents can retain and use past interactions. The benchmark tests four abilities: recalling earlier observations, tracking changes, remembering interaction feedback and applying past experience to new tasks.
What should I do next in practice?
In a review of 100 selected failure trajectories, memory related problems accounted for 91% of cases.
Sources
arxiv.orgBenchmarking Embodied Memory for Long-Horizon ...
163.com具身智能体长程记忆：看见不等于记住，浙大提出具身记忆基准