Generative Agents: memory, reflection, and planning for long-term coherence

Original source

Joon Sung Park et al., UIST 2023: Generative Agents: Interactive Simulacra of Human Behavior. This guide uses arXiv v2.

This is an original learning guide, not a reproduction. Copyright and the complete paper remain with the original source.

What you should be able to do after this guide

By the end, you should be able to:

1. Draw the complete loop from observation writing through retrieval, reflection, planning, action, and the next observation.
2. Explain what one Memory Stream record contains and why observations, reflections, and plans can share a retrieval space.
3. State the distinct retrieval failures addressed by recency, importance, and relevance, then score a simplified example by hand.
4. Explain when reflection runs, how concrete records become higher-level beliefs, and why evidence links matter.
5. Trace a coarse daily plan into detailed actions and show how a new event can revise its remaining steps.
6. Separate three claims: behavior appears believable, the system has a real personality, and a simulation represents a real society.

Start with the problem: one plausible response is not a coherent life

Given a persona, the current time, and a nearby scene, a language model can usually propose one plausible action. The hard part appears one minute later, the next day, and after repeated social interactions:

• a character may forget an invitation accepted yesterday;
• a model that reacts only to the current scene may eat lunch three times in one hour;
• the history keeps growing, so including all of it both exceeds context limits and hides relevant events;
• a list of concrete events does not automatically become a higher-level belief such as “research matters to me”;
• a new observation may conflict with the plan, so the system must decide when to continue and when to revise.

The paper targets believability: behavior should remain coherent with the character’s experiences, relationships, and current environment. Acting like a believable character is an interactive-system objective. It is not evidence of consciousness or genuine human agency.

The idea in one sentence

Generative Agents writes an expanding stream of experience in natural language, retrieves current records with recency, importance, and relevance, periodically turns concrete records into traceable higher-level reflections, and uses those memories to create, refine, and revise plans.

flowchart TD
    E["World state, other agents, and dialogue"] --> O["Observation: write a natural-language record"]
    O --> M["Memory Stream"]
    M --> R["Retrieval: recency + importance + relevance"]
    M --> G{"Accumulated importance reaches the reflection threshold?"}
    G -->|"Yes"| F["Reflection: questions → retrieval → higher-level beliefs"]
    F --> M
    R --> P["Planning: day → hour → 5–15 minute actions"]
    P --> A["Current action or dialogue"]
    A --> E
    E --> D{"Continue the plan or react?"}
    D -->|"Continue"| A
    D -->|"React"| P

The full mechanism contract: input, state, output, and termination

Part	What it means in the paper	Why it is needed
Input	Current time, perceived environment, nearby characters, persona, and current plan	Defines what the agent can actually observe now
Long-term state	Observations, reflections, and plan records in the Memory Stream	Lets past experience affect behavior instead of restarting every step
Retriever	Scores each memory on three dimensions for a current query	Avoids placing the entire history in every prompt
Reflector	Raises questions over recent important events and produces evidence-linked beliefs	Turns “what happened” into “what it may mean”
Planner	Produces plans with time, location, and duration at several granularities	Maintains coherence over longer time horizons
Reaction decision	Determines whether a new observation should interrupt the current plan	Keeps plans stable without making them rigid
Output	A natural-language action, utterance, or intended world operation	The sandbox engine converts it into movement or state change
Local termination	The action for one time slice completes and perception runs again	This is a continuing process, not a permanent stop after one answer

There are two feedback loops. The fast loop perceives, acts, and perceives again at every time step. The slow loop accumulates experience into reflections and future plans. Implementing only one does not reproduce the long-term coherence the paper studies.

Core mechanism 1: the Memory Stream stores experience that can affect later decisions

The paper implements the Memory Stream as a time-ordered list of memory objects. Its basic object is an observation: an event directly perceived by the agent. An object contains at least:

• a natural-language description, such as “Maria is studying chemistry in the cafe”;
• its creation timestamp;
• its most recent access timestamp;
• an importance score assigned at creation;
• a text representation used for relevance retrieval;
• and, for a reflection, references to the lower-level records supporting it.

Observations are not the only record type. Reflections and plans also re-enter the Memory Stream, so a later query can retrieve experienced events, higher-level beliefs, and future commitments together. The convenience creates a provenance risk: without type and source metadata, the model may treat “I infer that he cares about research” as if “he explicitly said he cares about research.”

A useful beginner schema

Start with id | type | text | created_at | last_accessed_at | importance | evidence_ids.

A vector index can propose semantically related candidates. The source table still owns time, record type, provenance, and audit. “We have a vector database” is not yet a long-term memory system.

Core mechanism 2: three retrieval scores cover three different gaps

Before a decision, the system builds a query from the current situation and scores each memory on three components.

1. Recency keeps the immediate situation in attention

The paper uses exponential decay over the number of sandbox hours since a memory was last accessed, with a decay factor of 0.995. Newly created or recently retrieved events remain prominent, while unused records gradually recede.

Recency alone creates a recent-history bias: trivial conversation from this morning can outrank a much older but critical commitment.

2. Importance protects consequential events from routine noise

At creation time, the implementation asks a language model to rate an event’s poignancy from 1 to 10. Cleaning a room may receive a low score; a major relationship change may receive a high one. Importance lets an older but consequential memory compete with many mundane observations.

This rating is not objective truth. It may depend on wording, culture, and persona. A production system should record how the value was produced and allow verified rules or human signals to override it.

3. Relevance connects the memory to the current question

The paper embeds the query and each memory description, then uses cosine similarity. When the query concerns a chemistry exam, memories about classes, the teacher, and studying should outrank breakfast.

Relevance alone also fails: a semantically similar but obsolete plan may outrank a recent change.

4. Combining and truncating the scores

The implementation min–max normalizes the three components into $[0,1]$ and computes:

$$scor{e}_i={\alpha }_r{r}_i+{\alpha }_i{i}_i+{\alpha }_l{l}_i$$

All three weights are 1 in the paper’s implementation. The system ranks records by the combined score and includes only the highest-ranked set that fits the language model’s context window.

This teaching example uses invented numbers:

Memory	recency	importance	relevance	total	Priority now?
Promised today to help decorate at 17:00	0.95	0.85	0.92	2.72	Yes
Drank a glass of water a minute ago	1.00	0.10	0.05	1.15	No
Attended a birthday party last year	0.10	0.60	0.70	1.40	Maybe
Has an exam tomorrow	0.45	0.95	0.15	1.55	Could rise under another query

Do not memorize the weights as a universal recipe. The important point is that the query and three distinct dimensions jointly determine retrieval. A bad query, miscalibrated scores, or a small context budget can make planning operate on the wrong evidence.

Core mechanism 3: reflection turns events into evidence-linked beliefs

Observations can answer “what happened,” but raw observations alone struggle with “what do these events jointly imply?” Reflection creates a second layer of memory.

When reflection runs

The implementation accumulates importance scores for recent events and triggers reflection when their sum exceeds 150. In the Smallville setting, agents reflected roughly two or three times per day. 150 is a parameter from this particular system, not a universal threshold for agent memory.

How one reflection is produced

1. Take the 100 most recent memory records.
2. Ask the model for three salient higher-level questions that those records could answer.
3. Use each question as a retrieval query over the Memory Stream.
4. Generate several higher-level insights from the retrieved evidence.
5. Require each insight to cite the record identifiers that support it.
6. Store the insight back as a reflection record.

Reflections can retrieve and cite earlier reflections. This creates a tree: direct observations are leaves, intermediate summaries occupy the middle, and increasingly abstract beliefs appear higher up.

Higher abstraction can amplify error

An incomplete observation or an embellished summary can be retrieved and summarized again. Preserve evidence_ids, make every belief auditable back to observations, and invalidate or recompute conclusions when their evidence changes.

Core mechanism 4: planning turns locally plausible actions into a coherent timeline

Asking only “what should happen now?” can produce steps that each sound reasonable but form an absurd sequence. The paper illustrates this with a character who could eat lunch at noon, 12:30, and 13:00. A plan constrains behavior across time.

A plan item has a location, start time, and duration. The system first uses the persona summary, recent experience, and a summary of the previous day to produce a coarse daily plan, usually five to eight blocks. It recursively decomposes each block into hours and then into 5–15 minute actions. The plan itself becomes a memory record.

The plan is not a locked script. At every time step, the agent perceives nearby objects and characters, stores the observation, and decides whether to:

• treat it as background and continue;
• react because it matters to the agent;
• rebuild the remaining plan from the current time because the event changes later commitments;
• or, for another agent, retrieve their relationship and shared context before generating dialogue.

The plan is the default path; observations provide interruption signals. Without a plan the character drifts, while a plan without interruption becomes rigid.

A complete run: how one party intention becomes coordinated behavior

The paper’s best-known case begins with one seed: Isabella intends to hold a Valentine’s Day party. The table separates the mechanism from the narrative effect.

Stage	What is stored or retrieved	How it changes the next step
Seed intention	“Hold a party at this place and time” becomes a memory or plan	Importance and relevance keep it available during scheduling
Meeting an acquaintance	The system perceives the person and retrieves the relationship plus party plan	The reaction decision initiates an invitation
Information diffusion	The invited agent stores the conversation in its own stream	The event now exists outside the host’s context
Daily planning	The invitee generates and refines a daily schedule	The party competes with other commitments for time
Preparation	The host schedules decorating on the preceding afternoon	A long-horizon plan coordinates several prerequisite actions
New interaction	Relationship memory and the party memory support a date invitation	Separate local loops create another social branch
Scheduled execution	Current time selects the planned place and action	Multiple characters navigate independently to the cafe
Continued memory	Arrival, conversation, and relationship changes are stored	The event can affect later retrieval, reflection, and planning

Many links can fail: the intention must be stored, the invitation expressed, the recipient’s memory retrieved, the schedule preserved, time and place interpreted consistently, and the action executed. “Emergent” does not mean “without a mechanism.” It means that the group outcome was not scripted separately for every character; it arose from interacting local memory-and-plan loops.

Pin down the fast and slow loops with pseudocode

def tick(agent, world, now):
    observation = perceive(agent, world)
    agent.memory.append(make_observation(observation, now))
 
    if recent_importance_sum(agent.memory) >= REFLECTION_THRESHOLD:
        questions = propose_questions(last_records(agent.memory, 100))
        for question in questions:
            evidence = retrieve(agent.memory, question)
            insights = infer_with_evidence(question, evidence)
            agent.memory.extend(make_reflections(insights, evidence, now))
 
    if needs_daily_plan(agent, now):
        coarse = create_daily_plan(agent.profile, agent.memory)
        agent.plan = recursively_decompose(coarse)
        agent.memory.append(make_plan_record(agent.plan, now))
 
    context = retrieve(agent.memory, query_from(observation, agent.plan))
    decision = decide_continue_or_react(agent.plan, observation, context)
 
    if decision.requires_replan:
        agent.plan = revise_remaining_plan(agent.plan, now, context)
 
    action = choose_current_action(agent.plan, decision, context)
    return world.execute(action)

The pseudocode keeps two budgets explicit: reflection should not run at every step, and retrieval should not include every memory. Otherwise the system spends its time endlessly summarizing itself or falls back into an oversized prompt.

Reading the evidence: component contribution and behavioral believability

The authors populate Smallville with 25 agents and conduct two evaluation types. Natural-language interviews probe self-knowledge, memory, planning, reaction, and reflection. A two-day sandbox run examines individual and social behavior. In the controlled evaluation, 100 participants rank responses from the full architecture and several ablations.

The full architecture receives the highest believability rating. Removing reflection lowers the rating; removing planning as well lowers it further; the condition without observational memory, reflection, or planning performs worst. This supports the claim that the components contribute under this implementation and interview design.

Evidence boundary

• The outcome is believability of character responses and behavior, not general task success or factual accuracy.
• To enable a controlled comparison, ablations receive memories from the same simulation. That is not identical to independently rerunning every architecture for two days.
• Twenty-five sandbox characters and a two-day scenario do not represent a real society. Information diffusion and coordination are demonstrations, not social-science forecasts.
• The paper reports retrieval misses, embellished memories, and overly formal speech. A memory component does not guarantee faithful recall.

The paper also discusses parasocial relationships, deepfakes, and tailored persuasion. A coherent character can encourage users to mistake believable performance for real understanding or relationship. Products should disclose artificial identity, preserve provenance logs, and avoid replacing real stakeholders with simulated ones.

Common failure modes and a useful debugging order

Symptom	Likely layer	Inspect first
The agent heard an invitation but says it did not	Retrieval miss	Query, component scores, top-k, and context truncation
It knows what to do at a party but doubts that the party exists	Incomplete fragment retrieval	Whether the original invitation was omitted and whether grouped retrieval is needed
A reflection contains a fact absent from its sources	Synthesis embellishment	Evidence IDs, generation constraints, and fact checks
The schedule repeats one activity	Plan or time-state error	Start/end time, duration, completion flags, and decomposition boundaries
An emergency appears but the plan never changes	Reaction/replanning error	Observation visibility, interruption threshold, and replanning scope
An old belief permanently controls behavior	No expiry or counter-evidence policy	Whether sources changed and conclusions can be revoked
Cost grows rapidly over time	Missing budgets or repeated summaries	Retrieval budget, reflection frequency, caching, deduplication, and archival

Historical contribution and today’s engineering interpretation

The historical contribution is a runnable combination of the Memory Stream, three-factor retrieval, recursive reflection, and hierarchical planning, accompanied by ablations and an end-to-end sandbox showing how the components jointly support coherent behavior.

Durable engineering lessons include:

• treat memory as recording, scoring, retrieval, synthesis, and invalidation—not a transcript array;
• distinguish observed facts, higher-level inferences, and future plans while preserving provenance links;
• use coarse-to-fine plans as a stable default and environmental observations as interruption signals;
• monitor retrieval misses separately from synthesis hallucinations because they require different fixes;
• put explicit budgets on memory operations and context.

The paper does not establish that its weights, thresholds, or prompts transfer unchanged; that a simulated character is a real person; or that a compelling multi-agent story reveals real social laws.

Relationship to later work

Reflexion stores why the previous trial failed and what the next trial should change. Voyager stores environmentally verified behavior as executable code skills. Generative Agents stores experiences, beliefs, and plans to maintain character and social coherence. All three use “memory,” but they preserve different objects under different success criteria.

Check whether you really understand the mechanism

1. Why is relevance alone insufficient? What distinct gaps do recency and importance fill?
2. How does the evidence status of a reflection differ from that of an observation, and how should provenance be preserved?
3. What planning levels connect “finish the research report today” to an action at 14:10?
4. If an agent forgets an invitation, how do you distinguish failure to record, failure to retrieve, and failure to use retrieved context?
5. What does the party case demonstrate, and what does it not demonstrate?

Answers and reasoning cues

1. Relevance measures semantic similarity and can favor old or trivial matches. Recency maintains immediate continuity; importance protects older but consequential events.
2. An observation records direct perception, while a reflection is an inference over records. Preserve the type and evidence_ids, allow inspection of sources, and invalidate beliefs when their support changes.
3. Generate a coarse daily plan, decompose it into hourly blocks, then into 5–15 minute actions with time, place, and duration. The remaining plan must be revisable from an interruption point.
4. Audit perception logs, Memory Stream insertion, retrieval candidates and scores, the final prompt context, and then the generated decision. Do not infer the layer from the final sentence alone.
5. It shows that interacting local memory, dialogue, and planning loops can coordinate behavior that was not scripted character by character. It does not demonstrate consciousness or validate the simulated town as a model of society.

Previous: Toolformer · Next: Reflexion · Back to the landmark reading index