LLM messages, streaming, and tool calling

Learning objectives

After this lesson, you should be able to:

• distinguish system, user, assistant, and tool messages;
• explain the difference between final text and a Tool Call returned by the model;
• describe the complete Tool Call feedback loop;
• explain how streaming deltas relate to the final result;
• read nanobot’s LLMResponse, ToolCallRequest, and tool-result messages.

Prerequisites

• Read Python dictionaries and lists.
• Know that JSON is a structured data format.
• Basic experience using an LLM is enough.

Recommended background:

• Messages, configuration, and version awareness
• Tool contracts and schema design
• Execution loop and call correlation

1. An LLM request is a role-based message sequence

A minimal example:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "List the files in the working directory."}
]

A role is part of the Provider and model protocol, not merely a presentation style:

role	Typical responsibility
system	identity, rules, tool contracts, and runtime context
user	a user request or later system-injected input
assistant	model text, reasoning-related fields, or a Tool Call
tool	the execution result for one Tool Call

nanobot’s ContextBuilder combines:

• the system prompt;
• bootstrap files from the workspace;
• Skills;
• Memory;
• Session history;
• the current user message;
• runtime metadata.

2. A Tool Call is a request to execute, not an execution

The model may return something like:

{
  "name": "list_dir",
  "arguments": {
    "path": "."
  }
}

This only means that the model wants to call a tool. The runtime still has to:

1. find the tool;
2. parse the arguments;
3. validate them against the schema;
4. check whether execution is allowed;
5. execute the tool;
6. associate the result with the correct Tool Call ID;
7. append a tool message to context;
8. call the model again;
9. repeat until the model produces final text or a stop condition is reached.

sequenceDiagram
    participant U as User
    participant A as Agent runtime
    participant M as LLM
    participant T as Tool

    U->>A: User message
    A->>M: messages + tool schemas
    M-->>A: Tool Call(name, arguments, id)
    A->>A: Parse, validate, authorize
    A->>T: execute(arguments)
    T-->>A: result / error
    A->>M: prior messages + assistant tool call + tool result
    M-->>A: Final text or another Tool Call
    A-->>U: Final response

3. Tool Call IDs preserve causality

One response may contain several tool calls. Every tool result must point back to its original call through tool_call_id; otherwise, a Provider may reject the message and the model cannot know which result belongs to which call.

nanobot:

• converts Provider output into ToolCallRequest;
• appends the assistant Tool Call message to the runtime messages;
• executes the tools;
• creates one role: tool message for each result;
• preserves the tool_call_id and tool name;
• repairs or drops invalid orphaned tool messages.

Related source:

• nanobot/providers/base.py
  • ToolCallRequest
  • LLMResponse
• nanobot/agent/runner.py
  • AgentRunner._run_core
  • AgentRunner._execute_tools
  • AgentRunner._drop_orphan_tool_results
  • AgentRunner._backfill_missing_tool_results

4. The finish reason participates in control flow

The runtime does not execute tools merely because a response contains a tool_calls field.

LLMResponse.should_execute_tools also checks whether the Provider’s finish reason permits the tool-execution branch. This avoids executing tools from a semantically inconsistent response.

Do not inspect content alone

A Provider response may also contain tool calls, a finish reason, usage, reasoning, errors, and cache-related metadata. Printing only the text can hide the actual control signal.

5. Streaming output is incremental, not a second final answer

A streaming interface emits deltas over time:

"This"
" is an"
" answer."

The caller presents those deltas to the user, but AgentRunner still needs to know when the complete response has ended so it can:

• decide whether to execute tools;
• record usage;
• save the final message;
• publish turn completion;
• close an SSE or WebSocket stream.

nanobot’s streaming paths may use:

• on_stream;
• on_stream_end;
• SDK stream events;
• API SSE;
• Channel deltas;
• WebSocket runtime messages.

These are different presentation surfaces over the same core model request and Turn state.

6. A Tool error can also be model-consumable data

Many tool errors do not terminate the entire Agent. The runtime can return a structured or readable error as a tool result, allowing the model to:

• correct its arguments;
• choose another tool;
• tell the user that the request cannot be completed;
• stop retrying after a security violation.

A security error, explicitly configured fatal tool error, or exhausted limit may still end the Turn.

When reading nanobot, distinguish this progression:

Python exception
  → ToolRunner converts it into a recoverable tool error
  → or classifies it as fatal
  → maps it to a stop_reason / final response

7. Context is not the raw Session file

Before the next model request, nanobot may:

• remove orphaned tool results;
• insert placeholders for missing results;
• compress older tool output;
• write an oversized result to the workspace and reference it from context;
• trim history;
• merge consecutive user messages;
• add runtime metadata.

When debugging, inspect three separate views:

1. messages saved in the Session;
2. initial messages built by ContextBuilder;
3. messages actually sent to the Provider on each AgentRunner iteration.

Map the concepts to source

Concept	Source
Normalized Provider response	nanobot/providers/base.py::LLMResponse
Tool Call representation	nanobot/providers/base.py::ToolCallRequest
Whether to execute tools	LLMResponse.should_execute_tools
Model-tool loop	nanobot/agent/runner.py::AgentRunner._run_core
Tool lookup and validation	nanobot/agent/tools/registry.py::ToolRegistry.prepare_call
Tool execution	AgentRunner._execute_tools, _run_tool
Tool-result normalization	AgentRunner._normalize_tool_result
Streaming Provider request	AgentRunner._request_model

Self-check

1. Why does a Tool Call usually require another model request afterward?
2. What problem does tool_call_id solve?
3. Why should the runtime not execute every response containing tool_calls?
4. If streaming deltas are already visible, why are a final response and end event still required?
5. Why might Session history differ from the history actually sent to the model in this turn?

Next step

Read Source reading and debugging minimum.