Streaming, errors, and stop

Corresponding source

Subject	Source
Streaming, timeout, and stop branches	nanobot/agent/runner.py::AgentRunner._request_model, AgentRunner._run_core
Provider streaming and retry	nanobot/providers/base.py::LLMProvider.chat_stream_with_retry, LLMProvider._run_with_retry
Provider fallback	nanobot/providers/fallback_provider.py::FallbackProvider
Progress/reasoning hooks	nanobot/agent/progress_hook.py, nanobot/agent/hook.py
Bus stream segments	nanobot/agent/loop.py::AgentLoop._dispatch
Cancellation entry	nanobot/agent/loop.py::AgentLoop.stop, nanobot/command/builtin.py

Learning objectives

After this lesson, you should be able to:

• distinguish content deltas, reasoning, progress, and the final response;
• explain LLM wall timeout and stream idle timeout;
• understand retry, fallback, empty-response recovery, and maximum iterations;
• map a stop reason back to the responsible Runner branch.

Prerequisites

• Understand the AgentRunner main loop.
• Understand asynchronous callbacks and cancellation.
• Know the basic categories of HTTP/Provider errors.

1. “Streaming” contains several event types

nanobot can produce:

Type	Meaning
content delta	incremental final text from the model
reasoning delta	thinking/reasoning content returned by the Provider
tool hint	notice that a tool is about to run
progress	Tool or Agent lifecycle progress
retry wait	a retry delay is in progress
stream end	the current stream segment has ended
final response	final text for the complete Turn
runtime event	Turn, Goal, or model state change

These are not one Channel metadata field and should not be collapsed into one UI flag.

2. Core streaming path

sequenceDiagram
    participant P as Provider
    participant R as AgentRunner
    participant H as Hook/callback
    participant E as CLI/API/Channel

    P-->>R: content delta
    R-->>H: on_content_delta
    H-->>E: Present delta
    P-->>R: Tool Call or final completion
    R-->>H: on_stream_end
    R-->>E: Final AgentRunResult

Entry-point adapters:

• CLI: terminal renderer;
• API: SSE chunk;
• SDK: typed stream event;
• Channel: send_delta();
• WebSocket: protocol message.

3. One Turn may contain several stream segments

If the model emits content, calls a Tool, and then continues its answer, streaming can pause and resume. AgentLoop creates segment IDs for Channel streams so deltas from separate phases are not combined into the wrong message bubble.

The first stream_end therefore does not necessarily mean the entire Turn is complete. Final completion still depends on:

• Runner returning;
• SAVE;
• RESPOND;
• TurnCompleted or the corresponding runtime wire event.

4. Two primary timeouts

LLM wall timeout

Limits the total time allowed for one model request.

• Environment variable: NANOBOT_LLM_TIMEOUT_S
• Verify the default in the current AgentRunner source.

Stream idle timeout

Limits how long a stream may remain silent without producing any event.

• Environment variable: NANOBOT_STREAM_IDLE_TIMEOUT_S
• Parsed in nanobot/providers/base.py
• It is not the total request duration.

wall timeout: total wall-clock boundary from request start to end
idle timeout: maximum silence between two stream events

AgentRunner._request_model() also prevents the two timeout types from stacking onto a healthy long stream:

if wants_streaming:
    coro = self.provider.chat_stream_with_retry(...)
else:
    coro = self.provider.chat_with_retry(**kwargs)
 
# Streaming requests already have provider-level idle timeouts.
outer_timeout_s = None if (wants_streaming or wants_progress_streaming) else timeout_s
response = (
    await coro
    if outer_timeout_s is None
    else await asyncio.wait_for(coro, timeout=outer_timeout_s)
)

This is a compressed source excerpt. A non-streaming request can be limited by the outer wall timeout; a streaming request relies on the Provider’s idle timeout so a stream that keeps producing events but takes a long total time is not terminated incorrectly.

5. Provider retry

LLMProvider retry logic considers:

• HTTP status;
• error kind;
• Provider retry-after;
• retry mode;
• whether stream content was already emitted;
• caller callbacks.

Not every error should be retried. For example:

• authentication failures usually should not retry indefinitely;
• rate limits and temporary service failures may be retried;
• retrying after partial output can duplicate text for the user.

6. Fallback model

providers/factory.py::make_provider() can create a configured fallback chain. A candidate can be:

• a named preset;
• an inline Provider/model configuration.

Key points:

• fallback occurs at the Provider layer;
• primary and fallback candidates may use different models, context windows, token limits, and reasoning parameters;
• Provider/fallback implementation decides which errors trigger fallback;
• fallback does not start a new AgentRunner Session.

7. Empty-response recovery

A model request can succeed while returning empty content. Runner:

1. retries an empty response a limited number of times;
2. may attempt a finalization retry;
3. produces a normalized empty-response outcome if no final content appears;
4. lets AgentLoop SAVE ensure that a Turn requiring a response does not persist an empty final answer.

This path differs from a Provider exception.

8. finish_reason=length

If model output is cut off by length after producing nonempty content, Runner can request the remaining content, subject to a recovery limit.

Distinguish:

• empty content;
• truncated content;
• Tool Call;
• Provider error;
• maximum iterations.

9. Maximum iterations

AgentRunSpec.max_iterations bounds the model-tool loop and prevents infinite calls.

At the limit, Runner may:

• perform finalization;
• return a maximum-iteration message;
• apply continuation policy for an active Goal.

Do not hide a nonterminating loop by setting an extremely large limit. First inspect:

• whether the model repeats the same Tool;
• whether the tool error offers actionable information;
• whether the goal predicate stays true forever;
• whether injections keep arriving;
• whether the prompt lacks a termination condition.

10. Tool error

An ordinary Tool error usually returns to the model. Runner is more likely to stop for:

• fail_on_tool_error;
• an explicitly fatal error;
• repeated workspace violations;
• an unrecoverable security error;
• an exception in Runner itself.

Inspect AgentRunResult.error together with stop_reason.

11. Cancellation

Cancellation can originate from:

• /stop;
• process shutdown;
• cancellation of a parent Task;
• timeout.

On cancellation, AgentLoop must:

• stop the corresponding active Task;
• restore or preserve the checkpoint and pending user turn;
• keep the Session in a valid state;
• close MCP and other resources;
• propagate real Task cancellation correctly.

“No final response” after cancellation does not mean nothing was saved. The user message may already have been persisted.

Debugging matrix

Symptom	Observe
UI shows deltas but never ends	on_stream_end, whether Runner returned, TurnCompleted
duplicate output	whether retry occurred after deltas were emitted
long silence	stream idle timeout, reasoning stream, Provider state
Tool repeats forever	iteration, tool event, tool result, termination prompt
history disappears after /stop	pending user turn, checkpoint, cancellation recovery
content stops abruptly	finish reason, length recovery
model changes after primary failure	fallback chain, runtime model event

Source and tests

• nanobot/agent/runner.py::_request_model
• nanobot/agent/runner.py::_run_core
• nanobot/providers/base.py
• nanobot/providers/fallback_provider.py
• nanobot/agent/progress_hook.py
• tests/agent/test_runner_progress_deltas.py
• tests/agent/test_runner_reasoning.py
• tests/agent/test_runner_errors.py
• tests/agent/test_runner_fallback.py
• tests/agent/test_task_cancel.py
• tests/providers/test_stream_idle_timeout_config.py

Self-check

1. Why is stream end not always Turn end?
2. How do idle timeout and wall timeout differ?
3. Why does retry require more care after partial deltas have been emitted?
4. How does empty-response recovery differ from Provider-error recovery?
5. Why should you not respond to maximum iterations by only raising the limit?

Next step

Read Persistence, response, and runtime events.