TreeOfThoughts

from loomflow.architecture import TreeOfThoughts, ThoughtNode

TreeOfThoughts explores several lines of reasoning at once instead of committing to the first one. At each step it proposes a few candidate “thoughts”, scores them, keeps the best, and continues from those; at the end it writes the answer from the best chain. Use it for puzzles, math and planning problems where a single straight-line attempt often goes wrong early. It costs many more model calls than ReAct. Based on Yao et al. 2023, Tree of Thoughts: Deliberate Problem Solving with Large Language Models.

For the conceptual page see Tree of Thoughts.

Class signature

class TreeOfThoughts:
    name: str = "tree-of-thoughts"
 
    def __init__(
        self,
        *,
        branch_factor: int = 3,
        max_depth: int = 3,
        beam_width: int = 2,
        solved_threshold: float = 1.0,
        min_score: float = 0.0,
        parallel: bool = True,
        proposer_prompt: str | None = None,
        evaluator_prompt: str | None = None,
        synthesize_final: bool = True,
        synthesizer_prompt: str | None = None,
        proposer_model: str | Model | None = None,
        evaluator_model: str | Model | None = None,
        synthesizer_model: str | Model | None = None,
        evaluator_decider: Any | None = None,
    ) -> None: ...

Constructor parameters

branch_factor

Candidate “thoughts” the proposer generates per frontier node, per depth. Higher = wider search; expensive. Must be >= 1.

max_depth

Maximum depth of the search tree. Root is depth 0; the tree grows to max_depth levels. Must be >= 1. The winner is the highest-scoring node of the deepest level reached (the solved node on an early exit), as in Yao et al.’s BFS (Alg. 1), so an interior step never beats the steps built on it.

beam_width

Top-k candidates kept per depth. Higher = wider beam; more LLM calls. Must be >= 1. With branch_factor=3, beam_width=2, max_depth=3, expect about 2 × (1 + 3) × 3 = 24 model calls, plus one closing synthesis call, before pruning and early exit.

solved_threshold

Score in [0.0, 1.0] at or above which a candidate triggers early exit, and its chain is used for the answer. With the default 1.0, only a perfect score (the evaluator’s “this is correct and final”) stops the search early.

min_score

Score floor in [0.0, 1.0] below which a candidate is dropped regardless of beam capacity, so bad branches die quickly instead of riding along just because the beam has room. 0.0 means no floor; typical production values are 0.2–0.4.

parallel

When True, proposer and evaluator calls within a level run concurrently via anyio.create_task_group. This only changes speed: the branch_factor × beam_width independent calls run at the same time instead of one after another. Set to False for deterministic test ordering or when your provider has tight rate limits.

proposer_prompt

Override the proposer’s system prompt. The default asks for branch_factor candidate next thoughts toward solving the problem.

evaluator_prompt

Override the evaluator’s system prompt. The default asks for a score in [0.0, 1.0] representing how promising the candidate branch is. The reply is read with parse_score, so a score on a scale works (score: 7/10 is 0.7, score: 40% is 0.4).

synthesize_final / synthesizer_prompt

When True, one closing model call turns the winning chain of thoughts into the final answer. When False, the raw winning thought is the output. synthesizer_prompt overrides the prompt for that closing call.

proposer_model / evaluator_model / synthesizer_model

Use a different model per role. The evaluator makes the most calls, so a cheap evaluator model saves the most. Role models are not wrapped with the agent’s retry policy. A model string is resolved with the agent’s secrets= backend, like the main model.

evaluator_decider

Score each thought with a decision model instead of an LLM evaluator call. Takes precedence over evaluator_model when both are set.

Methods

declared_workers

Returns {}. Single-agent.

run

1. Root. Depth-0 node containing the problem. No model call.
2. For each depth up to max_depth: a. Expand. For every frontier node, the proposer generates branch_factor candidates (parallel across nodes when parallel=True). b. Evaluate. The evaluator scores each candidate in [0.0, 1.0]. c. Prune. Drop candidates with score < min_score. Keep the top beam_width by score as the next frontier. d. Early exit. If any candidate scores >= solved_threshold, emit tot.solved and use that branch.
3. Best of the last level wins. The highest-scoring node of the deepest level reached (the solved node on an early exit) picks the winning chain of thoughts. With synthesize_final=True (the default) one more model call turns that chain into the final answer.

Events: tot.started, then per level tot.level_started, tot.proposed, tot.evaluated and tot.pruned. Possible early endings: tot.solved, tot.empty_beam, tot.no_thoughts. At the end: tot.synthesized (when synthesizing) and tot.completed.

Related types

ThoughtNode

class ThoughtNode(BaseModel):
    id: str
    parent_id: str | None
    content: str
    score: float = 0.0
    depth: int

Tree node. id is the node’s path in the tree ("n" for the root, then "n.1", "n.1.2", …), and each proposer / evaluator call is journaled under its node’s id, so a resumed search on a durable runtime replays instead of calling the model again. The whole tree is stored on session.metadata["tot_nodes"] after the run (and the winner’s id on session.metadata["tot_winner_id"]) for introspection. The full tree is preserved (not just the surviving beam) so you can analyze pruning decisions.

Cost model

Per depth: one batched proposer call per frontier node plus one evaluator call per candidate, so frontier_size × (1 + branch_factor) model calls. Total: about beam_width × max_depth × (1 + branch_factor), plus one synthesis call.

Real numbers for branch_factor=3, beam_width=2, max_depth=4: ~30 LLM calls vs ~5 for ReAct. Reserve for tasks where ReAct’s straight-line trajectory commits too early and produces wrong answers.

Example

from loomflow import Agent
from loomflow.architecture import TreeOfThoughts
 
agent = Agent(
    "Solve the puzzle by exploring multiple lines of reasoning.",
    model="claude-opus-4-7",
    architecture=TreeOfThoughts(
        branch_factor=3,
        beam_width=2,
        max_depth=4,
        solved_threshold=0.95,
        min_score=0.3,
    ),
)
result = await agent.run("Game of 24 with [3, 7, 8, 1].")

Or by string:

agent = Agent("...", model="claude-opus-4-7", architecture="tree-of-thoughts")

Source

loomflow/architecture/tree_of_thoughts.py