Token-budgeted injection

At the start of every run the agent injects recalled memory — working blocks, facts, episodes — as a SYSTEM block above the chat history. Historically that injection used item-count limits (5 facts / 3 episodes / all working blocks) with no token cap: one oversized episode could blow the context.

Tuning(memory_token_budget=) puts a hard token ceiling on the whole memory block, with optional recency decay:

from loomflow import Agent, Tuning
 
agent = Agent(
    "Answer using what you know about the user.",
    model="gpt-4.1-mini",
    memory="sqlite:./bot.db",
    tuning=Tuning(
        memory_token_budget=1500,          # cap the seed memory block
        memory_decay_half_life_days=30.0,  # optional: prefer fresh memories
    ),
)

What the budget covers

Three memory layers feed the seed, and the budget treats them differently:

1. Working blocks are pinned. They are injected first, count against the budget, but are never dropped or truncated — pinned context (a user profile block, say) is pinned. Oversized blocks shrink the recall allowance instead.
2. Facts + episodes fill the remainder. They are merged into one list, scored, and greedily packed into whatever the working blocks left over — rendered as a single Recalled memory (most relevant first): block.
3. Session history is untouched. The budget governs cross-session recall only; this conversation’s rehydrated turns are separate (see Run vs session); mid-run transcript growth is auto-compact’s job, not this budget’s.

Scoring: relevance × decay

Each candidate item is scored relevance * decay:

• Relevance comes from recall_scored() when the backend supports it — the same BM25 + vector + RRF ranking used everywhere else. Backends without it fall back to plain recall with a neutral relevance of 1.0.
• Decay is exponential: 0.5 ** (age_days / half_life_days). A 30-day half-life halves an item’s score every 30 days. Facts decay on recorded_at, episodes on occurred_at; items without a timestamp never decay. memory_decay_half_life_days=None (the default) means no decay — pure relevance ordering.

Packing is greedy, best-score-first. The first item that doesn’t fully fit is truncated to the remaining allowance with an …[truncated] marker, and packing stops there. The top-scored item is always included — truncated if even it doesn’t fit — so a tiny budget still surfaces the single most relevant memory rather than nothing.

# Fresh episode outranks a year-old one at equal relevance:
# score(fresh, 1 day)   = 1.0 * 0.5 ** (1/30)   ≈ 0.977
# score(old, 365 days)  = 1.0 * 0.5 ** (365/30) ≈ 0.0002

Token costs use the framework’s chars/4 heuristic — the same estimator family used for tool-result caps and tool-def budgeting.

None = legacy parity

With memory_token_budget=None (the default) the legacy path runs unchanged: item-count limits, partitioned Known facts: / Relevant past episodes: headers, byte-identical seed messages. Setting a half-life without a budget also changes nothing — decay only activates under a budget. Upgrades are zero-risk until you opt in.

Tuning()                                    # legacy behaviour, exactly
Tuning(memory_decay_half_life_days=30.0)    # still legacy — no budget set
Tuning(memory_token_budget=1500)            # budgeted, no decay

Choosing a budget

A useful mental model: the budget is a cap and a ranking policy, not a retrieval expansion. The fetch limits are unchanged (5 facts, 3 episodes per seed) — the budget decides how much of what was fetched actually ships, and in what order.

For what working blocks, facts, and episodes are, see the Memory overview and Bi-temporal facts.