Bounded in-process state

StandardBudget (per-user usage totals) and InMemoryMemory (per-user working blocks) keep per-user_id state in process. Without bounds, a runaway tenant or a flood of one-off user_ids (for example, someone sending one request per random UUID) grows that state until the process runs out of memory. Both are bounded by default with LRU and idle-TTL eviction; tune the bounds when you know your tenant count, or turn them off for a fixed set of users.

from loomflow import InMemoryMemory
from loomflow.governance.budget import BudgetConfig, StandardBudget
 
# Defaults: 100k users, 24h idle TTL.
budget = StandardBudget(BudgetConfig())  # implicit bounds
memory = InMemoryMemory()                # implicit bounds
 
# Tune for your workload:
budget = StandardBudget(
    BudgetConfig(),
    max_users=10_000,                # smaller cap for known tenant size
    user_idle_ttl_seconds=3_600,     # drop idle users after 1h
)
memory = InMemoryMemory(
    max_users=10_000,
    user_idle_ttl_seconds=3_600,
)
 
# Or disable bounding entirely (single-tenant or fixed N tenants):
budget = StandardBudget(BudgetConfig(), max_users=None, user_idle_ttl_seconds=None)

What eviction does

• LRU: When max_users is exceeded, the least-recently-touched user’s bucket is dropped. For StandardBudget that means the user’s running totals reset; for InMemoryMemory their working blocks are deleted.
• TTL: A user idle longer than user_idle_ttl_seconds has their bucket dropped on the next access. Lazy: no background thread; the sweep runs on each touch.