That's all we do, Tim.

myself

Context-Bench proves promising for the open source community: the gap between frontier open weights models and closed weights models appears to be closing.

Read our breakdown of the benchmark at letta.com/blog/context...

See the live leaderboard at leaderboard.letta.com

Context-Bench also measures total cost to complete the benchmark. Surprisingly, raw token costs ($/million tokens) do not map directly to total cost.

GPT-5 has lower per-token cost than Sonnet 4.5, but costs more in the benchmark because GPT-5 agents are more "token hungry".

Our goal in creating Context-Bench is to construct a benchmark that is (1) contamination proof, (2) measures "deep" multi-turn tool calling, (3) has controllable difficulty.

In its present state, the benchmark is far from saturated - the top model (Sonnet 4.5) takes 74%.

Agentic context engineering is the new frontier in AI agent capabilities. Models that are post-trained specifically for context engineering excel at long-horizon tasks where the task length far exceeds the native context window of the LLMs themselves. So which models do it best?

👋

where's that feed, kian

Watch the full 28-minute demo where @cameron.pfiffer.org walks through:

• Setting up the memory tool
• Agent self-improvement cycles
• Testing it on technical questions
• Complete memory architecture redesign