Don't chase leaderboards. There's always a score-latency tradeoff, and LLM providers often hide it.
We reproduced Tau-bench to see for ourselves. Here's what we learned:

medium.com/alan/benchma...

There are many LLM benchmarks such as MMLU and GSM8k, but they're useless for AI agents.
Real agents need to handle database state, tool calling, and multi-turn conversations. Stateful benchmarks show the path forward.

New post on agent evaluation 👇

You've heard of the recent ISO27001:2022 certification of Alan by SGS, but want to know more about our journey towards certification? Head up to Maxime's post and enjoy the read!
medium.com/alan/our-iso...

Static chatbots couldn't handle complex support tickets about insurance claims. So we built something different with tool calls and the ReAct framework.

Our Claim Agent investigates dynamically - just like human agents, but faster. Now automating 30% of tickets it receives.

After four years at Alan from Engineering to Security, I’ve seen what makes security succeed and here is what I learned: medium.com/alan/how-i-f...

🛠️ How we tamed the "works on my machine" chaos at Alan Engineering!

Our new blog post reveals how Devbox transformed our dev experience, slashed onboarding time, and created consistent environments across our entire team.

Check it out: medium.com/alan/from-ch...

In late January, DeepSeek shocked the world by dropping an open-weight successor to OpenAI's o1: R1. Their tech report discusses how to incentivize reasoning capability in LLMs. We share our learnings at:

In 2024, we automated 20% of customer contacts with AI while maintaining the same customer satisfaction (the number is still growing!) 🚀
Learn about our journey: