Key points
- Ask DoorDash shows grocery baskets 40% larger and half of conversations involving restaurants customers have never ordered from.
- DoorDash's inference spend is up 20x before plateauing over the past month and a half.
- DoorDash is live with DoorDash Dots in a couple of cities, though Fang says autonomy's operational problem remains unsolved.
DoorDash's Andy Fang on measuring AI against real work
DoorDash runs a physical-world logistics marketplace, and its frontier AI work is shaped by that. The company ships consumer AI features and runs an autonomy lab, but the binding constraints show up in fleet operations and data protection rather than in model capability. Fang measures progress in dollars and in his own benchmarks, because public leaderboards and lab claims do not match what his teams see.
Takeaways
- Ask DoorDash lifts baskets and merchant discovery. DoorDash launched Ask DoorDash two weeks before the conversation. Early data shows grocery baskets about 40% larger, and half of conversations involve restaurants the customer has never ordered from before.
- DoorDash built its own benchmark for real ROI. Public benchmarks are becoming more noise than signal, so DoorDash published Dash Bench, which tests models and harnesses on its internal code review agent. Fable plus Kimi K2.6 matched its previous models more cheaply.
- Inference spend rose 20x before plateauing. Inference spend is up 20x since the start of the year and has plateaued over the past month and a half. Fang watches dollar spend rather than token spend and cares about outcomes, not token maxing.
- Labs call accounting easy, accountants disagree. Fang describes a disconnect where the labs claim DoorDash's accounting tasks are easily solvable while his accounting team says models are not good enough. Stripping PII and real numbers out of RL environments is hard.
- Autonomy's technology is solved, operations are not. Cofounder Stanley has led DoorDash's autonomy lab since 2018. Fang says the technical problem is largely solved, while edge cases and fleet management are the harder part. DoorDash Dots are live in a couple of cities, and DoorDash Tasks lets Dashers pick up data work.
- Ordering through other surfaces stalled. DoorDash was an early partner on ChatGPT's app, but connecting the two was hard and the experience poor, leading to a command line interface beta. Fang says the main worry is user data protection, and the labs have not made ordering a priority.
- Internal AI gains 20-30%, not a step change. DoorDash rolled out Claude Cowork widely and saw 20-30% gains but no step change, with non-engineering teams using it to build their own agents. Engineering adoption has been organic. Every employee must still complete one dash per quarter through WeDash.
How DoorDash started
- Fang was 20 when DoorDash began as a class project called Palo Alto Delivery, open five days a week from 5 to 8 p.m. because the founders were in class.
- The idea came from interviewing hundreds of Bay Area merchants, including a macaroon shop manager with a book of delivery orders she could not fulfil.