Hamel Husain is an independent AI engineer who has spent the better part of a decade teaching and researching how to build reliable LLM applications. He co-created the AI Evals for Engineers and PMs course with Shreya Shankar, which has trained more than 4,500 practitioners from over 500 companies including OpenAI, Anthropic, and Google. His primary research focus is the gap between how teams approach evaluation in theory and what the process actually needs to surface real failure modes. He draws from hands-on work with more than 50 companies to identify recurring mistakes in how AI product teams build and use evals.
Operating themes
- Data-first before automation. Husain argues that teams should build a grounded understanding of their system's actual failure modes, from examining real outputs, before configuring any automated evaluation metric.
- Tool design shapes process quality. He studies how the design of evaluation tools creates biases in how teams measure quality, often anchoring them on generic metrics rather than domain-specific failure criteria.
- Error-discovery as skill. His most recent work treats the process of finding and categorizing failures as a skill to be taught and tooled for, not a preliminary step before the "real" work of building metrics.