All 30 phases › Eval engineering
Phase 5-6 · Chapter XXVII · World 5, Intelligence Range · Kata
Eval engineering
What to study in Eval engineering, in this order: 3 books from first step to reference, 1 AI skill to use at work and the people worth following.
Reading path, in order
1. Start here AI Engineering Chip Huyen See on Amazon
2. Go deeper Reliable Machine Learning Cathy Chen, Niall Richard Murphy et al. See on Amazon
3. Reference Trustworthy Online Controlled Experiments Ron Kohavi, Diane Tang, Ya Xu See on Amazon
AI skills for everyday work
/eval-set Changing the prompt and hoping it got better.
Builds test cases from real usage Defines a clear pass criterion Measures before and after every change Keeps the regressions it finds How to ask: Use the skill eval-set on the customer support assistant.
Play this phase on the map
Dev Saga is a pixel-art world map with the 30 phases of the journey to Staff Engineer. Track what you read and used, on your own or with a book club, and export your progress to show your leadership.
Open phase 5-6 on the map Bring it to my team
← 5-5 · AI engineering
5-7 · AI governance →