DEV SAGA

Phase 5-6 · Chapter XXVII · World 5, Intelligence Range · Kata

Eval engineering

What to study in Eval engineering, in this order: 3 books from first step to reference, 1 AI skill to use at work and the people worth following.

Reading path, in order

  1. 1. Start hereAI EngineeringChip HuyenSee on Amazon
  2. 2. Go deeperReliable Machine LearningCathy Chen, Niall Richard Murphy et al.See on Amazon
  3. 3. ReferenceTrustworthy Online Controlled ExperimentsRon Kohavi, Diane Tang, Ya XuSee on Amazon

AI skills for everyday work

/eval-set

Changing the prompt and hoping it got better.

  • Builds test cases from real usage
  • Defines a clear pass criterion
  • Measures before and after every change
  • Keeps the regressions it finds

How to ask: Use the skill eval-set on the customer support assistant.

Who to follow

Play this phase on the map

Dev Saga is a pixel-art world map with the 30 phases of the journey to Staff Engineer. Track what you read and used, on your own or with a book club, and export your progress to show your leadership.

Open phase 5-6 on the map Bring it to my team