The Orchestrator › Vocabulary › Evaluation set
Evaluation set
Cases used to test the system: common, boundary, adversarial, badly formed, and situations where refusal is the correct answer. Whatever the set contains is what will receive attention.
Discussed in the book
- Chapter 3: Delegation Is Architecture — 3.2 Contracts and grants mentioned 3×
- Chapter 4: Living with Nondeterminism — 4.2 The failure envelope mentioned 3×
- Chapter 4: Living with Nondeterminism — 4.4 Training the envelope mentioned 3×
- Chapter 6: Educating Orchestrators — 6.5 Safeguards and credentials mentioned 2×
- Chapter 2: The Instructions Are Part of the System — 2.4 Learning, traces, and authority mentioned 2×
- Chapter 3: What an Orchestrator Orchestrates — 3.7 Orchestrator mentioned once
- Chapter 4: How Orchestration Fails — 4.1 Look around mentioned once
- Chapter 4: How Orchestration Fails — 4.2 Premature overadoption mentioned once
- Search for every mention of “Evaluation set”
Related essays
- No published essay touches this term yet.