1. Write a one-sentence goal
Describe the task and the output, rather than choosing a tool first. For example: “Prepare a draft weekly update from approved notes, with the team lead reviewing all facts before it is shared.” This is an illustrative brief, not a client result.
2. Keep the boundary visible
Write down what the pilot may do and what it must not do. A draft-only test should not send emails, change records or trigger decisions. Choose invented examples first and keep the existing process available.
3. Compare the whole task
| Measure | What to record |
|---|---|
| Total effort | Preparing inputs, drafting, reviewing, correcting and finishing. |
| Quality | Wrong facts, missing items, misleading wording and unsupported claims. |
| Consistency | Whether repeated trials keep the required structure and important details. |
| Team experience | Whether the reviewer understands and trusts the process enough to use it appropriately. |
4. Include difficult examples
Test incomplete inputs, contradictory notes and questions without an answer. A polished draft can still be wrong. Record whether uncertainty is clearly marked and whether the reviewer can spot problems.
5. Make a decision, not a promise
Agree a review date and choose: continue within the same boundary, revise and retest, or stop. Do not extrapolate a few examples into a savings claim. Wider deployment needs a separate decision.
Your pilot brief
Task and current problem: Owner and reviewer: Approved test inputs: Expected draft output: Actions explicitly excluded: Current baseline: Quality and consistency checks: Stop criteria: Manual fallback: Review date: Decision and evidence: