Human in the Loop
Human in the loop means a person has a defined decision, authority, evidence and opportunity to intervene in an AI-enabled workflow.
- AI products
- Workflow oversight
What human in the loop means
Human in the loop means a person has a defined decision, sufficient evidence, real authority and a timely opportunity to intervene in an AI-enabled workflow.
The term is about meaningful oversight, not just adding a person to a workflow after the important decision has already been made.
When human review is meaningful
A confirmation button does not create meaningful oversight if the action has already happened or the reviewer cannot inspect the basis for the action.
Human review should be placed according to consequence, uncertainty and reversibility.
- 01
Defined decision
The person needs to know what decision they are responsible for making or changing.
- 02
Sufficient evidence
The reviewer needs enough information to inspect the basis for the AI-enabled action.
- 03
Real authority
The workflow must allow the person to intervene rather than simply acknowledge an outcome.
- 04
Timely opportunity
Review needs to happen before the action is irreversible or before its consequence has already taken effect.
Review AI workflows with clear authority
Discuss where human review should sit in an AI-enabled product workflow.