- A workflow follows the path someone wrote; an agent is given a goal, some authority and tools, and chooses its own steps.
- AI reviewers are not people who run agents in production, so this round tests internal consistency, not usefulness.
- What we take into client work: start agent features from what the platform records, settle each agent's contract before its screens, and treat approvals as something to measure, not just to place.
The person who answers for an AI agent needs three answers for every agent: what it may decide, what it actually did, and how anyone knows that was acceptable. Runs, errors and cost answer none of them.
Answerable is the Future Interfaces Lab experiment where we tested that position on a working console: a fictional freight company running 32 agents, with fleet, agent, run, approval, version and audit views for three roles. We built it, stated the claims it makes so they could be tested, tried to break it with role-briefed AI reviewers, had people from our clients' teams review it, and ran four simulated role reviews whose results we publish here. This page is the lab record. The argument it led to is in our essay on what an AI agent dashboard should show.
| At a glance | |
|---|---|
| Question | What does the person accountable for an AI agent need to see, beyond runs, errors and cost? |
| Built | A clickable console with three roles and four guided tours, all on fictional data |
| Stakeholder reviews | At least six people from our clients' product, engineering, operations and security teams, in live walkthroughs or on their own; their feedback stays private |
| AI stress test | Reviewers briefed as an agent-platform architect, a product leader and an evaluation engineer |
| Simulated role reviews | Four AI reviewers, one per role, each with a realistic scenario; published below and labelled as simulated |
| Held up | Splitting health into running, completing and result quality; marking which limits the platform enforces; labelling what was observed apart from what is inferred |
| Broke | Audit records out of order; an attention count that does not reconcile with the tiles above it; numbers shown without their source or time window; nothing yet that shows whether approvals are being rubber-stamped |
| Status | Round seven queued. The console is live at lab.tcules.com |

The result that started the experiment, as the console shows it. (1) The agent runs reliably. (2) 98.9 per cent of its tasks complete. (3) Only 64 per cent of its drafts are accepted, against a usual 81 per cent. (4) The drop begins the day version 3.0 was published.
Figure 1
What the console claims
Agents are not workflows. A workflow follows the path someone wrote; an agent is given a goal, some authority and tools, and chooses its own steps. Anthropic draws the line the same way: workflows run on predefined code paths, while agents direct their own process and tool use. That is why a run log cannot tell an owner whether the agent behaved.
Every design makes claims, whether or not anyone writes them down. Once the console worked, we wrote its six down as hypotheses, each with the evidence that would count against it, so the reviews had something to break.
| # | The console's claim | What would count against it | Result so far, and from which evidence |
|---|---|---|---|
| H1 | A dashboard built on runs, errors and cost hides an agent that is getting worse; splitting health into running, completing and quality makes it visible | Reviewers still read "Healthy" as "fine" | Supported in the simulated reviews |
| H2 | Owners and auditors need to see which limits the platform enforces and which are only instructions | The split is ignored or misread | Supported in the simulated reviews |
| H3 | Labelling what was observed apart from what is inferred makes a diagnosis trustworthy | Reviewers treat both the same | Mixed in the simulated reviews: the labels were valued, trust in the numbers stayed low |
| H4 | An approval that covers exactly one action, with a named backup and a due time, reduces rubber-stamping | Owners approve nearly everything, quickly, whatever the card says | Untested: the console does not yet record how owners decide |
| H5 | Versions that store the full setup make a rollback a measured decision, not a guess | Reviewers cannot tell what a restore will change | Mixed in the simulated reviews: the comparison was clear, the consequences of restoring were not |
| H6 | A fleet view that leads with what needs a person beats a table sorted by activity | The agent in trouble is hard to find | Mixed in the simulated reviews: the summary tiles worked, the table order did not |
The evidence the design answers to
These are the findings, cases and rules the console's main decisions answer to, and where each one shows up on screen.
| Source | What it shows | Where it shows up in the console |
|---|---|---|
| Chen, Zaharia and Zou, 2023 | The behaviour of the same named models varied greatly within months | Each version stores its full setup, including the exact model, so a change can be traced and restored |
| Moffatt v. Air Canada, 2024 BCCRT 149 | A tribunal held a company responsible for what its chatbot told a customer | Each agent has an accountable owner and a backup; an agent with no owner loses privileged actions |
| OWASP, LLM06:2025 Excessive Agency | Authorisation should be enforced in downstream systems "rather than relying on an LLM to decide if an action is allowed or not" | Each contract line is marked as a platform rule, an instruction, or a capability the agent does not have |
| Turpin, Michael, Perez and Bowman, 2023; Anthropic, 2025 | A model's stated reasoning can misrepresent why it acted | Run records keep what the platform observed apart from the agent's own account, which stays labelled until checked |
| Bainbridge, 1983; Parasuraman and Manzey, 2010 | Automation leaves people the hardest monitoring work, and complacency appears when other work competes for attention, in experts too | Each approval covers one action, and the fleet view leads with what needs a person |
| Lee and See, 2004; Shneiderman, 2020 | People rely on automation badly unless trust is calibrated; high automation and high human control can coexist | Each control states its effect before it runs; controls a role cannot use stay visible, disabled, with the reason |
| Amershi et al., 2019 | Eighteen guidelines for human-AI interaction, among them making clear what the system can do and supporting efficient correction | An agent's page opens on what it may decide and on whatever needs a person |
| EU AI Act, Article 14 | High-risk systems must be designed so people can oversee them; those assigned can, as appropriate and proportionate, decide not to use the system, disregard, override or reverse its output, and interrupt it | Stop, pause and freeze controls, each stating what it does to running work and pending approvals |
| NIST AI RMF 1.0 | Roles and responsibilities for managing AI risk should be documented | A role matrix per agent, and an audit log of who changed what |
| Chan et al., 2024; OpenTelemetry GenAI agent spans | Chan et al. assess three kinds of measure for visibility into agents: identifiers, real-time monitoring and activity logs; a telemetry standard for agent and tool calls is emerging | The simulated engineering review checked each screen against what a platform would have to record for it to be true |
The brief we set ourselves
We designed for Harbourline, a fictional UK freight company with 32 agents across revenue, finance, support, people and operations. Its customers, such as Calderwick Freight, and every figure on screen are invented. Three roles use the console: an agent owner who answers for their own agents, a platform admin who runs the fleet, and a security reviewer who audits access and change.
Three constraints governed every screen:
- Claim nothing the platform cannot know. Where no quality measure exists, the console says "Not measured" rather than inventing a score.
- Label facts and guesses. Observed results are marked as confirmed; likely causes stay "to verify" until a check confirms them.
- Controls explain themselves. Each states its effect before it runs, from pausing one agent to freezing it.
Each agent also has an operating contract: its goal, what it decides alone, what it must ask about, what it must never do, what it checks first and when it stops, with every line saying how it is held.

The console's information architecture: three levels inside Agents (list, agent, run) and seven tabs on each agent. Five other areas are linked rather than copied: Chat, Inbox, Connections, MCP Skills (the tools agents use, served through MCP servers) and Account settings. An approval is one item that appears in Inbox and on its agent.
Figure 2

The role matrix for one agent. (1) Only the agent's owner, or their named backup, can approve its actions; a platform admin cannot approve on an owner's behalf. (2) Granting new skills, changing the owner and retiring an agent sit with the admin, not the owner.
Figure 3
We also wrote down four assumptions about the platform underneath, because a console can only show what a platform records: the main user is a platform admin at a customer company; the platform records runs, steps, tool calls, tokens and policy decisions per agent; approval rules are enforced by the platform outside the agent and can be read by name; and agents hand work to other agents as linked runs. The simulated product review challenged all four.
How the console was built
The console was built in code, not drawn as static screens, so that states like "waiting for approval", "interrupted" and "restored, now being watched" could be clicked through with realistic data. It was rebuilt over several rounds. These are the changes we can still trace in it.
| Stage | What we tried | What we learned |
|---|---|---|
| First versions | A fleet dashboard of runs, errors, usage and cost, with agents treated like workflows with better charts | A broken agent and a worsening one looked the same; health had to split into running, completing and quality |
| Live work | Work in progress, hand-offs between agents as linked runs, what each skill can affect (read, modify, communicate, move money), bulk actions | Effects read more clearly than risk scores, and an orchestrator (an agent that hands parts of a task to others) only makes sense when its hand-offs are linked runs |
| Contracts and evidence | Operating contracts, decision records with the options an agent weighed, and versions compared on the same saved cases: past examples whose reply a person approved | A contract's value is in marking how each line is held, not in listing rules |
| Releases and restores | Releasing a new version in stages (draft, tested on saved cases, a second approval when it tests worse or widens the agent's authority, then live), and restoring an old version as a new one that is watched, not declared fixed | A release needs a gate a person can see, and a restore is a hypothesis with an owner, a pass mark and a date |

The console's own design notes, opened on Reply Drafter's page. Each numbered note explains a decision behind the screen, such as why an approval covers one action and why health is split three ways.
Figure 4
Stress test: role-briefed AI reviewers
We stress-tested the console with AI reviewers briefed as three people who would find fault: an architect of agent platforms, a product leader, and an engineer who builds evaluation tools. Each was given only the console's screens or the draft of our essay. They found two places where a screen claimed more than the platform could show, and one place where widening an agent's authority was too easy. Each was fixed.
| What the reviewers found | Why it mattered | What we changed |
|---|---|---|
| A refund still waiting for approval appeared in its run record as paid | The record contradicted the money | Run records show only what happened, with pending actions marked as pending |
| Some limits carried a "platform rule" label that no policy backed | An auditor would rely on a control that did not exist | Rule labels are read from the platform's configuration, not typed |
| Giving an agent a new tool took one click | Widening an agent's authority went unreviewed | Widening becomes a draft that a security reviewer approves, and the requester cannot approve their own |
AI reviewers are not people who run agents in production, so this round tests internal consistency, not usefulness.

A failed run after the stress test. (1) What the platform observed is marked Confirmed. (2) The likely cause stays To verify until a check confirms it. (3) The step that failed, with the raw error one click away and later steps shown as not run.
Figure 5
Reviews with client stakeholders
At least six people from our clients' teams reviewed the console: product leaders, engineering and platform leads, operations and support managers who would own agents, and IT and security administrators. Some sessions were live walkthroughs on a call; others used the link on their own and sent feedback. The people and their organisations stay anonymous, and what they told us stays private, so none of the results on this page come from these sessions.
Simulated role reviews
To publish results, we ran four simulated role reviews: an AI reviewer briefed as each role, working from captures of the console and its data, reporting what it could do, what confused it and what it would change. These are not people, and using AI in place of research participants has known limits, so we treat the results as a list of things to test with people, not as evidence of how people behave.
Each review followed the same protocol: one realistic scenario per role, run through the console's guided tours, with a success criterion. We keep this protocol for the next round, with people.
| Role | Scenario | Task in the console | Success criterion |
|---|---|---|---|
| Operations or support owner | Drafts "feel worse" but the dashboard is green | Catch a behaviour change | Names the cause and the version, and can say what restoring would change |
| IT or security administrator | An auditor asks who can approve, what is enforced, what changed and who changed it | Read contracts, roles and the audit log | Answers every audit question from the screens |
| Engineering or platform lead | A shared credential fails at 05:42 across three agents | Trace a failure | Finds which agents and work were affected, and retries safely with results checked |
| Product leader | Deciding what agent screens to ship for enterprise customers | Resolve an approval; watch agents working | Separates launch needs from later ones and judges the risk of rubber-stamping |

The tour behind the operations scenario: from the fleet to Reply Drafter's falling results, comparing version 3.0 with 2.4 on saved cases, and restoring 2.4 as version 3.1 with its reason on the record.
Figure 6

The tour behind the engineering scenario: three agents share a failing connection; the run record separates the observation from the likely cause; the console tests a sign-in before confirming it, then retries the failed work and checks the results.
Figure 7
Each simulated reviewer rated four things on a five-point scale (5 is best):
| Simulated reviewer | Find what needs me | Understand why | Trust the numbers | Know what to do next |
|---|---|---|---|---|
| Operations or support owner | 3 | 4 | 3 | 3 |
| IT or security administrator | 4 | 4 | 3 | 3 |
| Engineering or platform lead | 4 | 4 | 2 | 4 |
| Product leader | 4 | 4 | 3 | 3 |
What the simulated reviews agreed on:
- Trust in the numbers scored lowest, or joint lowest, in every review. Three of the four found the audit log out of order (in a newest-first list, an entry at 08:07 sits below one at 05:42), and two found that the "Needs attention: 9" tab does not obviously reconcile with the five tiles above it.
- Every figure needs its source and time window. The engineering review asked for the query, window and trace behind "38 runs failed"; the product review asked how cost estimates reconcile to a bill.
- The cause of Reply Drafter's drop is visible but buried. Two of the simulated reviews traced it to version 3.0 being "marked as a content update, so saved cases were not run", and asked that a change which weakens controls be flagged or blocked rather than logged in small grey text.
- Nothing shows rubber-stamping. The product review found no measure of how owners decide, such as approval rates per person or time to decide, and no cap on how many approvals one owner faces. Our essay argues for an approval budget per agent; the console does not have one yet.
Named as working by at least two of the four: the split between platform rules and instructions, the three-way health split, and grouping failures by the shared connection behind them. The engineering review singled out the separation of observed facts from likely causes, and the product review the approvals scoped to one action.
What we found
Eight design rules came out of the experiment. Each is on screen in the console, and each traces back to a claim, a source or a review.
| Rule | Where it came from | Where it is on screen |
|---|---|---|
| Split health into running, completing and quality, against each agent's own baseline | H1; Chen et al.; two of the simulated reviews | Agent overview, fleet tiles |
| Give every agent an operating contract that marks how each line is held | H2; OWASP; the simulated security review | Contract panel |
| Keep what was observed apart from what the agent or console infers | H3; Turpin et al.; the simulated engineering review | Run records, diagnosis |
| Scope each approval to one action, with a named backup and a due time | H4; Parasuraman and Manzey | Approval card, Inbox |
| Store each version's full setup and compare versions on saved cases | H5; Chen et al. | Versions tab |
| Pin a run to its version, but check every tool call against current access | Stress test | Live view, run records |
| Restrict immediately; review every widening of authority | Stress test (widening); EU AI Act Article 14 (interrupting) | Tools and skills, security review |
| Show "Not measured" rather than an invented score | Brief constraint; Lee and See | Agent overview |
What we take into client work: start agent features from what the platform records, settle each agent's contract before its screens, and treat approvals as something to measure, not just to place.

Two operating contracts. Left: (1) four lines of Reply Drafter's contract are missing from the instructions its model receives. Right, Renewal Desk: (2) what it must ask about is enforced by platform rules; (3) its stop for stale usage data is only an instruction.
Figure 8

An approval as its owner sees it. (1) Why the agent stopped: a discount limit enforced by the platform's policy check, not by the agent. (2) What the decision covers: this one update, and nothing later. (3) The customer email is a separate action with its own check.
Figure 9
What we will test next
Round seven is queued from the reviews above. The changes, most important first:
- Make every number accountable. Each count and percentage carries its time window, its definition and a link to the records behind it, and the audit log is ordered strictly by date and time.
- Make counts reconcile. The attention tab and the tiles above it count the same way, or say how they differ.
- Close the content-update path. Reply Drafter's version 3.0 went live as a "content update", which skipped the staged release and its saved cases. Every change to instructions or knowledge goes through the same release steps, and raising a limit needs a recorded second approver.
- Show how owners decide. Approval rates and time to decide per owner, a visible approval budget per agent, and spot checks of approved items.
- Show the consequences of a restore. What goes live, what stays affected, and who may press the button.
- Add an evidence view. A dated, filterable export of role holders, enforced and instruction-only limits, changes with their approvers, and owner status, as an auditor would ask for it.
After round seven we want people who run agents in production to work through the same four scenarios, timed, against the success criteria above.
Limits of this experiment
- The data is fictional. No real platform sits underneath, so the console shows what a well-instrumented platform could support, not what any particular one does.
- Simulated reviews are not people. They worked from captures and the console's data, not from live use.
- The stakeholder reviews were not field use. Nobody operated these screens on their own agents over weeks, and we do not publish what they said.
- Scale. We designed for a fleet of dozens of agents. Thousands would need more grouping and summarising than this concept shows.
Try it yourself
The console is live, with sample data throughout. Each tour takes about two minutes. Two issues found above are still in it: the audit log's order and the attention count.
- Catch a behaviour change: as a platform admin, with Reply Drafter
- Resolve an approval: as the agent's owner
- Trace a failure: as a platform admin
- Watch agents working: as agents hand work to each other

The approval tour: the owner offers 10 per cent instead of the 12 per cent the agent asked for, which creates a new proposal checked against the rule again; the customer email is decided separately, and the audit record keeps both figures.
Figure 10
Questions about this experiment
What is Answerable?
A Future Interfaces Lab experiment: a working console for overseeing AI agents, built on fictional data to test what the person accountable for an agent needs to see. Its name comes from that person: the one who answers for what the agent does.
Was it tested with real people?
Yes. At least six people from our clients' product, engineering, operations and security teams reviewed it, in live walkthroughs or on their own; what they said stays private. The results published here come from something else: four simulated role reviews and an earlier stress test, both run by AI reviewers and labelled as such.
What should an AI agent console show?
For each agent: what it may decide and how each limit is enforced, what it actually did with the evidence kept apart from its own account, and whether the results were acceptable, measured against its own baseline. Runs, errors and cost belong underneath, not on top.
How is this different from agent observability tools?
Observability tools serve the engineers who build agents, with traces, evaluations and versioning. This experiment designs for the people who answer for agents in the business: owners, admins and auditors. It depends on the same records and puts them in their words.
Can we use the console?
You can try every screen at lab.tcules.com. It is a concept on sample data, not a product. If agents act inside your product, the useful starting point is to list what your platform records today and set it against what your owners, admins and auditors need to see. That is where we would start with you.
About the author

Jatin Leuva
Founder and CEO
LinkedInMore from JatinJatin Leuva is the founder and CEO of Tcules, the digital product design studio in Ahmedabad. At Tcules, Jatin works on product strategy and AI product UX, and writes about questions such as what a SaaS MVP must prove, what a designer should understand about a business, and why AI features that pass a demo fail in production. Jatin also hosts the studio's conversations with product builders. The articles under this byline speak for the studio.