Answerable: a lab experiment in designing oversight for AI agents

What a person accountable for AI agents needs to see: our lab experiment, its claims, evidence, reviews, results and what still fails.

  • AI Agents & Copilots
  • AI Product UX
  • AI products and developer tools
  • AI Products & Developer Tools
  • Data, Analytics & Decision Products
Insights
  • A workflow follows the path someone wrote; an agent is given a goal, some authority and tools, and chooses its own steps.
  • AI reviewers are not people who run agents in production, so this round tests internal consistency, not usefulness.
  • What we take into client work: start agent features from what the platform records, settle each agent's contract before its screens, and treat approvals as something to measure, not just to place.

The person who answers for an AI agent needs three answers for every agent: what it may decide, what it actually did, and how anyone knows that was acceptable. Runs, errors and cost answer none of them.

Answerable is the Future Interfaces Lab experiment where we tested that position on a working console: a fictional freight company running 32 agents, with fleet, agent, run, approval, version and audit views for three roles. We built it, stated the claims it makes so they could be tested, tried to break it with role-briefed AI reviewers, had people from our clients' teams review it, and ran four simulated role reviews whose results we publish here. This page is the lab record. The argument it led to is in our essay on what an AI agent dashboard should show.

At a glance
QuestionWhat does the person accountable for an AI agent need to see, beyond runs, errors and cost?
BuiltA clickable console with three roles and four guided tours, all on fictional data
Stakeholder reviewsAt least six people from our clients' product, engineering, operations and security teams, in live walkthroughs or on their own; their feedback stays private
AI stress testReviewers briefed as an agent-platform architect, a product leader and an evaluation engineer
Simulated role reviewsFour AI reviewers, one per role, each with a realistic scenario; published below and labelled as simulated
Held upSplitting health into running, completing and result quality; marking which limits the platform enforces; labelling what was observed apart from what is inferred
BrokeAudit records out of order; an attention count that does not reconcile with the tiles above it; numbers shown without their source or time window; nothing yet that shows whether approvals are being rubber-stamped
StatusRound seven queued. The console is live at lab.tcules.com
Concept AI agent dashboard, fictional data: Reply Drafter shows Healthy and 98.9% of tasks completing, but only 64% of drafts accepted against a usual 81%, with results falling from the day version 3.0 was published.

The result that started the experiment, as the console shows it. (1) The agent runs reliably. (2) 98.9 per cent of its tasks complete. (3) Only 64 per cent of its drafts are accepted, against a usual 81 per cent. (4) The drop begins the day version 3.0 was published.

Figure 1

What the console claims

Agents are not workflows. A workflow follows the path someone wrote; an agent is given a goal, some authority and tools, and chooses its own steps. Anthropic draws the line the same way: workflows run on predefined code paths, while agents direct their own process and tool use. That is why a run log cannot tell an owner whether the agent behaved.

Every design makes claims, whether or not anyone writes them down. Once the console worked, we wrote its six down as hypotheses, each with the evidence that would count against it, so the reviews had something to break.

#The console's claimWhat would count against itResult so far, and from which evidence
H1A dashboard built on runs, errors and cost hides an agent that is getting worse; splitting health into running, completing and quality makes it visibleReviewers still read "Healthy" as "fine"Supported in the simulated reviews
H2Owners and auditors need to see which limits the platform enforces and which are only instructionsThe split is ignored or misreadSupported in the simulated reviews
H3Labelling what was observed apart from what is inferred makes a diagnosis trustworthyReviewers treat both the sameMixed in the simulated reviews: the labels were valued, trust in the numbers stayed low
H4An approval that covers exactly one action, with a named backup and a due time, reduces rubber-stampingOwners approve nearly everything, quickly, whatever the card saysUntested: the console does not yet record how owners decide
H5Versions that store the full setup make a rollback a measured decision, not a guessReviewers cannot tell what a restore will changeMixed in the simulated reviews: the comparison was clear, the consequences of restoring were not
H6A fleet view that leads with what needs a person beats a table sorted by activityThe agent in trouble is hard to findMixed in the simulated reviews: the summary tiles worked, the table order did not

The evidence the design answers to

These are the findings, cases and rules the console's main decisions answer to, and where each one shows up on screen.

SourceWhat it showsWhere it shows up in the console
Chen, Zaharia and Zou, 2023The behaviour of the same named models varied greatly within monthsEach version stores its full setup, including the exact model, so a change can be traced and restored
Moffatt v. Air Canada, 2024 BCCRT 149A tribunal held a company responsible for what its chatbot told a customerEach agent has an accountable owner and a backup; an agent with no owner loses privileged actions
OWASP, LLM06:2025 Excessive AgencyAuthorisation should be enforced in downstream systems "rather than relying on an LLM to decide if an action is allowed or not"Each contract line is marked as a platform rule, an instruction, or a capability the agent does not have
Turpin, Michael, Perez and Bowman, 2023; Anthropic, 2025A model's stated reasoning can misrepresent why it actedRun records keep what the platform observed apart from the agent's own account, which stays labelled until checked
Bainbridge, 1983; Parasuraman and Manzey, 2010Automation leaves people the hardest monitoring work, and complacency appears when other work competes for attention, in experts tooEach approval covers one action, and the fleet view leads with what needs a person
Lee and See, 2004; Shneiderman, 2020People rely on automation badly unless trust is calibrated; high automation and high human control can coexistEach control states its effect before it runs; controls a role cannot use stay visible, disabled, with the reason
Amershi et al., 2019Eighteen guidelines for human-AI interaction, among them making clear what the system can do and supporting efficient correctionAn agent's page opens on what it may decide and on whatever needs a person
EU AI Act, Article 14High-risk systems must be designed so people can oversee them; those assigned can, as appropriate and proportionate, decide not to use the system, disregard, override or reverse its output, and interrupt itStop, pause and freeze controls, each stating what it does to running work and pending approvals
NIST AI RMF 1.0Roles and responsibilities for managing AI risk should be documentedA role matrix per agent, and an audit log of who changed what
Chan et al., 2024; OpenTelemetry GenAI agent spansChan et al. assess three kinds of measure for visibility into agents: identifiers, real-time monitoring and activity logs; a telemetry standard for agent and tool calls is emergingThe simulated engineering review checked each screen against what a platform would have to record for it to be true

The brief we set ourselves

We designed for Harbourline, a fictional UK freight company with 32 agents across revenue, finance, support, people and operations. Its customers, such as Calderwick Freight, and every figure on screen are invented. Three roles use the console: an agent owner who answers for their own agents, a platform admin who runs the fleet, and a security reviewer who audits access and change.

Three constraints governed every screen:

  • Claim nothing the platform cannot know. Where no quality measure exists, the console says "Not measured" rather than inventing a score.
  • Label facts and guesses. Observed results are marked as confirmed; likely causes stay "to verify" until a check confirms them.
  • Controls explain themselves. Each states its effect before it runs, from pausing one agent to freezing it.

Each agent also has an operating contract: its goal, what it decides alone, what it must ask about, what it must never do, what it checks first and when it stops, with every line saying how it is held.

Information architecture of a concept AI agent console with fictional data: agents list, agent dashboard and run detail, seven agent tabs, and linked areas for Chat, Inbox, Connections, MCP skills and settings.

The console's information architecture: three levels inside Agents (list, agent, run) and seven tabs on each agent. Five other areas are linked rather than copied: Chat, Inbox, Connections, MCP Skills (the tools agents use, served through MCP servers) and Account settings. An approval is one item that appears in Inbox and on its agent.

Figure 2

Role matrix in a concept AI agent console with fictional data: what an agent owner, a platform admin and a security reviewer may each do, with approvals reserved for the owner and widening authority for the admin.

The role matrix for one agent. (1) Only the agent's owner, or their named backup, can approve its actions; a platform admin cannot approve on an owner's behalf. (2) Granting new skills, changing the owner and retiring an agent sit with the admin, not the owner.

Figure 3

We also wrote down four assumptions about the platform underneath, because a console can only show what a platform records: the main user is a platform admin at a customer company; the platform records runs, steps, tool calls, tokens and policy decisions per agent; approval rules are enforced by the platform outside the agent and can be read by name; and agents hand work to other agents as linked runs. The simulated product review challenged all four.


How the console was built

The console was built in code, not drawn as static screens, so that states like "waiting for approval", "interrupted" and "restored, now being watched" could be clicked through with realistic data. It was rebuilt over several rounds. These are the changes we can still trace in it.

StageWhat we triedWhat we learned
First versionsA fleet dashboard of runs, errors, usage and cost, with agents treated like workflows with better chartsA broken agent and a worsening one looked the same; health had to split into running, completing and quality
Live workWork in progress, hand-offs between agents as linked runs, what each skill can affect (read, modify, communicate, move money), bulk actionsEffects read more clearly than risk scores, and an orchestrator (an agent that hands parts of a task to others) only makes sense when its hand-offs are linked runs
Contracts and evidenceOperating contracts, decision records with the options an agent weighed, and versions compared on the same saved cases: past examples whose reply a person approvedA contract's value is in marking how each line is held, not in listing rules
Releases and restoresReleasing a new version in stages (draft, tested on saved cases, a second approval when it tests worse or widens the agent's authority, then live), and restoring an old version as a new one that is watched, not declared fixedA release needs a gate a person can see, and a restore is a hypothesis with an owner, a pass mark and a date
Concept AI agent console with fictional data, showing an agent's overview with numbered pins and a design-notes panel explaining decisions such as one-action approvals and a three-way health split.

The console's own design notes, opened on Reply Drafter's page. Each numbered note explains a decision behind the screen, such as why an approval covers one action and why health is split three ways.

Figure 4


Stress test: role-briefed AI reviewers

We stress-tested the console with AI reviewers briefed as three people who would find fault: an architect of agent platforms, a product leader, and an engineer who builds evaluation tools. Each was given only the console's screens or the draft of our essay. They found two places where a screen claimed more than the platform could show, and one place where widening an agent's authority was too easy. Each was fixed.

What the reviewers foundWhy it matteredWhat we changed
A refund still waiting for approval appeared in its run record as paidThe record contradicted the moneyRun records show only what happened, with pending actions marked as pending
Some limits carried a "platform rule" label that no policy backedAn auditor would rely on a control that did not existRule labels are read from the platform's configuration, not typed
Giving an agent a new tool took one clickWidening an agent's authority went unreviewedWidening becomes a draft that a security reviewer approves, and the requester cannot approve their own

AI reviewers are not people who run agents in production, so this round tests internal consistency, not usefulness.

Failed AI agent run in a concept console with fictional data: what was observed is marked Confirmed, the likely cause (an expired stored credential) is marked To verify, and a timeline shows the failed step and steps not run.

A failed run after the stress test. (1) What the platform observed is marked Confirmed. (2) The likely cause stays To verify until a check confirms it. (3) The step that failed, with the raw error one click away and later steps shown as not run.

Figure 5


Reviews with client stakeholders

At least six people from our clients' teams reviewed the console: product leaders, engineering and platform leads, operations and support managers who would own agents, and IT and security administrators. Some sessions were live walkthroughs on a call; others used the link on their own and sent feedback. The people and their organisations stay anonymous, and what they told us stays private, so none of the results on this page come from these sessions.


Simulated role reviews

To publish results, we ran four simulated role reviews: an AI reviewer briefed as each role, working from captures of the console and its data, reporting what it could do, what confused it and what it would change. These are not people, and using AI in place of research participants has known limits, so we treat the results as a list of things to test with people, not as evidence of how people behave.

Each review followed the same protocol: one realistic scenario per role, run through the console's guided tours, with a success criterion. We keep this protocol for the next round, with people.

RoleScenarioTask in the consoleSuccess criterion
Operations or support ownerDrafts "feel worse" but the dashboard is greenCatch a behaviour changeNames the cause and the version, and can say what restoring would change
IT or security administratorAn auditor asks who can approve, what is enforced, what changed and who changed itRead contracts, roles and the audit logAnswers every audit question from the screens
Engineering or platform leadA shared credential fails at 05:42 across three agentsTrace a failureFinds which agents and work were affected, and retries safely with results checked
Product leaderDeciding what agent screens to ship for enterprise customersResolve an approval; watch agents workingSeparates launch needs from later ones and judges the risk of rubber-stamping
Animated tour of a concept AI agent console with fictional data: opening Reply Drafter's falling results, comparing version 3.0 with 2.4 on saved cases, and restoring 2.4 as version 3.1 with a typed reason.

The tour behind the operations scenario: from the fleet to Reply Drafter's falling results, comparing version 3.0 with 2.4 on saved cases, and restoring 2.4 as version 3.1 with its reason on the record.

Figure 6

Animated tour of a concept AI agent console with fictional data: filtering to agents on a failing connection, opening a failed run's diagnosis, reconnecting, retrying with checked results, and ending in the audit log.

The tour behind the engineering scenario: three agents share a failing connection; the run record separates the observation from the likely cause; the console tests a sign-in before confirming it, then retries the failed work and checks the results.

Figure 7

Each simulated reviewer rated four things on a five-point scale (5 is best):

Simulated reviewerFind what needs meUnderstand whyTrust the numbersKnow what to do next
Operations or support owner3433
IT or security administrator4433
Engineering or platform lead4424
Product leader4433

What the simulated reviews agreed on:

  • Trust in the numbers scored lowest, or joint lowest, in every review. Three of the four found the audit log out of order (in a newest-first list, an entry at 08:07 sits below one at 05:42), and two found that the "Needs attention: 9" tab does not obviously reconcile with the five tiles above it.
  • Every figure needs its source and time window. The engineering review asked for the query, window and trace behind "38 runs failed"; the product review asked how cost estimates reconcile to a bill.
  • The cause of Reply Drafter's drop is visible but buried. Two of the simulated reviews traced it to version 3.0 being "marked as a content update, so saved cases were not run", and asked that a change which weakens controls be flagged or blocked rather than logged in small grey text.
  • Nothing shows rubber-stamping. The product review found no measure of how owners decide, such as approval rates per person or time to decide, and no cap on how many approvals one owner faces. Our essay argues for an approval budget per agent; the console does not have one yet.

Named as working by at least two of the four: the split between platform rules and instructions, the three-way health split, and grouping failures by the shared connection behind them. The engineering review singled out the separation of observed facts from likely causes, and the product review the approvals scoped to one action.


What we found

Eight design rules came out of the experiment. Each is on screen in the console, and each traces back to a claim, a source or a review.

RuleWhere it came fromWhere it is on screen
Split health into running, completing and quality, against each agent's own baselineH1; Chen et al.; two of the simulated reviewsAgent overview, fleet tiles
Give every agent an operating contract that marks how each line is heldH2; OWASP; the simulated security reviewContract panel
Keep what was observed apart from what the agent or console infersH3; Turpin et al.; the simulated engineering reviewRun records, diagnosis
Scope each approval to one action, with a named backup and a due timeH4; Parasuraman and ManzeyApproval card, Inbox
Store each version's full setup and compare versions on saved casesH5; Chen et al.Versions tab
Pin a run to its version, but check every tool call against current accessStress testLive view, run records
Restrict immediately; review every widening of authorityStress test (widening); EU AI Act Article 14 (interrupting)Tools and skills, security review
Show "Not measured" rather than an invented scoreBrief constraint; Lee and SeeAgent overview

What we take into client work: start agent features from what the platform records, settle each agent's contract before its screens, and treat approvals as something to measure, not just to place.

Two AI agent operating contracts in a concept console with fictional data: one warns four lines are missing from its instructions; the other marks what it must ask about as platform rules and a stop condition as an instruction.

Two operating contracts. Left: (1) four lines of Reply Drafter's contract are missing from the instructions its model receives. Right, Renewal Desk: (2) what it must ask about is enforced by platform rules; (3) its stop for stale usage data is only an instruction.

Figure 8

AI agent approval card in a concept console with fictional data: the requested change, the platform rule that stopped the agent, linked evidence, exactly what the decision covers, and a separate customer email.

An approval as its owner sees it. (1) Why the agent stopped: a discount limit enforced by the platform's policy check, not by the agent. (2) What the decision covers: this one update, and nothing later. (3) The customer email is a separate action with its own check.

Figure 9


What we will test next

Round seven is queued from the reviews above. The changes, most important first:

  1. Make every number accountable. Each count and percentage carries its time window, its definition and a link to the records behind it, and the audit log is ordered strictly by date and time.
  2. Make counts reconcile. The attention tab and the tiles above it count the same way, or say how they differ.
  3. Close the content-update path. Reply Drafter's version 3.0 went live as a "content update", which skipped the staged release and its saved cases. Every change to instructions or knowledge goes through the same release steps, and raising a limit needs a recorded second approver.
  4. Show how owners decide. Approval rates and time to decide per owner, a visible approval budget per agent, and spot checks of approved items.
  5. Show the consequences of a restore. What goes live, what stays affected, and who may press the button.
  6. Add an evidence view. A dated, filterable export of role holders, enforced and instruction-only limits, changes with their approvers, and owner status, as an auditor would ask for it.

After round seven we want people who run agents in production to work through the same four scenarios, timed, against the success criteria above.


Limits of this experiment

  • The data is fictional. No real platform sits underneath, so the console shows what a well-instrumented platform could support, not what any particular one does.
  • Simulated reviews are not people. They worked from captures and the console's data, not from live use.
  • The stakeholder reviews were not field use. Nobody operated these screens on their own agents over weeks, and we do not publish what they said.
  • Scale. We designed for a fleet of dozens of agents. Thousands would need more grouping and summarising than this concept shows.

Try it yourself

The console is live, with sample data throughout. Each tour takes about two minutes. Two issues found above are still in it: the audit log's order and the attention count.

Animated tour of a concept AI agent console with fictional data: an owner changes a requested 12% discount to 10%, the new proposal passes the rule check, the customer email is sent separately, and the audit record opens.

The approval tour: the owner offers 10 per cent instead of the 12 per cent the agent asked for, which creates a new proposal checked against the rule again; the customer email is decided separately, and the audit record keeps both figures.

Figure 10


Questions about this experiment

What is Answerable?

A Future Interfaces Lab experiment: a working console for overseeing AI agents, built on fictional data to test what the person accountable for an agent needs to see. Its name comes from that person: the one who answers for what the agent does.

Was it tested with real people?

Yes. At least six people from our clients' product, engineering, operations and security teams reviewed it, in live walkthroughs or on their own; what they said stays private. The results published here come from something else: four simulated role reviews and an earlier stress test, both run by AI reviewers and labelled as such.

What should an AI agent console show?

For each agent: what it may decide and how each limit is enforced, what it actually did with the evidence kept apart from its own account, and whether the results were acceptable, measured against its own baseline. Runs, errors and cost belong underneath, not on top.

How is this different from agent observability tools?

Observability tools serve the engineers who build agents, with traces, evaluations and versioning. This experiment designs for the people who answer for agents in the business: owners, admins and auditors. It depends on the same records and puts them in their words.

Can we use the console?

You can try every screen at lab.tcules.com. It is a concept on sample data, not a product. If agents act inside your product, the useful starting point is to list what your platform records today and set it against what your owners, admins and auditors need to see. That is where we would start with you.

About the author

  • Jatin Leuva

    Founder and CEO

    Jatin Leuva is the founder and CEO of Tcules, the digital product design studio in Ahmedabad. At Tcules, Jatin works on product strategy and AI product UX, and writes about questions such as what a SaaS MVP must prove, what a designer should understand about a business, and why AI features that pass a demo fail in production. Jatin also hosts the studio's conversations with product builders. The articles under this byline speak for the studio.

    LinkedInMore from Jatin

Designing oversight for your agents

If agents act inside your product, we would be glad to compare notes.