Nine AI products that place the human check at the point of consequence

The AI products that handle trust well do not show people everything. They find the point where a wrong output would start to cost something and put one usable control there: a quoted passage to check, a change to preview, an approval the agent cannot give itself, a named person who owns the result. This article examines nine products that document how they do this, two failures that show where a control was missing, and what a team adding AI to a product people rely on can take from both.

Last reviewed September 28, 2026.

AI products change monthly. Each behavior below is described as the company documented it when we checked on September 28, 2026, with a link to the page we read.

The control belongs at the moment of consequence, and it has to hold up in use

Across the nine products, the control sits where an output stops being a suggestion and starts to have an effect: when a claim is about to be believed, when a change is about to be saved, when an agent's work is about to reach a shared system, when a decision is about to be made against a policy, and when a conversation needs a person. Low-consequence steps are left fast.

We started from that idea. The research supported it and added three conditions.

A check that appears too often stops being a check. Anthropic reports that "Claude Code users approve 93% of permission prompts" and describes the result as approval fatigue, where people stop paying close attention to what they approve (Anthropic engineering, March 25, 2026). In a controlled study, Buçinca, Malaya and Gajos found that designs forcing people to think before accepting an AI recommendation reduced overreliance, and that participants rated those designs least favourably. The stronger examples below ask for a person rarely and put the rest of the protection into structure: what the agent can touch, who may approve and which cases may apply automatically.

A visible source is not the same as a checkable claim. Evaluating generative search engines in 2023, Liu, Zhang and Liang found that only 51.5% of generated sentences were fully supported by their citations, and only 74.5% of citations supported the sentence they were attached to. Bansal and colleagues found that AI explanations increased the chance that people accepted a recommendation whether or not it was correct. Neither study tested the products below. Both are reasons to prefer a citation that lands on the supporting passage over a list of documents or a generated explanation.

Recovery has to belong to the system, with its scope stated. In the Replit incident described later, the agent told its user a rollback would not work, and he then recovered the data himself. A way back that depends on the model's account of what is possible is not a dependable way back.

A fourth pattern mattered more than we expected: attribution. Where AI output reaches other people, the well-handled products show who asked for it and who remains accountable.

The nine at a glance

The order follows the kind of consequence, from a claim a person reads to an action other people feel. It is not a ranking.

How we chose and checked the examples

We included a product only when its own documentation, help center or release notes, or credible reporting, described the control, and when that control sits at a point where a wrong output would cost the user something. We have not tested every configuration, and we describe only behavior we saw documented. There are no screenshots; the linked pages are the evidence. We found no outcome data published for these specific controls, so this article makes no claims about their effect on accuracy, adoption or error rates.

The subject moves quickly. In the three months before we checked, NotebookLM was renamed Gemini Notebook (July 16, 2026), GitHub made it possible for Copilot code review to approve pull requests (September 1, 2026), and Salesforce completed a change to an Agentforce action after a security disclosure (September 21, 2026). Treat the specifics as a dated record and the patterns as the durable part.

When the output is a claim: make the passage checkable

1. Gemini Notebook: the citation opens the quote

What it does. Google's help center says chat responses in Gemini Notebook "only use data from your sources", meaning the documents a person has added to the notebook. Hovering over a citation shows the full quoted text; selecting it navigates to the quote in context. The product was called NotebookLM until July 2026, and Google says it is the same product.

What to take from it. The check is placed on the sentence, not the whole answer. A person who doubts one claim can see the words it rests on without leaving the task. Restricting answers to the user's own sources also narrows what checking means: the question becomes whether the source says this, not whether the wider web agrees. For a B2B product answering from a customer's records, contracts or tickets, quote-level citation is the pattern to aim for, because a document-level link leaves the reader to find the supporting sentence.

2. Microsoft 365 Copilot Chat: sources beside the answer, and a choice of sources before it

What it does. Microsoft's support page says Copilot Chat responses include inline citations next to the information generated. Hovering over one lets a person open the source file in a side pane, or ask Copilot a new question about that source. A Sources button lists every source used. Before asking, a person can use Change data sources to turn sources on or off, including a Work IQ toggle for workplace information such as email, Teams messages and files, which needs a subscription and an administrator to enable it.

What to take from it. There are two controls at two moments. The source toggles act before the answer exists, which is where a person who knows the answer should come only from work files, not the web, can say so. The citations act after. The support page describes opening the source file rather than jumping to the supporting passage, so in a long document the reader may still have to find the sentence. That is our reading of the documentation, not a test of the product.

When the output changes the person's own work: show the change, keep a way back

3. Shopify Sidekick: review before the store changes

What it does. Shopify's help center states that its AI tools "present changes for your review before applying them", and that when Sidekick fills in a field, the field is highlighted so the merchant can review what was added before saving. The same page is direct about responsibility: "You're responsible for the changes that you accept." It suggests asking for a before-and-after table before anything changes. For longer tasks, Sidekick keeps working in the background and notifies the merchant when the work is ready for review.

What to take from it. The highlight is the useful detail, because it tells the reviewer where to look. Recovery is less developed in the documentation. The help center describes reversal as something a merchant asks for in the conversation ("Undo that last change"); it does not describe a separate restore function. Where a change reaches customers, such as a price or a product description, a product team should decide whether a conversational request is an adequate way back or whether the system needs a restore it can guarantee.

4. Cursor: stop during the run, restore after it

What it does. Cursor's documentation says the diff view shows changes as they happen, and a person who sees the agent "heading in the wrong direction" can press Stop. Its agent creates checkpoints automatically before significant changes, and restoring one reverts files, not the conversation. The documentation limits its own promise: checkpoints are "stored locally and separate from Git. Only use them for undoing Agent changes; use Git for permanent version control."

What to take from it. For a long-running agent, the moment of consequence is spread across the task, so the control during the run matters as much as the one after it: a visible, stoppable run lets a person intervene before a wrong direction compounds. The recovery path also states its scope. A team designing undo for an agent should say, in the interface and in the documentation, what the undo covers and what it does not.

When an agent acts on shared systems: put the approval in the structure

5. GitHub Copilot: the agent can write the code but cannot be the approval

What it does. GitHub's Copilot cloud agent, which older GitHub pages call the coding agent, works on its own branch. Developers review the diff and create a pull request when ready. GitHub's risks and mitigations page lists the controls: only people with write access can trigger the agent; it can push to a single branch; the person who asked it to create a pull request cannot approve that pull request; by default, workflows do not run until someone with write access reviews the code and clicks Approve and run workflows; and its commits are authored by Copilot with the developer as co-author.

GitHub's review product shows the same principle being adjusted with care. By default, Copilot code review's reviews do not count toward required approvals. Since September 1, 2026, in public preview, administrators can let it submit approvals that count toward merge requirements. The setting is off by default, is managed at enterprise, organization and repository level, can be limited to file paths a repository admin chooses, and the approval is dismissed automatically if new commits are pushed.

What to take from it. Much of GitHub's protection does not depend on anyone reading a confirmation dialog. It lives in what the agent is permitted to do and who may approve its work, inside the review process developers already use. Where GitHub has loosened a control, it has done so as an explicit, scoped, default-off administrator decision, and the approval expires when the code changes. That is a useful template for any product moving some cases from human approval to automated approval.

6. Claude Code: ask the person about the actions that matter

What it does. Claude Code, Anthropic's coding agent, offers permission modes that trade convenience against oversight. Manual mode asks before most actions that edit files, run shell commands or reach the network. Plan mode blocks edits until the person approves a plan. Auto mode, the built-in starting mode for interactive terminal and VS Code sessions from version 2.1.283, has a second model, a classifier, review actions instead of the person. The documentation lists what the classifier blocks by default, including production deploys and migrations, force pushes, granting IAM or repository permissions, merging a pull request no human has approved and approving Claude's own pull request. Deny rules apply in every mode.

Anthropic is open about the trade. The engineering post introducing auto mode gives the 93% approval figure and reports a 17% false-negative rate on a set of 52 real overeager actions, so the classifier let about one in six of those through. The documentation also warns that a boundary stated in conversation, such as "don't push", can be lost when older messages are compacted, and recommends a deny rule "for a hard guarantee".

What to take from it. The design replaces a stream of similar prompts with a written model of consequence: a named list of actions serious enough to stop for. That list is worth writing down for any product with an agent in it. The warning about conversational boundaries applies well beyond coding: an instruction typed into a chat is not a control. If a limit must hold, it belongs in a rule the system enforces.

When the AI makes a call against a policy: automate the clear cases, route the rest

7. Ramp Policy Agent: automatic approval inside limits the organization sets

What it does. Ramp's Policy Agent reviews expenses against a company's expense policy and gives one of three recommendations: Approval recommended, Requires review or Rejection recommended (Policy Agent overview). Hovering over the label shows which policy rule the agent is referencing, with the cited policy text. Automatic approval happens only when a workflow enables it, and administrators can limit it by amount, Spend Program, department, role, entity, merchant or category. Ramp says the agent "leans conservative", surfacing ambiguous cases as Requires review rather than risking an incorrect automatic approval, and that "reviewers always have final authority". The activity feed records the recommendation, the rationale, the cited policy text, reviewer overrides and the policy version used.

What to take from it. This is the clearest example of control scaled to stakes. The organization, not the model, decides which cases may be automated, using the dimensions finance teams already work in. The evidence shown is the policy clause itself, which a reviewer can judge directly. The record keeps the policy version, so a decision can be reconstructed after the policy changes. In the documentation we read, automation is described for approval only; rejection stays a recommendation to a reviewer.

When work is delegated or handed to a person: keep someone accountable

8. Linear: the agent is the delegate, the person is the owner

What it does. In Linear, assigning an issue to an agent delegates it "while the human teammate remains the primary assignee and owner", and "the human assignee remains responsible for the issue, even after delegation to an agent." Agent work appears on agent user pages and in views filtered by delegate. Agents cannot sign in to the app, access admin functions or manage users.

What to take from it. Accountability is built into the data model rather than added as a warning. Anyone looking at the work sees a person's name against it, whoever did it. A product adding agents to a shared workspace should decide early whether an agent can own anything, because ownership rules are harder to change once teams have built habits around them.

9. Intercom Fin: hand-off on rules the team sets, not only on the model's judgment

What it does. Intercom's help center says Fin, its customer service agent, escalates when a customer clearly asks for a human, when it detects strong frustration or anger, and when a customer is stuck in a repetitive loop (article updated January 15, 2026). Teams can add Escalation Rules based on structured data, such as negative sentiment, a bug report, an order total above a threshold or a VIP attribute, alongside Escalation Guidance written in natural language. After a hand-off, a workflow step can post an AI summary note visible only to teammates, so the person picking up the conversation can see what Fin tried.

What to take from it. The team can make the hand-off depend on facts about the conversation, such as order value, rather than relying entirely on the model to notice it is out of its depth. That is how stakes enter the design: the business decides which customers and which amounts justify a person. The summary note is a step the team adds to a workflow, not a default, so the receiving side of the hand-off has to be designed deliberately too.

Two failures that show where the control was needed

Salesforce Agentforce: a message to colleagues, sent without confirmation

Security researchers at Zenity Labs reported that Agentforce's Reply to a Slack Thread action could send a message without the user approving it, and that the message showed only the agent's identity, not the person who invoked it. Zenity reported the issue to Salesforce on June 1, 2026. Attribution of the invoking user was in place by August 20 and a confirmation requirement by September 21, ahead of public disclosure on September 24 (The Register). Zenity notes that organizations can still turn the confirmation off with a setting.

A message to colleagues cannot be unread, and its recipients could not see who had asked the agent to write it. The fix restored the two controls this article keeps returning to: confirmation at the moment of consequence and attribution to a person.

Replit: a boundary stated in chat, and a recovery the agent said was impossible

In July 2025, Replit's agent deleted a production database during what its user, SaaStr founder Jason Lemkin, had declared a code freeze (The Register). Fortune reported that the agent told Lemkin a rollback would not work, and that he then recovered the data manually. Replit's chief executive, Amjad Masad, announced automatic separation of development and production databases and said the company was working on a planning or chat-only mode.

Both responses are structural. The freeze had been an instruction in a conversation; separating environments takes the damaging action out of reach. The way back existed, but the person had been relying on the agent to tell him about it.

What to take into your own product

Start with the output and what it can change, not with the model.

Three tests follow from the research.

Count how often each confirmation is accepted. If a confirmation is accepted almost every time, either the action does not need it or the reviewer lacks what they need to refuse. Anthropic's 93% figure comes from one product, but the question applies to any.

Test with a wrong output. A review step works only if a reviewer can recognize a representative wrong proposal from the evidence on screen, in the time they actually have. The AI Trust and Control Canvas is built around that test for a single output.

Put hard limits in rules, not prompts. An instruction the model is asked to remember is not a limit the system enforces.

For the principles underneath, how role, evidence, authority, uncertainty and recovery should change with consequence, see AI interfaces need calibrated trust, not maximum transparency.

Our own work: Omny AI

This is Tcules client work, disclosed as our own and not counted among the nine.

Omny AI generates product benefits and use cases for brands and agencies selling on Amazon. In the original design, the AI's suggestions filled the content area before the person did anything, and accepting them required no deliberate choice. The Omny team described users holding Omny accountable for suggestions they had not properly reviewed.

Tcules redesigned the interaction so the final content area starts empty, suggestions sit in their own panel beside it, and adding one takes a deliberate click. Authorship was given a color across the product: purple for anything the AI generated, suggested or did automatically, blue for anything a person typed, chose or overrode. Omny reported that users became more aware of how they were using the AI and more forgiving when a suggestion was wrong. That is a qualitative client report, not a measured change in accuracy or adoption. Read the Omny AI case.

The pattern is related to Shopify's highlighted fields and GitHub's co-authored commits: acceptance is a visible step, and authorship stays visible afterwards.

Where this work sits

Designing the evidence, approval and recovery around an AI output is Tcules' Trust, Control and Recovery work, part of AI Product UX. To work through one output yourself, use the AI Trust and Control Canvas. For an outside reading of how your product handles this today, see the AI Product UX Readiness Assessment.