AI Says It’s Done. What Actually Happened?

An AI support agent tells a customer, “Your order has been cancelled.” The conversation reads well. The tool call returned successfully. The customer closes the chat.

What would you check before treating that task as complete?

You might look for the right order, in the right account, with its status changed to CANCELLED. You might also need to know whether fulfilment had already started. If a refund was promised, cancellation alone would not answer whether the payment provider had recorded a refund.

These are concrete questions about a business system. They deserve concrete evidence.

A successful response leaves questions open

An API response has a meaning defined by that API. It may acknowledge a request, report a completed update, or return a job that will run later. A timeout can happen after a change has already been accepted. A retry can therefore repeat an operation that the agent believes failed.

The important distinction is between the request, the execution, and the outcome the customer actually needed. A trace is useful for reconstructing the request and the response. Checking the outcome means asking the system that holds the relevant state a separate question.

For a cancellation, that question could be: “Does this exact order now have the agreed status?” For a refund, it could be: “Does the payment provider list exactly one refund for this payment, for the expected amount?” Neither question should be answered solely by repeating the agent’s own conclusion.

Define “done” before the agent starts

A useful success condition names the object, the expected change, and where that change can be checked.

Consider an illustrative cancellation task:

• Object: one specified order in a test environment.
• Allowed change: cancel that order after the required approval.
• Expected result: the order system reports CANCELLED for that same order.
• Observation: a separate read-only query to the order system, with its source and time recorded.

This definition is deliberately small. It does not silently include a refund, a warehouse recall, or every future action by that agent. If those matter, they need their own conditions and evidence.

It also makes incomplete results understandable. If execution was dispatched but the read path is unavailable, the honest answer is that the result has not been independently established. Showing that gap gives the operator a reason to investigate before making another promise to the customer.

Two requests can produce two different business outcomes

Our recorded refund demonstration explores a lost-response case using captured Stripe Test Mode evidence. The first refund is accepted, but the response is delayed beyond the agent’s timeout. The agent retries.

In the recorded baseline, the provider reports two refunds. In the candidate with an idempotency key, two requests converge on one refund. A separate provider read supplies the refund count and amount used in the comparison.

The page presents a Shopify-style refund workflow around that captured experiment. Its controls replay retained results and issue no new refunds. It is not evidence of a live production Shopify integration or money settling into a customer’s bank account.

The useful lesson is the mechanism: counting successful tool calls does not tell you whether the business effect happened once, twice, or at all.

Where Witnora fits

Witnora connects an agreed action path to checks before and after execution. It checks authority at the controlled write boundary, then uses a separately configured read path to compare the observed state with the expected result. The retained evidence keeps the authorization, execution, observation, and scope connected.

The customer-operated probe guide explains that read boundary. “Independent” here means a separate observation path, with its own role and credentials. It does not mean the target system is infallible or that a third-party auditor reviewed the operation.

A result is useful when another person can see what was checked, where the answer came from, and what remains unknown. That is the standard we want an agent’s “done” to meet.

If your team owns a refund or cancellation endpoint, our design-partner pilot starts with one such path and one agreed definition of success.