Agentic AI risk becomes much less abstract when the model can spend money. Cambridge researchers reported that agentic AI was already in active adoption among 52% of surveyed financial industry respondents, while model hallucinations and unreliable outputs were rated as a top-two risk by surveyed AI vendors, industry firms, and regulators.[s] That is the new problem: the old chatbot failure mode is meeting payment rails, bank operations, and financial markets.
The one who signs the checks put this one on the desk, a neat joke because this is exactly the moment when software stops chatting and starts spending.
A hallucination in a search answer can mislead a person. A hallucination inside an agent that has permission to place orders, approve invoices, move liquidity, or trigger trades can create an action that other systems must process. Visa has announced a collaboration with OpenAI to enable Visa payments within agentic commerce, and said those transactions would operate under user permissions and controls such as spending limits, merchant categories, or required approvals.[s] Mastercard describes agents as chatbots enabled to take actions such as sending emails or ordering food, and warns that legitimate agent transactions may look suspicious when they happen at odd hours, across geographies, or in rapid repeated patterns that resemble fraud bots.[s]
Agentic AI risk starts with permission
An agent is not simply a model with a friendlier interface. Joint cyber agencies describe agentic AI systems as one or more agents that rely on an AI model to interpret the world, make decisions, and take actions.[s] In normal language, the important verb is “take.” Once software can take action, the safety question changes from “Is the answer true?” to “What can this system do when it is wrong?”
That is why the AI denial hallucination is not only a user annoyance in finance. If an assistant invents a refund policy, misreads a merchant category, or mistakes a company instruction for a payment authorization, the loss is no longer contained in the chat window. The compliance gap moves from wording to workflow: who approved the action, what limit applied, which system logged it, and whether a human could stop it in time.
Payment systems make this sharper. BIS researchers call payment systems the lifeblood of modern economies because they transfer value among individuals, businesses, and governments.[s] In a BIS experiment, a generative AI agent replicated key cash-management tasks without specialized training, including choices around liquidity buffers, urgent payments, and settlement delays.[s] That is promising. It also shows why agentic AI risk belongs in the same conversation as financial infrastructure, not only consumer chatbot safety.
Why a single bad answer can spread
Systemic risk does not require every agent to fail. It can emerge when many systems behave similarly, respond too quickly, or depend on the same few providers. BIS warned that AI can speed up trading and portfolio adjustments, which may intensify short-term price movements when conditions change.[s] It also warned that the widespread use of similar AI models, data, or decision rules can make institutions respond to shocks in similar ways, increasing behavioral correlation.[s]
That is the bridge from hallucination to systemic risk. The bad event is not only “the model made something up.” The bad event is “many systems trusted a similar process, acted faster than review could keep up, and fed each other confirming signals.” Federal Reserve researchers found that AI agents in experiments could be induced to herd when explicitly guided toward profit-maximizing decisions.[s] In markets, herding is not science fiction. It is one route by which automation can make a crowded trade more crowded.
Security adds another route. Joint cyber agencies warn that agentic AI complexity can introduce cascading failures and multi-step attacks, where compromised or unexpected behavior in one component propagates through later steps and affects the whole system.[s] In a bank, that can mean the issue is not the model alone. It is the model plus memory, tools, permissions, logging, approvals, vendor systems, and the SaaS reliability failures that sit underneath the workflow.
The bank account changes the error budget
Existing controls for bad inputs, fraud attempts, and operational incidents are only part of the answer. Agentic AI risk is different because the timing changes. BIS warned that, under stress, AI and digital finance may compress the time available for institutions and authorities to respond as liquidity pressures, operational disruptions, or market reactions unfold more rapidly.[s] Manual oversight can become a ritual after the action instead of a control before it.
The ECB makes the same point from the operational side. It warned that a bank can have ample capital and liquidity yet still face severe operational issues, or even fail, if it lacks preparedness and contingency planning for operational shocks.[s] It also warned that the speed, scale, and accessibility of advanced cyber capabilities are increasing while the time available to defenders is shrinking.[s]
The practical answer is not to ban agents from finance. It is to treat them like high-risk operators with narrow authority. Spending limits, merchant restrictions, separate approvals for new counterparties, immutable logs, fast revocation, and testing against prompt injection are not polish. They are the boundary between a useful assistant and a payment-capable system that can turn a wrong inference into a real obligation.
The useful mental model is simple: hallucination is a content error until the system has authority. After that, agentic AI risk is an operational risk, a cyber risk, and sometimes a financial stability risk. Handled this way, agentic AI risk is a design problem before it is a headline problem. The dangerous part is not that the software sounds confident. It is that confidence can now be connected to a button marked “send.”
Agentic AI risk is a privilege and propagation problem. Cambridge researchers reported that agentic AI was already in active adoption among 52% of surveyed financial industry respondents, and that hallucinations and unreliable outputs were rated as a top-two AI risk by surveyed vendors, financial firms, and regulators.[s] When those systems are connected to payment credentials, market workflows, or bank operations, hallucination becomes only one failure input in a much larger control system.
This topic came from the one who signs the checks, which is fitting because the engineering question is what happens when a probabilistic system gets spending authority.
Visa has announced a collaboration with OpenAI to enable secure Visa payments within agentic commerce, and said transactions will use defined permissions, policies, and controls such as spending limits, merchant categories, or required approvals.[s] Mastercard defines agents as chatbots enabled to take actions such as sending emails or ordering food, and says agentic commerce can confuse fraud systems because automated purchases may occur at odd hours, across geographies, or in rapid repeated patterns.[s]
Agentic AI risk as a control problem
Joint cyber agencies define agentic AI systems as one or more agents that rely on an AI model, such as an LLM, to interpret the state of the world, make decisions, and take actions.[s] The architecture matters because the failure surface includes the model, tool calls, memory, data sources, triggers, identities, approval paths, and downstream systems.[s]
Traditional hallucination control asks whether an output is true. Agentic control asks whether the system should be allowed to execute the next step. The AI denial hallucination is therefore a symptom, not the full risk. A model that rationalizes its own mistake can be annoying in a document editor. In a payment workflow, that same behavior can become an audit problem if the system also has authority to initiate or approve a transaction.
The core engineering rule is least privilege.[s] Agents should receive only the tool access, transaction limits, and data scope needed for the task. The compliance gap appears when organizations treat policy prompts as if they were access controls. A prompt can express intent, but payment networks, bank ledgers, procurement systems, and identity providers need enforceable constraints that do not depend on the model interpreting the instruction correctly.
From hallucination to correlated behavior
The financial stability issue is not only individual model error. BIS warned that AI can accelerate trading and portfolio adjustments, which may intensify short-term price movements when market conditions change.[s] BIS also warned that widespread use of similar AI models, data, or decision rules can cause institutions to respond to shocks in similar ways, increasing correlation in behavior.[s]
That is the path from a local inference problem to a market problem. A single hallucinated data point may be caught. A shared reasoning pattern embedded across advisory agents, cash-management agents, and trading support tools can create synchronized behavior before supervisors see the aggregate shape. Federal Reserve researchers found that AI agents in experimental settings could be induced to herd when explicitly guided to make profit-maximizing decisions.[s]
Payment infrastructure shows why this matters outside trading desks. BIS researchers call payment systems the lifeblood of modern economies because they transfer value among individuals, businesses, and governments.[s] In prompt-based experiments, BIS found that a generative AI agent could replicate key cash-management tasks even without specialized training.[s] A system that can prioritize urgent payments and manage liquidity can be useful; it also needs controls for false premises, manipulated inputs, and stress conditions.
The systemic part is speed plus coupling
Agentic AI risk becomes systemic when several properties combine: autonomy, speed, shared infrastructure, similar objectives, and weak interruption points. BIS warned that under stress, AI and digital finance may compress the time available for institutions and authorities to respond as liquidity pressures, operational disruptions, or market reactions unfold faster.[s] Cambridge similarly warned that rapid deployment of agentic AI compounds cyber vulnerabilities and renders manual oversight increasingly ineffective.[s]
Cyber agencies describe the structural version of the same problem: agentic AI complexity can introduce cascading failures and multi-step attacks, where compromised or unexpected behavior in one component propagates across later steps and affects the entire system.[s] That is why SaaS reliability failures matter here. A payment-capable agent can inherit the reliability, identity, and logging weaknesses of every service it depends on.
The ECB has been blunt about the operating environment. It warned that a bank can have ample capital and liquidity but still face severe operational issues, or even fail, without preparedness and robust contingency planning for operational shocks.[s] It also warned that advanced cyber capabilities are increasing in speed, scale, and accessibility while defenders have less time to respond.[s]
What good design has to assume
For banks, agentic AI risk should be tested at the workflow boundary: the point where a recommendation becomes an instruction to another system.
A serious deployment has to assume the agent will sometimes misread intent, overfit to an objective, trust hostile context, or take a locally rational action that is globally unsafe. The control stack should therefore sit outside the model: hard transaction ceilings, merchant and counterparty allowlists, anomaly detection tuned for authorized agents, human approval for irreversible actions, durable logs, scoped credentials, and immediate kill switches.
The most important design move is to separate recommendation from execution. Let the model propose, compare, and explain. Let deterministic controls decide whether the action is allowed. Let humans approve actions that create legal, financial, or operational commitments beyond a narrow preauthorized scope. In that architecture, agentic AI risk is managed as a systems problem instead of wished away as a prompt-quality problem.
The bank account is the threshold. Before it, a hallucination is a claim. After it, a hallucination can become a payment, a trade, a declined legitimate purchase, a liquidity decision, or a cyber incident. Finance can use agents, but only if it designs for the moment when the answer is wrong and the system is still fast enough to act.
