Securing the Agentic Enterprise

What happens when software starts acting on its own

Recently I was experimenting with Claude in our company’s general Slack channel.

I asked what Across does. Good answer. It had enough of my history to draw on and its answer sounded like it really understood our company, including the latest changes.

Then I made a mistake… asking what was top of my mind. With confidence it started posting my personal to-do list in front of the whole company. Now of course I deleted that about as fast as I can click.

Here’s the thing that stuck with me afterwards… on paper it did nothing wrong. It answered my question, factually, and leveraging the permissions and information access I had provided. The problem was context. Every human in that channel understood (without needing an explicit policy or permission) that you don’t read someone’s private to-do list out loud to the entire company…

That context gap was a running theme through our last dinner, co-hosted with BCG, on securing the agentic enterprise. We had Swamy Kocherlakota anchoring the conversation. He’s spent years as a CIO at S&P Global before moving to run agentic security at Zscaler.

Access is no longer the whole question

The old model was built for people. You have an identity, that identity carries permissions, and if something goes wrong there is usually a way to trace the action back to a person who can explain what they thought they were doing.

That model starts to bend with agents, because the “user” is no longer the only actor that matters. An agent can act on your behalf, call another agent, read from one system, write to another, and make a few judgment calls before anyone has stopped to inspect the chain. So asking “who is the user?” is still necessary, but it is not enough. You also have to ask what the agent is trying to do, why it is doing it, and whether that action makes sense in this particular context.

That was what bothered me about the Slack moment. The permissions were right, in the narrow access-control sense, but the social and organisational context was missing.

Swamy framed the emerging agentic security model around three questions: who are you, what are you trying to do, and why. The “why” is the part we did not have to formalise as much when the actor was human, because people carry a lot of implicit judgment around with them, and if they make the wrong call they can at least be asked to explain it afterwards. An agent has none of that instinct unless the system around it gives it one. It will follow the instruction it was given and use the access it has, even when every person in the room would have known to stop.

Agents need policy beyond permissions

A lot of teams start by asking whether an agent should just inherit the permissions of whoever invoked it. That is where we started too at Across AI.

It helps to do this in a multi-agent system because you can use the person’s permissions as the baseline and further limit it based on that agent’s role. You might have access to both Workday, Oracle Financials and Payroll systems, but you probably do not want one agent moving across both in a single autonomous step.

We can think of it as segregation of duties for agents. Permissions are not just about who invoked the agent, or what the agent can technically access. They depend on the task, the data involved, where the output is going, and whether a human is watching. This shift from role-based access to intent- and task-based access is much harder to build, but without it, enterprises will struggle to let agents do consequential work safely.

Visibility comes before control

As we got deeper into the governance discussion the most practical discussion that came up was also potentially the most concerning: do most companies really know where AI is already running inside their organizations?

AI and Agentic traffic is everywhere. It is embedded in various existing protocols. There is embedded AI, end-point AI, Co-pilots, AI Studios. It’s already in the SaaS tools people use, the browser extensions, the coding assistants, the meeting summaries, the latest agent harness trending on X. Before you can govern any of it, you have to be able to see it.

Only then can you decide what to allow, what to block, which models and vendors are approved, and which data flows need stricter rules. Visibility remains the part many companies still have to do.

Who is actually responsible and accountable when an agent gets it wrong?

This was the part of the night with the least agreement, and the part I am still thinking about.

My own view has moved. If you had asked me a few months ago whether agents should have their own identities, I would probably have said no. My answer would have been: just use the identity of the person running the process, log everything, and stay inside a system we already have deployed at most companies and understand. But that answer gets less comfortable as the agents take over more and more work.

If a process runs for months and outlives the person who set it up, as the agents we build at Across do, then maybe it does need an identity of its own, scoped to that process and only the permissions it needs. That being said, identity does not equal accountability, and that is where the conversation got harder.

Swamy’s view was blunt: an agent is not a legal entity, so it cannot be responsible. If it causes harm, responsibility lands on the organization, and in his view it ultimately rolls up to the person in charge, the CIO, Chief Risk Officer and others involved in the decision to roll it out. If an employee uploads a document they were explicitly told not to upload, that is a policy violation. But if the organization deployed the system, gave it access, and did not build the circuit breaker, the organization owns the outcome.

The circuit breaker analogy stayed with me. In financial markets, specifically in electronic trading systems with an order book, you do not just build the automated system, you also build the thing that stops it, because by the time everyone does the root cause and is assigning blame the damage has already happened. That feels like the right way to think about agentic systems too: not just who approved the agent, but who designed the stop conditions.

One BCG experiment made the risk more concrete. When the same task was framed as work done by an “AI employee” rather than an “AI tool,” people caught fewer errors and pushed more responsibility onto the agent, which has none to give. That is the part that worries me. The identity question is genuinely open, and reasonable people in the room disagreed. But we should be careful not to let agent identity become a way to smuggle in agent responsibility. Enterprises do not have an operating model for that yet.

The hard problems are still ahead

The honest note the conversation ended on was that most of the hard problems here are not solved. Agents do not sit still in the way traditional software does, because the model can change, the context can change, and the same instruction can produce a different answer tomorrow. So review cannot be something the system passes once before launch. It has to stay live.

Inline monitoring is the clearest example. If you want to check an agent’s behavior in real time, you have to do it at full speed and full scale, and LLM-based evaluation is still slow and expensive. Every guardrail and judge adds latency and compute to the work itself.

That runs into the deeper architecture question: when should a task be deterministic software, and when does it actually need a probabilistic model? Not everything needs an agent. Not every decision should touch an LLM.

What I came away with

I came away from the dinner less convinced that agent security is a separate category from agent orchestration. The same questions kept showing up in different forms: what can the agent see, what is it allowed to do, who can stop it, and who understands what happened afterwards?

That is very close to how we think about the work at Across. If agents are going to execute real business processes, they need more than access to data and tools. They need context, reasoning, learning, policy, auditability, and clear boundaries around when to act and when to stop.

The Slack moment was small, but it has stayed with me because it made the abstract problem very concrete. The system had the permission. It did not have the judgment.