Secure Agentic Engineering: Security by Design for AI Agents

Short answer: what secure agentic engineering is

Secure agentic engineering is the practice of making security decisions alongside an AI agent’s architectural decisions — at design time, not after deployment. The question “which tools does this agent get” and the question “what’s the worst it does if it’s fooled” get answered in the same conversation, by the same team, at the same point in the process. This is not a checklist bolted onto a finished product. It is a property of the architecture itself: tool scope, execution environment, trust boundaries around data, and human-approval checkpoints are part of the agent’s design in exactly the same way as its system prompt or its choice of model.

The difference from a “post-hoc audit” is practical, not ideological. An audit inspects a finished system and finds what was already built wrong — and fixing architecture after it’s built is expensive, because it touches foundations that are hard to move without a rebuild. Secure agentic engineering makes the same decisions earlier, when a change costs choosing a different design pattern on a whiteboard instead of refactoring a running production system. It’s the same road the software industry already walked once with “shift left” testing and with privacy by design in data protection — AI agent security is now walking the same road.

This article isn’t about a single defensive technique. It’s a frame that shows where in the process of building an agent each decision that maps to a specific risk in the OWASP Top 10 for Agentic Applications actually gets made — and it links out to the material that breaks each of those techniques down in full. If you’re looking for detail on a specific defensive mechanism, you’ll find a pointer to the right place in the text. If you’re looking for an answer to “when do I even think about this” — you’re in the right article.

Why a post-hoc audit always arrives too late

The typical scenario looks like this: a team builds an agent focused on function — it should summarize documents, handle tickets, query internal systems. Security shows up at the end, as a line item before launch: “let’s do a security review.” By that point the agent already has broad permissions, because nobody deliberately narrowed them — tools were just added as functional needs came up. It already has access to sensitive data, because that was convenient while building it. It runs generated code on the same machine as the rest of the service, because that was faster at the prototype stage.

A security review at that point finds dozens of problems — and every one of them requires rebuilding something that already works. Narrowing a tool’s permissions means rewriting an integration. Moving code execution into an isolated environment means changing infrastructure. Adding human approval for irreversible actions means redesigning a flow users have already gotten used to in the beta. None of this is impossible — but all of it is more expensive than if these decisions had been made three weeks earlier, before anyone had written a single integration.

The worse version of this scenario skips the security review entirely and goes straight to an incident. Teardowns of real AI agent security incidents show the same pattern with striking regularity: an agent with broader permissions than the task required processed content it shouldn’t have treated as an instruction, and executed an action that couldn’t be undone. None of these incidents happened because someone knowingly accepted a risk. They happened because nobody asked, early enough, the question secure agentic engineering asks at the start: what’s the worst this agent does if it’s fooled by content it reads?

That question is cheap to ask at the design stage and expensive to ask after the fact. The rest of this article is one idea worked through in detail: ask it early, at every architectural decision, not once, at the end, as an audit.

What agentic engineering is, and where security sits inside it

Agentic engineering is the discipline of designing, building, and operating AI agents reliably in production — not a single prompt or a single model integration, but the whole lifecycle of a system that pursues a goal autonomously, calling tools, processing context, and taking further steps without a human approving each one individually. In practice it spans layers that constantly cross into each other during design: tool selection and construction, context and memory management, the execution layer (harness) responsible for running actions, evaluation, and observability.

In that arrangement, security isn’t a separate layer sitting next to the others. It’s a criterion present in every one of them. The decision about which tools an agent gets is simultaneously a functional decision (what the agent can do) and a security decision (what’s the worst it does if it’s fooled). The decision about memory architecture is simultaneously a decision about how well the agent retains task context and a decision about whether that memory can be poisoned by a malicious entry that shapes future actions. Splitting these into “the product team decides on function” and “the security team does a review at the end” is exactly the source of the problem described above — it treats as two separate decisions something that was always one.

One distinction is worth making here: secure agentic engineering doesn’t replace security testing or pre-launch reviews. Red-teaming and systematic evaluation remain a necessary part of the process — but their role shifts. Instead of hunting for problems that now need expensive fixing, they verify that decisions made earlier actually hold up in practice. That’s the difference between a test that finds a design flaw and a test that confirms a design holds.

Six points in the process where the outcome of an attack gets decided

Below I break the process of building an agent into six decision points where the architecture either narrows the blast radius of a future attack or widens it — usually unintentionally, because nobody thought about it at that moment. Each point maps to a specific risk class in the OWASP Top 10 for Agentic Applications (ASI01–ASI10) and links to the material that covers the matching defensive technique in depth.

Process point	The question to ask	Related ASI risk	Where it’s covered in depth
Goal and scope definition	What exactly should the agent do — and what should it explicitly not do?	ASI01 (agent goal hijack)	covered below
Tool selection and scope	What tools does this task actually need, not “just in case”?	ASI02, ASI03	least-privilege agent tools
Provenance of tools and data	Where does every component the agent relies on come from — and has anyone verified it?	ASI04 (supply chain)	trust boundaries with malicious MCP servers
Execution environment	Where does what the agent generates or fetches physically run?	ASI05 (unexpected code execution)	agent tool sandboxing and isolation
Input/output boundary	What gets filtered before it reaches the model, and what gets filtered before the model’s output executes?	ASI01, ASI06	guardrails: input/output filtering
Approval and oversight checkpoints	Which actions are irreversible — and who signs off before they run?	ASI08, ASI09, ASI10	defense-in-depth: the full model

Each of these points is worked through below — not as a separate technique to bolt on at the end, but as a question that should be raised at a specific moment in the process, before the next line of code gets written.

Goal and scope definition: where ASI01 begins

The first decision every team building an agent makes is: what should it do? A second question, equally important, is asked far less often: what should it explicitly not do, even if it technically could? That question looks obvious, but in practice it gets lost, because teams define an agent by what it can accomplish, not by the boundaries of what it’s allowed to do.

This gap has a name in the risk taxonomy: ASI01, agent goal hijack. An agent whose task scope was never clearly bounded at the start is vulnerable to a situation where content from the outside — a document, an email, a search result — suggests a new “goal” to it, and it has no built-in criterion to tell that apart from the original task. If the scope was never written down precisely, there’s also nothing to check a deviation against.

The practical design consequence: before any decision about tools or architecture gets made, write down the agent’s task scope in a form that can be enforced in code, not just described in documentation. “This agent summarizes incoming documents and never initiates outbound communication” is a scope that translates into the absence of a message-sending tool. “This agent helps with customer support” doesn’t — because it doesn’t say what the agent isn’t allowed to do.

Tool selection and scope: where ASI02 and ASI03 either get a chance or don’t

This is the single most important decision in the whole process, because it determines how much there is to lose if the previous layer fails. The rule is simple to state and hard to consistently apply: an agent gets only the tools its current task requires, and nothing “just in case.” Every tool beyond that scope is attack surface handed over for free — because if an agent ever gets fooled (and sooner or later it will), the toolset is exactly what decides how bad the consequences are.

This isn’t a checklist to review at the end. It’s a question asked at every new tool during design: what’s the worst this agent does with this tool if it’s used under the influence of an injected instruction? If the answer is “nothing serious” — the tool stays. If the answer is “it wires money” or “it deletes production data” — the tool needs an extra gate before it’s even added to the set.

The full set of techniques — capability scoping, permission gating outside the model, dynamic toolset narrowing per task stage, and human-in-the-loop checkpoints for irreversible actions — is broken down separately in the article on least-privilege agent tools. It’s the direct defense against ASI02 (tool misuse) and ASI03 (identity and privilege abuse) — and, architecturally, a decision that’s cheaper to make at the start than to unwind later, since narrowing an already-deployed toolset means rewriting integrations other things now depend on.

Provenance of tools and data: where ASI04 lives

An agent is rarely built entirely from its own code. It relies on libraries, plugins, tool registries, and MCP (Model Context Protocol) servers supplying it with external capabilities. Every one of these is a component whose integrity the team building the agent doesn’t directly control — and the decision to connect it usually takes five minutes, because “this server does exactly what we need.”

This is exactly the moment where ASI04 — agentic supply chain vulnerabilities — gets settled. An MCP server connected without provenance verification runs with the agent’s privileges and dictates its tool behavior; a substituted or compromised component influences the agent with the same force as if someone had edited its source code — except nobody reviewed that code. The article on trust boundaries with malicious MCP servers covers exactly how rogue servers get in, and how to verify, pin, and isolate them.

The design decision worth making at the start, not after an incident: an allow-list of trusted sources for tools and components the agent can draw on, plus a rule that nothing outside that list gets added without deliberate review. This is the same mechanism OWASP points to in the documented GitHub MCP integration abuse — trust in a ready-made connector turned out to be the weak link, because nobody had asked earlier exactly what privileges that connector was actually getting.

Execution environment: where ASI05 gets decided

If an agent generates or fetches code and runs it, the question “where does this physically happen” is one of the most consequential architectural decisions — and one of the most frequently postponed, because at the prototype stage “it runs on the same machine” is simply faster. The problem is that moving code execution into an isolated environment after the system is built requires an infrastructure change that’s much cheaper to design in from the start.

This is where ASI05 — unexpected code execution — gets settled. Executing arbitrary, malicious, or simply buggy code bypasses every higher security layer’s logic: an attacker no longer needs to convince the model of anything, since code that’s already running just does what it does. The guide to agent tool sandboxing and isolation breaks down the available isolation primitives — from shared-kernel containers to microVMs — and shows when each one is the right call. The key design decision: untrusted code generated by the agent never lands on the same machine as the rest of the system, no matter how much faster that would make the first version.

The input/output boundary: where ASI01 and ASI06 meet filtering

Every piece of content an agent pulls in from outside — a page, a document, a tool call result — and every response it generates before it’s executed passes through a control point, or it doesn’t. Whether that point exists at all is decided while designing how data flows through the agent, not during a security review.

Input and output filtering is the first line of defense against ASI01 (goal hijack) at the content level, and against ASI06 (memory and context poisoning) — if content headed for long-term memory passes through the same check as content headed into the current context. The guardrails article covers the mechanics: labeling untrusted content, scanning input for known patterns, and checking model output before it executes. One important architectural caveat: this is a first-sieve layer, not a last line of defense — and designing a system as if guardrails were sufficient on their own is a mistake I come back to in the next section.

Approval and oversight checkpoints: where ASI08, ASI09, and ASI10 live

The final decision point covers what happens when every earlier layer fails at once. Which of an agent’s actions are serious enough to require explicit human sign-off before they run? Which signals should halt the whole pipeline before an error propagates further? Who — and how — can shut an agent down if it starts behaving outside the expected pattern?

These questions answer three related risks on the ASI list: cascading failures (ASI08), human-agent trust exploitation (ASI09), and rogue agents (ASI10). The common thread: no single defensive layer holds forever, so the architecture needs a point where a human or an oversight mechanism takes control before the consequence becomes irreversible. That’s exactly the topic defense-in-depth for autonomous AI agents develops as a coherent model — five layers of defense arranged so a gap in one doesn’t bring down the whole system.

What real incidents confirm: one specific point fails, not the whole architecture

The six points above aren’t a theoretical grid — every documented AI agent incident maps to a specific process point where a decision was made badly, or never made at all.

EchoLeak (CVE-2025-32711, responsibly disclosed by researchers at Aim Security and publicly detailed in June 2025; Microsoft had already patched it server-side in May 2025, before public disclosure) is a failure of exactly the fifth point: the input/output boundary. Content from an attacker-crafted email landed in the same context Copilot used to process the user’s sensitive data, and the output mechanism allowed an auto-fetched image whose URL carried exfiltrated data. No individual control was skipped by accident — the architecture simply had no point that cleanly separated untrusted content from privileged context and controlled where the model’s output could go. The full teardown of this and two other incidents is covered separately in AI agent incident teardowns.

The GitHub MCP tool abuse Invariant Labs documented in May 2025 is a failure of the third point instead: verifying the provenance and scope of components. The access token wired into the agent spanned far more repositories than the task required — and whether that scope was even needed is a question that should have been asked when the integration was connected, not after someone demonstrated it could be abused.

The TeamPCP / LiteLLM campaign from March 2026, which compromised a widely used AI gateway proxy and harvested credentials across many systems at once, shows the same pattern one level up: as the connective infrastructure of the agent stack — gateways, tool registries, MCP servers — consolidates, one compromise pays off against many victims at once for an attacker. That’s exactly the risk the allow-list of trusted component sources from the third point defends against — except here the blast radius spans an entire organization’s connected agents, not one.

The common denominator across all three: none required a “bad model” or a sophisticated exploit. Each required one skipped decision, early in the process — a decision that would have cost one question raised in an architecture meeting at design time, and cost considerably more after the incident.

What layered defense looks like when it’s part of the process, not an add-on

The diagram below shows a simplified cross-section of these layers at the moment a real attack tries to reach a privileged action from input content. It isn’t a checklist to tick off after launch — it’s the shape that results from decisions made at the earlier stages above: tool scope (input filter and privilege), execution environment (isolation), and approval points (human checkpoint).

Notice the order in which these layers come into existence during the engineering process — it isn’t arbitrary. The input filter and the permission scope get decided together, at the tool-selection stage. Isolation gets decided at the execution-environment design stage. Human approval gets decided at the stage of mapping irreversible actions. None of these decisions waits for someone to penetration-test the finished system — by that point, all of them are already part of the architecture.

Why the order of decisions matters, not just their presence

Teams trying to retrofit security into an already-built agent often reach for guardrails first — because that’s the easiest layer to add without touching the architecture: a few rules in the system prompt, maybe a safety classifier. It’s an understandable reflex, but it’s the reverse of what actually limits damage.

Guardrails are a language-layer control operating at the same level as the content they’re meant to filter — a rule in the system prompt is text sitting in the same context where an injected instruction lands, and the model weighs both on its own terms. A sufficiently well-crafted attack can defeat that rule. Narrowing the toolset works differently: a tool the agent doesn’t have can’t be misused by any wording at all, because the decision was made outside the model’s reach, in code injected content has no access to.

Hence the practical priority order when designing (and when fixing) agent security: tool and permission scope first, because it delivers the biggest effect for the smallest cost and limits the damage before anything even goes wrong. Then the execution environment, if the agent runs code. Then approval checkpoints for irreversible actions. Guardrails and content filtering come last on this priority list — not because they’re unnecessary, but because as a noise-filtering layer they only make sense once a solid foundation already exists underneath: narrow permissions, isolation, and clearly defined oversight points. This same ordering logic runs through the full defense-in-depth model — here I’m only showing that it follows directly from when, in the process of building an agent, each of these decisions actually gets made.

Verification as part of the process, not a one-off test

Secure agentic engineering doesn’t end with the architecture design. Decisions made at earlier stages need checking — not once, before the first launch, but on a recurring basis, because an agent’s toolset grows with the product, and every new tool is new attack surface that someone had to deliberately assess.

This is where red-teaming and systematic evaluation fit into the engineering process as a permanent part of it, not a one-time gate before launch. Offensive tools — Garak, PyRIT, or Promptfoo among them — automate checking whether decisions about permission scope and isolation actually hold under a coordinated attack, not just on paper. The practical pattern: any change to an agent’s toolset or its permission boundaries triggers the same suite of offensive tests the original version went through — exactly the way a code change triggers unit tests. Security stops being a point on a timeline and becomes a gate in the pipeline every subsequent version has to pass through.

A map: where each ASI risk enters the agent lifecycle

Put together, it looks like this — not a list of techniques to implement in any order, but a map of decision points in the process of building an agent, each one responsible for a specific class of risk.

1. Goal and scope definition — a written, code-enforceable task scope. Prevents ASI01 at the source, before any tool even exists.
2. Tool selection — capability scoping, permission gating outside the model, human approval for irreversible actions. Direct defense against ASI02 and ASI03.
3. Component provenance verification — an allow-list of trusted sources for tools and MCP servers, pinned versions. Defense against ASI04.
4. Execution environment design — isolation for code the agent generates or fetches. Defense against ASI05.
5. Input/output boundary — labeling untrusted content, filtering output before it executes. Defense against ASI01 at the content level and against ASI06.
6. Approval and oversight checkpoints — circuit breakers, rate limiting, a kill switch, human sign-off. Defense against ASI08, ASI09, and ASI10.
7. Recurring verification — red-teaming built into the release pipeline, triggered by every toolset change, not run once per project.

Each of these seven points corresponds to a moment in the engineering process where a specific design question gets asked — not to a checklist someone ticks off afterward. That’s the entire difference between security built in and an audit.

Where to start if you’re building an agent from scratch

If you’re at the start of the process, the order in which it’s worth asking the questions above matches the order in which the architectural decisions actually get made. Start by writing down the task scope in a form that translates into constraints in code, not just a description in a design document. For every tool, before it’s added to the set, ask what the worst outcome of its misuse would be. Decide on the code execution environment before writing the first integration that generates that code. Map out irreversible actions and put an approval point in front of them before anyone starts relying on a flow that doesn’t have one. Only at the end of this list add content filtering and guardrails, as a noise-filtering layer sitting on top of an already-solid foundation.

None of these steps requires waiting for the rest to be complete — each one, on its own, shrinks the blast radius of a potential attack. But the order in which you take them decides whether you’re building security into the architecture, or bolting it on afterward, at a higher cost, with a worse outcome.

---

This article is part of the “Defending & Hardening” area at secagentlabs and serves as a connecting frame — it shows at which stage of the engineering process each security decision gets made, and points to the material that covers each technique in depth. Risk classifications are drawn from the OWASP Top 10 for Agentic Applications (ASI01–ASI10), published December 9, 2025. It is not configuration guidance for a specific product — every deployment needs to be adapted to its own threat model.