1 Key Risks
The most critical security risks an operator inherits when deploying this agent in its documented default configuration. The dominant risk concentrates in tool execution authority that persists even when the model's safety behavior is overridden, combined with approval gates that are architecturally bypassable in a single step.
2 AIRQ Scores
The four headline scores quantify how exposed the agent is, how damaging a successful attack would be, and how much the agent’s own controls reduce that risk. Pulumi Neo sits at the boundary between moderate and elevated risk, with defense controls providing meaningful but incomplete mitigation of its infrastructure-scoped blast radius.
The agent lands in the Humble Providers quadrant with attack surface just below the midpoint and blast radius in the upper-moderate band, held back from Humble Providers by a single-step approval bypass and absent prompt filtering.
Each axis measures a distinct dimension: attack surface exposure out of ten, blast radius reach out of ten, defense controls out of fifteen, and the composite AIRQ score out of fifteen.
| Metric | Score | Comments |
|---|---|---|
| AIRQ Score | 5.93 | Composite reflects meaningful vendor-implemented isolation partially offset by demonstrated safety-boundary bypass and approval-gate weakness. |
| Blast Radius | 6.38 / 10 | Infrastructure-scoped reach constrained by Firecracker isolation and RBAC-bounded credentials, though shell and network access remain unrestricted within the container. |
| Attack Surface | 4.8 / 10 | Moderate surface driven by full-shell tool execution and trifecta floor application, with session-scoped memory and managed configuration keeping remaining surfaces at Band 2. |
| Defense Controls | 6 / 15 | Firecracker microVM and audit logging provide the strongest controls; input guardrails and output filtering remain absent on the documented default. |
3 Attack Surface
Attack surfaces are the entry points and interaction patterns through which adversarial input can reach the agent’s reasoning loop and steer its behavior. The dominant exposure is the full-shell tool execution surface operating without interpreter sandboxing, while remaining surfaces benefit from the managed cloud deployment model and session-scoped memory.
One surface reaches Band 3 reflecting full shell access, while the remaining nine sit at Band 2 or below reflecting the managed cloud architecture and vendor-curated integrations.
Each row maps a scored surface to its architectural exposure band and the supporting evidence that grounds the assessment.
| Surface | Score | Comments |
|---|---|---|
| User Input | 2 / 4 | Multiple input channels including web UI, CLI, Slack, and GitHub with no prompt shield or injection detection; safety relies on the underlying model guardrails. [9] |
| External Data | 2 / 4 | Ingests infrastructure state and MCP tool outputs from vendor-managed sources with centralized credential handling and per-org configuration. [7] |
| Memory | 1 / 4 | Session-scoped context only with no cross-session persistence; task cache expires after idle periods, eliminating persistent memory poisoning vectors. [5] |
| Reasoning | 2 / 4 | Multi-step reasoning with visible chain-of-thought constrained to declared task scope; delegates to Claude on AWS Bedrock with vendor-managed grounded context. [2] [3] |
| Planning | 2 / 4 | Multi-step planning with user-visible plan and configurable approval before execution; Plan mode available for research-before-action workflows. [5] |
| Tool Execution | 3 / 4 | Full shell access via cloud CLIs inheriting operator privileges with approval gates; MCP command filtering with timeouts but no interpreter sandboxing. [8] |
| Orchestration | 2 / 4 | Single-agent task execution with background continuation; no subagent spawning, scheduling, or concurrent multi-agent orchestration. [5] [11] |
| Inter-Agent | 2 / 4 | Accepts delegation from external agents via MCP bridge tool with restricted toolsets; connects to vendor-curated MCP catalog with org-admin configuration. [7] [12] |
| Output Processing | 2 / 4 | Output through console, CLI, and pull requests with credential separation from model context; no documented DLP or exfiltration blocking on outputs. [7] |
| Configuration | 2 / 4 | Configuration through Pulumi Cloud settings UI and org-level admin setup; MCP integrations from managed catalog requiring administrator approval. [7] |
The Lethal Trifecta is triggered when an agent processes untrusted content, accesses private data, and communicates externally in the same session — the three conditions that turn an isolated prompt injection into full-chain exfiltration. Pulumi Neo ingests third-party MCP outputs and infrastructure state, accesses cloud credentials and deployment secrets through CLI integrations, and transmits bytes externally through unrestricted outbound HTTP and cloud API calls from within the Firecracker container.
Pulumi Neo exhibits all three of these conditions in its documented default configuration:
- Untrusted input — MCP tool outputs from six third-party services and infrastructure state from cloud providers enter the reasoning loop without dedicated filtering. [7]
- Sensitive data — CLI integrations grant access to AWS, Azure, GCP, and Kubernetes credentials managed through Pulumi ESC secrets store. [8]
- External egress — Unrestricted outbound network from within the container and authenticated cloud API calls through CLI integrations provide external egress channels. [8]
4 Blast Radius
The blast radius is what an attacker who controls the agent can reach — which systems they touch, which credentials they read, and which actions they take without operator approval. Compromise of the agent grants shell-level code execution, unrestricted network egress, and access to cloud provider credentials within the Firecracker container boundary.
Three factors reach the upper band reflecting full shell, network, and credential access within the container, while file system and autonomous action are constrained by the isolation boundary.
Each row maps a blast factor to the demonstrated or documented capability reach and the containment boundary that limits lateral movement.
| Factor | Score | Comments |
|---|---|---|
| Code execution | 3 / 4 | Full shell and Python with unrestricted standard library access inside the Firecracker microVM; locally inherits all operator-authenticated CLIs. [8] |
| File system access | 2 / 4 | Cloud mode provides no host filesystem access; CLI mode scopes tool execution to the specified working directory with symlink-escape hardening. [8] |
| Network access | 3 / 4 | Unrestricted outbound network access from within the container with access to cloud metadata service endpoints documented by independent research. [1] |
| Credential access | 3 / 4 | Inherits all operator-authenticated CLIs, environment variables, and kubeconfigs; IAM credentials accessible from within the execution environment. [8] |
| Autonomous action | 2 / 4 | Background task execution continues when operator closes browser, but destructive actions require approval in the default Review mode. [5] |
| Deployment access | 2 / 4 | Infrastructure deployment via pulumi up requires explicit operator approval in Review mode; the agent cannot autonomously trigger production changes by default. [6] |
5 Defense Controls
Defense controls are what the agent’s own architecture does to detect, contain, and report attacks before they reach the operator’s systems. The vendor provides Firecracker microVM isolation and comprehensive audit logging as default controls, while input filtering and output guardrails remain absent from the documented architecture.
Higher scores indicate stronger vendor-implemented safeguards; the total reflects meaningful isolation and monitoring with gaps in input and output filtering.
Each component is scored on what the vendor ships as a default control, with non-default options noted separately as potential hardening targets.
| Component | Score | Comments |
|---|---|---|
| Input Guardrails | 0 / 3 | No documented prompt shield, ML-based injection detection, or instruction hierarchy separation; safety relies on the underlying model system prompt. [4] |
| Execution Isolation | 2 / 3 | Firecracker microVM with all Linux capabilities dropped, NoNewPrivs enabled, no Docker socket, and no host filesystem access in cloud mode. [1] |
| Action Controls | 1 / 3 | Review mode requires approval for preview and deploy by default, but Auto mode removes all approval gates via a single task-creation setting. [5] |
| Output Guardrails | 1 / 3 | Credentials architecturally separated from model context and encrypted at rest with per-org keys, but no documented DLP or output content filtering. [7] [14] |
| Monitoring | 2 / 3 | Pulumi Cloud audit logs capture all actions with SIEM export capability in CEF format; task transcripts preserved with full conversation history. [10] [13] |
6 Hardening Tips
Concrete actions an operator can take to reduce the risks reported above, grouped by which defense control each tip strengthens. Priority hardening targets are deploying a prompt-injection detection layer before the reasoning loop and restricting network egress from the agent container to break the exfiltration chain.
Input Guardrails
Input guardrails intercept adversarial content before it reaches the reasoning loop.
- Policy Require all Neo tasks to pass through an organizational prompt-review queue before execution begins — counters absent input guardrails on the default configuration.
- Configuration Enable session length limits and periodic context resets to prevent sustained conversational manipulation — counters demonstrated context drift attack vector.
- Engineering Deploy a prompt-injection classifier upstream of Neo's reasoning loop to flag authority-escalation patterns in conversation history — counters absent input guardrails.
Execution Isolation
Execution isolation contains what a compromised agent can do on the host.
- Policy Mandate read-only mode for all non-deployment investigation tasks to cap Neo's execution authority by organizational default — counters full shell access within the container.
- Configuration Restrict CLI integration scope to read-only cloud operations where full write access is not operationally required — counters unrestricted shell access within the container.
- Engineering Build an egress proxy that allowlists only known-good outbound destinations from the Neo container — counters unrestricted network access from within the execution environment.
Action Controls
Action controls govern which tools and actions the agent can invoke autonomously.
- Policy Prohibit Auto mode organization-wide via Pulumi Cloud policy and require manager approval for any task-mode elevation — counters single-step approval bypass.
- Configuration Configure Review mode as the organization-wide floor with no user override to Auto — counters Action Controls weakness with documented single-step bypass.
- Engineering Implement a secondary out-of-band approval channel for destructive operations that cannot be bypassed from within the Neo conversation — counters context drift exploitation path.
Output Guardrails
Output guardrails inspect what the agent sends to other systems and users.
- Policy Define a credential-access policy that blocks Neo tool calls targeting metadata services and credential file paths — counters absent output guardrails on default configuration.
- Configuration Block agent access to the cloud metadata service endpoint within the container network policy — counters demonstrated credential extraction through metadata service access.
- Engineering Build a DLP scanner on Neo's tool-call output stream that redacts credential patterns before they reach the conversation context — counters absent output content filtering.
Monitoring
Monitoring captures what the agent did and surfaces anomalies for review.
- Policy Require audit log export to organizational SIEM for all Neo-enabled organizations regardless of pricing tier — counters monitoring gap on non-Enterprise tiers.
- Configuration Enable continuous audit log export to an external SIEM destination and configure alerting on credential-access tool calls — counters silent credential extraction risk.
- Engineering Build behavioral anomaly detection that flags conversation-length outliers and escalating tool-call patterns indicative of context drift — counters unmonitored safety boundary dissolution.
7 References
The evidence base behind every score and finding in the profile, grouped by source type so the reader can verify any claim. Numbers in brackets throughout the report (e.g. [7, 13]) refer to entries below, listed in citation order.
Selected Vulnerabilities
- Context Drift on Pulumi Neo Independent red-team writeup demonstrating safety boundary dissolution on Pulumi Neo through sustained conversation, extracting IAM credentials and executing reverse shells within the Firecracker container.
Selected Research
- Pulumi Neo Compliance Automation The New Stack coverage documenting Neo compliance remediation capabilities including configurable guardrails and approval workflows for infrastructure changes.
- Grounded AI Architecture Vendor blog explaining Neo grounded context architecture that reasons from infrastructure state rather than generic prompts, reducing hallucination risk.
Vendor Documentation
- Pulumi Security Page Vendor security page documenting SOC 2 Type II certification and linking to the platform security whitepaper covering cryptographic controls and RBAC.
- Neo Task Documentation Official task documentation covering Review, Balanced, and Auto approval modes, permission model, task isolation, and background execution behavior.
- Neo Read-Only Mode Blog post introducing read-only mode that caps Neo permissions at task creation, preventing infrastructure mutations while allowing analysis and PR creation.
- Neo MCP Integrations Documentation of MCP integration architecture including credential encryption at rest, per-org key management, and model-level credential isolation.
- Neo CLI Mode CLI mode documentation showing local tool execution inheriting operator authenticated CLIs, environment variables, and kubeconfigs with path containment.
- Neo Getting Started Getting started guide documenting the RBAC permission model, read-only option, and the principle that Neo never gets more access than the authenticated user.
- Pulumi Cloud Audit Logs Audit log documentation covering immutable event recording, SIEM export in CEF format, and Enterprise tier access requirements.
Other Sources
- Neo Product Page Primary product page documenting approval workflows, full audit trail, multi-cloud visibility, and IDE integration via MCP server.
- Neo Integrations Blog Blog post introducing MCP server integrations and CLI integrations covering six third-party services and four cloud provider CLIs.
- Pulumi SOC 2 Milestone Blog post announcing SOC 2 Type II certification completion, documenting the audit scope and ongoing compliance commitment.
- Pulumi Cloud Security Whitepaper Platform security whitepaper detailing three-tier key hierarchy, role-based access control, audit logging architecture, and multi-tenant isolation controls.