Enterprise software teams are shipping code faster than any point in computing history, and AI-assisted development is the accelerant. The appeal is real: faster iteration cycles, lower entry barriers for non-specialist contributors, and the ability to prototype in hours rather than weeks. But velocity without governance creates a different kind of liability that accumulates silently inside production systems, often invisible until an audit or incident forces the question.
In this episode of AppDevANGLE, I spoke with Nico Waisman, CISO at XBOW, about the security consequences of AI-assisted development and what enterprises need to get right before autonomous AI tooling becomes a permanent fixture in their pipelines. XBOW operates at the intersection of AI and offensive security, applying autonomous techniques to identify and validate exploitable vulnerabilities in enterprise environments.
Our conversation explored five major themes: why AI-generated code introduces a structurally different security challenge than human-written code; how feedback loops and pipeline discipline are the real solution to AI-native technical debt; what organizations consistently underestimate when they attempt to build autonomous AI security tooling in-house; how the “paved road” philosophy offers a scalable model for security by default; and why AI governance is a cultural and organizational problem as much as a technical one.
The broader market signal is this: the enterprise security conversation is shifting from perimeter defense and static scanning toward continuous, automated validation embedded in the development pipeline itself. That shift is not optional; it is being driven by the volume of AI-generated code now entering production systems.
AI-Generated Code Creates Security Debt at Machine Speed
The core problem with AI-assisted development is not that AI writes bad code. It is that AI writes code without organizational memory. A model has no understanding of a specific company’s risk tolerance, its compliance history, its legacy architectural constraints, or the security decisions its engineering team made and documented over years. That context gap produces a category of security debt that is structurally different from the kind accumulated through developer shortcuts or time pressure.
When human engineers take shortcuts, there is usually an implicit awareness of the compromise. AI-generated code carries no such signal. The output looks clean, often compiles without warnings, and passes basic functional tests. The vulnerabilities embedded in dependency choices, architectural assumptions, and missing edge-case handling don’t announce themselves.
Waisman framed the problem plainly: “Pretty much everyone is an engineer. And so that means that you’re starting to collect more and more technical debt.” The democratization of code generation is not inherently a security problem, but it removes the human layer of experience that previously served as an informal check on certain categories of risk. Combined with the speed at which AI-assisted teams ship, the inventory of unexamined decisions in production systems grows faster than security teams can inspect.
The implication for enterprises is that the existing model (periodic penetration testing, manual code review, pre-release security gates) was designed for a cadence that no longer matches reality. Those practices remain valuable, but they are insufficient as primary controls when code is being generated and deployed continuously.
Feedback Loops Are the Structural Answer to Automated Debt
If the problem is that AI generates vulnerabilities faster than humans can find and fix them, the logical response is to automate finding and fixing at the same speed. That is the direction the market is moving. But Waisman was careful to distinguish between automation that actually closes the loop and automation that creates a new category of risk.
Remediation by AI introduces its own uncertainty. A patch generated by a model may fix the immediate vulnerability while altering the behavior of the surrounding code or creating a new attack surface. Without retesting, there is no way to know. “The patch might not fix the problem, which is normal with humans too,” Waisman observed. “We have seen in many years where Microsoft came out with a new patch and suddenly they didn’t fix all the different angles to trigger it, and so the bug continues after the patch.”
The engineering response is to build closed feedback loops: identify the vulnerability, generate a patch, retest that specific vulnerability, and validate that system behavior has not changed in ways that introduce new exposure. This requires not just functional testing but security-specific regression testing at the point of change. It is a more demanding pipeline requirement than most enterprise teams currently operate.
This distinction matters because many organizations are treating AI-assisted remediation as a destination rather than a component of a larger system. The remediation step alone, without automated retesting and behavioral validation, shifts risk rather than eliminating it. The feedback loop is the unit of control, and enterprises that invest only in vulnerability discovery without closing the loop are building a false sense of coverage.
Building Autonomous AI Security Tools In-House Is Harder Than It Looks
The build-versus-buy question is a live debate in enterprise security and application development teams right now. The appeal of internal tooling is straightforward: control over model behavior, customization to specific environments, no dependency on a vendor’s roadmap, and the perception that modern AI tools make it faster and cheaper to build than it used to be. That perception is typically accurate for the prototype phase and misleading for everything that follows.
Waisman described the pattern directly: “It’s really easy to build a PoC, something that works in a specific target. It’s quite simple to do it.” He went on to describe what happens next: “Moving that into something that is productized, that can run in a real environment, and then later you have to maintain that — this is where the real challenge is happening.” Multiple XBOW customers built competing internal tools, encountered safety and reliability problems, and returned to evaluate the commercial product.
The underlying issue is LLM non-determinism. Any application built on a large language model will behave differently when the underlying model changes. That means every model update, whether driven by a vendor’s training cycle, a provider’s deprecation decision, or an internal upgrade, becomes a regression risk for the application built on top of it. Managing that requires benchmarking infrastructure, continuous evaluation, and the operational capacity to update benchmarks as model capabilities evolve. That is a significant ongoing engineering commitment that does not appear in most build-versus-buy analyses.
Waisman noted that XBOW released approximately eighty public benchmarks for web penetration testing, only to find that GPT-4 was solving 85% of them at release. As model capabilities advanced, those benchmarks became insufficient, requiring the team to build harder, more realistic evaluation sets. This kind of benchmark maintenance is non-trivial and illustrates the depth of operational investment required to run autonomous AI security tooling reliably over time.
Security by Default Scales Where Security by Review Cannot
The traditional model of security review, where a team of specialists examine code, infrastructure, or configurations before they ship, was already under pressure before AI entered the development pipeline. At the deployment cadence of modern engineering-led organizations, human review at the gate is not a scalable control. AI-assisted development accelerates that incompatibility considerably.
Waisman pointed to a philosophy that emerged from Netflix and has been refined at engineering-driven organizations including Lyft: the “paved road” or “golden path.” The concept is architectural: make the secure path the easy path. When the patterns, libraries, and deployment procedures that meet security requirements are also the fastest way to ship, engineers and AI tools alike will naturally follow them. When deviation from the standard path introduces friction, the default behavior trends toward compliance.
“If you go through the road, you will move as fast as you can and you want,” Waisman explained, “and that means that you have to follow certain procedures, certain patterns, certain libraries that the security team and engineer team has built for you.” The key insight is that this model works for AI-generated code for the same reason it works for human developers: neither is making a deliberate choice about security when given a fast path. They are optimizing for velocity, and the paved road aligns that optimization with a secure outcome.
This approach also reduces alert noise, which is a meaningful operational problem. Our survey data shows that 21.8% of respondents identified too many disconnected security tools causing alert fatigue as the biggest barrier to achieving a unified view of application security posture across multi-cloud or hybrid environments. The paved road approach addresses this partly by reducing the volume of deviations that need to be flagged, because secure behavior is the default rather than something that must be monitored and enforced after the fact.
Exploitation Risk, Not Vulnerability Count, Is the Right Metric
Security teams have historically measured their posture by the number of vulnerabilities identified and the speed of remediation. That metric made sense when the volume of issues was manageable and the relationship between vulnerability and risk was reasonably linear. At the scale of AI-generated code, counting vulnerabilities without prioritizing by exploitability produces an unworkable signal.
Waisman was direct on this point: “To me, it’s all about risk. It’s not even whether you’re using something or not. And the point to me is, exploitation is the ultimate risk. Can these bugs actually be exploited or not?” This framing reflects his background in offensive security, a discipline that evaluates risk from the attacker’s perspective rather than from the inventory of potential problems. A vulnerability that cannot be exploited in the specific context of a given application carries different weight than one that can.
This matters for how enterprises allocate remediation resources. When AI can generate and ship code at high volume, the number of potential vulnerabilities in any given production environment will grow faster than teams can address them in sequence. Prioritization based on exploitability becomes a necessary triage mechanism rather than an optional refinement.
The practical implication is that security teams need visibility into actual exploitability, not just theoretical risk scores derived from CVE severity ratings. That requires a different kind of tooling: automated systems capable of attempting exploitation in safe, controlled conditions and reporting on what succeeds. This is the specific capability that autonomous penetration testing platforms are designed to provide, and it represents a meaningful shift in what enterprises should expect from their security tooling.
AI Governance Cannot Be Owned by a Single Team
The question of who owns AI governance in an enterprise engineering organization does not have a clean answer yet, and the ambiguity is creating real risk. The common instinct is to assign it to a security team, an AI center of excellence, or a platform engineering group. That instinct is wrong, not because those teams lack capability, but because centralized ownership of a distributed problem creates a gap between policy and practice.
Waisman drew an analogy to the evolution of reliability and security ownership: “Modern teams have been owning that idea of reliability doesn’t belong to an SRE, it belongs to the team that is building the software. Security doesn’t belong to a security team; it belongs to a team that actually is building the software. And same is going to be with AI and testing.” The shift from centralized specialist ownership to distributed team ownership is a cultural change that takes time, but it is the only model that scales.
This aligns with survey data, which found that 18.3% of respondents identified siloed teams — development, operations, and security — as the biggest barrier to a unified application security posture. When AI governance is added to a siloed structure, it inherits the same fragmentation problem. Strategy can be set centrally, but execution has to be embedded in the teams building and operating AI-assisted systems.
Analyst Take
The enterprise security market is at an inflection point driven not by a new class of attacker or a novel vulnerability category, but by a change in the production rate of the software it is supposed to protect. AI-assisted development has compressed the time between design decision and production deployment to a degree that makes most existing security practices structurally insufficient. That is not a criticism of those practices. They were designed for a different environment, and the environment has changed.
The traditional approach of reviewing code before it ships, running periodic penetration tests, and triaging vulnerabilities by severity was always a heuristic approximation of real security posture. It worked well enough when human engineering velocity was the primary constraint on how fast systems could change. It will not work well enough when AI is generating and modifying code continuously. The gap between the speed of change and the speed of security review will widen unless security itself is automated and embedded in the pipeline rather than applied as a gate around it.
What enterprises need to recognize is that autonomous security tooling (continuous vulnerability detection, AI-assisted remediation, automated retesting, and exploitability validation) is not a future-state aspiration. Organizations operating at any significant engineering velocity are already beyond the threshold where manual approaches can maintain adequate coverage. The question is not whether to automate security, but whether the automation is properly governed, benchmarked, and embedded in a pipeline with closed feedback loops.
The risks and constraints are real. AI-based security tools introduce their own non-determinism, their own maintenance burden, and their own governance requirements. Building in-house without the operational infrastructure to support model updates and benchmark evolution is a path to a different kind of technical debt. And the cultural change of distributing security ownership across engineering teams rather than concentrating it in specialist groups takes longer than deploying a tool.
The larger shift is this: the enterprise security boundary is moving from the perimeter of the application to the pipeline that produces it. Enterprises that invest in automated, closed-loop security embedded in their development process will be positioned to manage AI-generated code at scale. Those that continue to treat security as a checkpoint rather than a continuous system will find that the gap between what they ship and what they can verify grows faster than they can close it.
