Episode #293 - AppSec's Reality Gap

While Seth and Ken are recovering from their trip to Vegas for DEF CON and BlackHat, we’ll dive into last week’s episode of Absolute AppSec. In the 293rd episode of Absolute AppSec, we dove deep into the practical application of security in an organization’s SDLC, inspired by a recent article from Venture in Security. Covering a range of issues from gaps in contextual understanding to disingenuous vendor claims, Seth and Ken share their experiences dealing with small and large organizations with varying levels of maturity. Some degree of nihilism is warranted, but recent developments using generative AI are cause for optimism in the space. To find this episode, go to https://www.youtube.com/@AbsoluteAppSec, or find us wherever you get your podcasts.

The main topic of this episode is an article from Venture and Security by Nilay De Mello, a security engineer at Datadog, titled ‘AppSec/ProdSec’s reality gap: why theory doesn’t match practice’. Ken and Seth were particularly struck by the article's core premise that says that application security operates in a space where theory meets reality, and it often does not match up neatly. Seth noted that this succinctly describes the problem faced by AppSec consultants and engineers, where abstract concepts learned at conferences often don't directly apply to daily work with custom applications. The article highlighted a common issue: what people claim works in conference talks (like security champion programs or tooling) often doesn't translate to real-world success, or it's presented with a rose-colored lens. He emphasized a critical gap discussed in the article: the disconnect between purported successes and actual delivery. Ken then elaborated on another gap identified in the article, the challenge of gaining context in application security. He pointed out that AppSec professionals need to ensure secure libraries and code, often relying on automated tooling like SAST products. When these tools act as gates in the development pipeline, they cause blockages and lead to a backlog of bug tickets, burning political capital with developers. Ken strongly advocates for guardrails instead of gates – providing security controls that enable development velocity rather than constrain it. He acknowledged that with guidance instead of gating, there's a risk of developers ignoring advice, but that hard gates often lead to an unmanageable backlog and frustrated teams.

Ken then discussed the architectural aspect of security advice, citing his past experience with securing GitHub's Codespaces. In Ken’s view, there is astonishing complexity to interconnected systems and a need for tools that can quickly summarize and provide meaningful data from various sources to inform security decisions, especially considering factors like cost and latency. Seth agreed that the ideal situation is to have quick access to context to answer questions. He also acknowledged a point made in the chat that many AppSec professionals resort to managing scanners and pushing teams to fix findings from those scans, even if they lack real security impact. Seth expressed a desire to move beyond this, focusing on collaborating with developers on more relevant security issues such as securely implementing OAuth or new frameworks. Ken believes that the future of AppSec doesn't involve manually sifting through scanner results; instead, AI-powered tools will handle the bulk of vulnerability identification, leaving human AppSec professionals to focus on high-signal risks and act as internal consultants who provide guidance and manage bug bounty programs, incident response, developer training, GRC requirements, and design considerations for new features. The increasing volume of data, e.g., 17,000 diffs in a Java application over a month, makes manual analysis impossible. The current conditions underscore the need for AI to summarize and contextualize information to identify true risks. Ken expressed concern about current market conditions, where budgets for headcount in security are problematic, despite the growing need for AppSec and ProdSec expertise due to LLMs and AI-integrated IDEs. Building code with LLMs can lead to slip-ups, like wildly different changes across multiple files instead of unified logic, and non-skilled individuals using these tools for code generation are a particular concern. Ken warned that some organizations are treating AI as a “panacea," leading to headcount reductions and shipping more code with potentially unaddressed security issues instead of solving the problems with more AppSec and ProdSec eyes.

Seth agreed with the article's conclusion that the fundamental challenge in AppSec is philosophical, not just technical. He stressed the importance of aligning security philosophies with engineering workflows and customizing AppSec programs to an organization's unique context, something that many tools fail to do. He also noted the constant battle against new CISOs or CTOs who might change tooling or deprioritize security because of a belief that security is a “solved domain” with the presence of security tools like SAST, SCA, and DAST. These tools, though, fail to cover emerging threats like those in generative AI, and they fail to provide necessary context. He described a common scenario where security champion programs, when initially introduced, were often mishandled due to a lack of proper guidance, leading to problems that aren't widely discussed publicly. Seth then brought up the idea of supporting educated C-level decisions, noting that AppSec is so technical that executives can get lost in what he calls the “vendor sauce." He recounted how his own eyes glaze over when discussing topics he's not deeply interested in, illustrating the challenge of communicating complex security concepts to non-experts. He expressed frustration that many security professionals still don't understand the necessity of manual code reviews or why a simple DAST scan isn't enough, perpetuating a reliance on tooling that leaves significant gaps. He argued that most breaches stem from simple misconfigurations and application security issues that could be caught with the right eyes. Tooling should support human security professionals, offloading mundane tasks and feeding them with the right information to empower their meaningful work. Ken then proposed a solution: focus on a single, most pressing problem set within an organization and use new technical capabilities, like LLMs, to solve it. He emphasized avoiding what Seth calls “a solution without a problem.”

Do you know someone going back to school who needs a place to take notes? Well, the Absolute Appsec merch store has you covered. Pick out a size for the student who needs some swag in their lives.

And, if you have thoughts you’d like to share with Seth and Ken and the Absolute AppSec audience, join us in Slack. Show topics often originate as discussion points with the Slack audience, and the newsletter incorporates that discussion as well. So, join in there to offer ideas for what you’d like the podcast to cover as well as pick the brains of a number of industry experts.

Stay Secure,

Seth & Ken

Related Episodes

https://www.youtube.com/watch?v=3AcL_37gnhY - Episode #170 - Security Basics, Social Engineering, Plan for Failure - Discussion of the mixed signals that security industry sends to the rest of the world highlights one of the problems at the heart of the appsec ideal/reality gap.

https://www.youtube.com/watch?v=lGSFHzkEJcI - Episode #132 - Supply Chain Attacks, What I Wish I Knew Starting in Security – Seth and Ken’s discussion of lessons they’ve learned in the course of their careers highlights a lot about how communication helped improve results when you come in for the first time to a new organization or pentesting engagement. Asking the right questions especially can really help focus on and address what the people in an organization every day see as the security risks.

https://www.youtube.com/watch?v=M_cPSszRFb0 - Episode #86 - Rohan Joshi - QA Security Testing, Security Champions, Paypal Vulnerabilities - Rohan Joshi joins Seth and Ken to discuss building out a security program across an organization and the problems that arise. What happens to a security team’s relationship with the organization when they use their credibility budget on tasks or scanners that dev teams don’t like?

Absolute AppSec Happenings

https://arxiv.org/abs/2508.03385 - “Can We Fix Social Media? Testing Prosocial Interventions using Generative Social Simulation” from Maik Larooij, Petter Törnberg. An academic article that does seem to demonstrate an instance of Betteridge’s law of headlines ("Any headline that ends in a question mark can be answered by the word no."). Worth reading for what the researchers attempted to help improve social media behavior.

https://portswigger.net/research/http1-must-die - “HTTP/1.1 must die: the desync endgame” - James Kettle’s warning about the problems with HTTP/1.1 garnered a fair amount of attention during BlackHat and DEFCON festivities. albinowax, the head of research at portswigger, argues, “HTTP request smuggling must be recognized as a fundamental protocol flaw. The past six years have demonstrated that addressing individual implementation issues will never eliminate this threat.” 

https://zed.dev/blog/why-llms-cant-build-software - “Why LLMs Can't Really Build Software” by Conrad Irwin writing at ZED - This nugget from the CTO/Founder of Superhuman points out the benefits of LLMs as software engineers (fast development of code that can work for many applications), but he acknowledges a problem in relying on them that amplifies a theme of this week’s episode. Irwin writes, “For anything non-trivial, [LLMs] are not capable of maintaining enough context accurately enough to iterate to a working solution. You, the software engineer, are responsible for ensuring that the requirements are clear, and that the code actually does what it purports to do.”

Upcoming Events

Where in the world are Seth and Ken?

December 8-11, 2025 - Next-Gen Secure Code Review: Black Hat Edition - Seth and Ken are bringing a four-day exclusive course to Black Hat Europe in London, UK. This is a great opportunity to get a truly in-depth understanding of Secure-Code Review and how it can be empowered through LLM-tooling. Seth and Ken have innovated industry-leading trainings in both of these topics, so this four-day course promises to provide a lot of valuable insight.