Episode #313

This week on Absolute AppSec #313, Ken Johnson (@cktricky) and Seth Law (@sethlaw) explore the intensifying pressure on security practitioners as AI-driven development accelerates industry velocity. A prominent theme in this episode is the emergence of ‘shadow AI,’ where developers utilize unauthorized AI coding assistants and personal agents, introducing significant data classification risks and supply chain vulnerabilities. They also touch on 1Password’s SCAM (Security Comprehension Awareness Measure), highlighting broad performance gaps in an AI’s ability to detect phishing, and how to combat what they call “reasoning drift” and maintain safety. To find this episode, head over to https://www.youtube.com/@AbsoluteAppSec/streams, or find us wherever you get your podcasts.

Seth and Ken jump into a discussion of the emergence of Agentic AI, which uses tools to execute tasks on the user’s behalf, and the inherent risks it introduces. Ken is skeptical of deterministic, old-school-style tooling like Snyk or Veracode, arguing that these platforms are largely unequipped to find vulnerabilities in agentic applications. Seth echoes this sentiment, suggesting that the industry may need to entirely redo SDLCs with AI in mind, as the attack surface has changed more drastically than at any point since the advent of the web. They liken the current era to the early days of web frameworks like Rails or Flask, where the cognitive load required to understand new paradigms was immense, yet today the pace of change is even more relentless.

Recently, Seth had an experience testing for a government client with a strict No AI policy. The process of reverting to three-year-old testing methodologies is like “trying to swim through molasses,” illustrating how quickly AI has become a superpower for context-building and exploit discovery. This leads to a discussion on Shadow AI, where developers, hindered by corporate restrictions, may use personal AI assistants on their phones to analyze sensitive code—effectively leaking secrets through a channel that bypasses all traditional security controls. Ken notes that while some tools can detect AI-generated code during a push, there is currently no foolproof solution for preventing screenshot-based data exfiltration.

Finally, our hosts analyze a new security benchmark from 1Password called SCAM (Security Comprehension Awareness Measure), which evaluates how well AI models resist phishing and social engineering. The data reveals a wide disparity in safety; for instance, Claude 3.5 Sonnet performed significantly better than Gemini 1.5 Flash in avoiding the disclosure of sensitive data to malicious sites. However, the most critical insight is that skills—the specific markdown instructions that guide an LLM—can drastically improve safety scores and rectify gaps in model reasoning. Ken and Seth conclude that the future of the security professional will likely involve acting as a “Jarvis” to these AI entities, spending less time on manual testing and more time maintaining the complex guardrails, institutional knowledge, and skills files that keep autonomous agents from going off the rails.

Smatterings of snow storms pushing back Spring-time can’t really get you down. You can force the issue by getting ready for warmer weather in our merch store. Pick a tee-shirt, and get a size and color for yourself or friends:

If you’ve got thoughts about AI or if you’re tired of hanging with your botfriends at Moltbook, come hang out on our Slack. As they say, on the internet no one knows you’re a…. If there’s something you’d like to have Seth and Ken discuss, by all means float the idea there.

Stay Secure,

Seth & Ken

Related Episodes

https://www.youtube.com/watch?v=VreqmGPAK7I - Episode Ep. #189 - Security Bypasses, AppMap, Dastardly - At the beginning of this episode, Seth and Ken discuss a research article about how health care workers incorporated security bypasses as part of making sure they delivered care to patients. Security restrictions that get in the way of day-to-day work often meet bypass innovations, so the growth of Shadow AI shouldn’t be an unexpected development.

https://www.youtube.com/watch?v=1_L0DmOZf5I - Episode #290 - Authentication Fatigue, Browser AI Agents - Authentication Fatigue and the susceptibility of Browser Agents to phishing style attacks were both covered on this more recent episode. The 1PW SCAM results discussed in this week’s episode were presaged a bit here.

https://www.youtube.com/watch?v=8eWfjUsvLu8 - Ep. #220 w/ Erik Cabetas (Include Security) - The discussion here on the experience of running AppSec Consulting companies circa 2023 highlights a good deal about customer expectations, misunderstandings related to industry terminology and more. Worth a listen to see think through a baseline and to think through industry practitioners may be facing in this period of flux.

Absolute AppSec Happenings

https://www.thepragmaticcto.com/p/your-ai-pentester-found-1000-bugs – AI penetration testing tools like XBOW are rapidly surfacing high volumes of vulnerabilities, but they primarily find "surface-level" bugs through pattern matching. The article warns that while AI excels at breadth and speed, it currently lacks the human creativity and contextual reasoning needed to discover complex, high-impact business logic exploits.

https://blog.sshh.io/p/taste-is-not-a-moat – While the idea of AI being devoid of “taste” has been a mantra of comfort—Shrivu Shankar argues that it’s a fragile competitive advantage. While taste helps launch great products, it is easily replicated and doesn't scale. True moats require structural barriers like network effects, high switching costs, or proprietary data that persist beyond individual intuition.

https://unit42.paloaltonetworks.com/qr-codes-as-attack-vector/ — Unit 42 reports that attackers increasingly use QR codes to bypass security controls. Tactics include using URL shorteners to mask malicious sites, leveraging in-app deep links to hijack accounts or trigger payments, and bypassing app stores through direct malware downloads. Vigilance is critical as these codes exploit weaker mobile security.

Upcoming Events

Where in the world are Seth and Ken?

March 21-22, 2026 - BSidesSF - Watch this space for the details regarding a panel with Seth and Ken taking place in San Francisco in March.

April 7-8, 2026 - Harnessing LLMs for Application Security - IN PERSON at Kernel Con. Come join Seth and Ken in Omaha for the Harnessing LLMs course!

April 26-27, 2026 - Harnessing LLMs for Application Security - In-person training at DEF CON Singapore. Be sure to register now if you’re looking to enhance your day-to-day AppSec processes with the power of LLM agents.