Episode 326 - AppSec Jobs, Benchmarking LLMs, Open Web Standards

In episode 326 of Absolute AppSec, sponsored by mobile application security provider GuardSquare (guardsquare.com), the hosts start with a deep-dive into pre-show discussions about the shifting macroeconomic landscape of AppSec jobs. They analyze an industry-wide trend where corporate hiring is pivoting away from external third-party consultancies and contractors. Instead, maturing organizations are forming internal product security "tiger teams" and hiring dedicated security software engineers across general development lifecycles to handle the exponential volume of code generated by artificial intelligence. Turning to AI-driven engineering, they dissect a research paper tracking security vulnerability mitigations through large language model (LLM) feedback. The paper reveals a distinct degradation in code quality and an explosion of "false positives" or unreachable flaws after the fourth or fifth iteration due to compressed context windows and "context drift." Ken highlights his own grueling experience benchmarking AINative software. He heavily cautions that letting models self-score or automatically review code introduces dangerous biases, reinforcing the absolute baseline requirement for humans to critically audit all LLM outputs. Finally, they examine Open Web Docs' new web security guidelines community group, comparing its browser-centric standard party focus to OWASP's broader, audit-driven charter. They close by promoting an upcoming July podcast collaboration with Coffee, Chaos, and ProdSec.