January 18, 2025
Dodging Latent Space Detectors: Obfuscated Activation Attacks with Luke, Erik, and Scott.
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player AnalysisAbout this episode
In this episode of The Cognitive Revolution, Nathan explores the groundbreaking paper on obfuscated activations with 3 members from the research team - Luke Bailey, Eric Jenner, and Scott Emmons. The team discusses how their work challenges latent-based defenses in AI systems, demonstrating methods to bypass safety mechanisms while maintaining harmful behaviors. Join us for an in-depth technical conversation about AI safety, interpretability, and the ongoing challenge of creating robust defense systems. Do check out the "Obfuscated Activations Bypass LLM Latent-Space Defenses" paper here: https://obfuscated-activations.github.io/ Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Shopify: Dreaming of starting your own business? Shopify makes it easier than ever. With customizable templates, shoppable social media posts, and their new AI sidekick, Shopify Magic, you can focus on creating great products while delegating the rest. Manage everything from shipping to payments in one place. Start your journey with a $1/month trial at https://shopify.com/cognitive and turn your 2025 dreams into reality. Vanta: Vanta simplifies security and compliance for businesses of all sizes. Automate compliance across 35+ frameworks like SOC 2 and ISO 27001, streamline security workflows, and complete questionnaires up to 5x faster. Trusted by over 9,000 companies, Vanta helps you manage risk and prove security in real time. Get $1,000 off at https://vanta.com/revolution RECOMMENDED PODCAST: Check out Modern Relationships where Erik Torenberg interviews tech power couples and leading thinkers to explore how ambitious people actually make partnerships work. This season's guests include: Delian Asparouhov & Nadia Asparouhova, Kristen Berman & Phil Levin, Rob Henderson, and Liv Boeree & Igor Kurganov. Apple: https://podcasts.apple.com/us/podcast/id1786227593 Spotify: https://open.spotify.com/show/5hJzs0gDg6lRT6r10mdpVg YouTube: https://www.youtube.com/@ModernRelationshipsPod CHAPTERS: (00:00:00) Teaser (00:00:46) About the Episode (00:05:11) Latent Space Defenses (00:08:41) Sleeper Agents (00:15:06) Three Case Studies (Part 1) (00:17:02) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite (00:19:42) Three Case Studies (Part 2) (00:24:09) SQL Generation (00:26:17) Understanding Defenses (00:32:52) Out-of-Distribution Detection (Part 1) (00:35:37) Sponsors: Shopify | Vanta (00:38:52) Out-of-Distribution Detection (Part 2) (00:45:13) Loss Function Weighting (00:57:49) Who Moves Last? (01:11:41) High-Level Triggers (01:25:33) Open Source vs. Access (01:38:57) Internalizing Reasoning (01:53:07) Representing Concepts (02:06:38) Final Thoughts (02:09:33) Outro