Vibe School: Making dumb devices smart with AI

Vibe School: Making dumb devices smart with AI

Speakers: Dr Katie Paxton Fear (Security Advocate, Smrat)

Conference: DEF CON

YouTube: https://www.youtube.com/watch?v=CM_8gKlz2-o

Overview

Dr. Katie Paxton Fear, known online as Insider PhD and a Security Advocate at Smrat, took the DEF CON audience on a "deeply unserious" yet highly insightful journey into the practicalities and pitfalls of using large language models (LLMs) for hardware hacking. Titled "Vibe School: Making dumb devices smart with AI," her talk meticulously documented an attempt to build a smart weather station from scratch, relying almost exclusively on AI guidance. The core premise was to test whether an AI, specifically Google's Gemini, could serve as an effective "vibe coding" partner for a hardware novice, guiding them through complex electronic projects without requiring extensive prior knowledge or manual research.

The project began with a seemingly straightforward goal: integrate an inexpensive radio weather station with an ESP32 microcontroller and Home Assistant to create a smart, home-integrated weather monitoring system. This endeavor was motivated by the high cost of commercial smart weather stations and the speaker's burgeoning, albeit recent, confidence in hardware hacking following a successful prior project. However, the subsequent weeks revealed a stark contrast between the AI's theoretical capabilities and its practical application in the nuanced world of embedded systems, low-level protocols, and specific hardware configurations.

Dr. Paxton Fear's talk serves as a crucial case study for anyone considering leveraging AI for technical projects, particularly in domains requiring precise hardware interaction and debugging. It highlights the current limitations of LLMs when confronted with the intricacies of real-world electronics, the challenges of debugging AI-generated code, and the often-frustrating experience of relying on an AI that struggles with context, specific documentation, and subtle timing issues. Ultimately, the presentation underscored that while AI can offer high-level direction, the journey from concept to a functioning, reliable hardware device still demands significant human expertise, perseverance, and a willingness to confront the often-unforeseen complexities that AI currently cannot navigate autonomously.

Background

▶ Watch: Introduction to "Vibe School" and project goal (0:00)

The genesis of "Vibe School" stemmed from Dr. Katie Paxton Fear's recent foray into hardware hacking. Having successfully designed and programmed e-ink labels for her 3D printing filament, a project where AI assisted significantly with the user interface and API interactions, she felt a surge of confidence. This newfound prowess, however, was quickly contextualized by her partner's insightful, if slightly cynical, observation: "It's expensive to be bad at this, huh?" This comment, made after numerous AliExpress orders, became a recurring motif throughout the talk, foreshadowing the financial and temporal costs of an experimental approach to hardware development.

The specific project, a smart weather station, was born out of a common British preoccupation: the weather. Dissatisfied with the £135 price tag of commercial smart weather stations compatible with Home Assistant, Dr. Paxton Fear decided to build her own. Her plan was ambitious: acquire an inexpensive radio weather station, pair it with an ESP32C6 microcontroller, and enlist an AI to handle the heavy lifting. The strategy was distilled into three simple steps: define the goal (weather station), get AI to do it, and then, implicitly, profit (or at least save money).

For this ambitious undertaking, Dr. Paxton Fear selected Gemini as her AI companion. Her choice was based on several key features: its ability to use the internet as a source, theoretically reducing hallucination; its perceived proficiency in code generation; its "deep research" capabilities; and, pragmatically, its inclusion as a free perk with her smart doorbell subscription. To tailor Gemini's responses, she created a "Gem," a custom configuration fed with 10 files of foundational cybersecurity and hardware hacking literature. These included "From Day Zero to Zero Day" by Eugene Lim, "Practical IoT Hacking," "Hacking APIs" by Corey B., "The Hardware Hacker's Handbook," and various ESP programming guides. This curated knowledge base was intended to imbue Gemini with the necessary expertise, prompting it to consistently remind her, "As a cybersecurity expert..."

Strict rules were established for the experiment to ensure the integrity of the "AI-only" approach: no Googling (any external search was considered cheating), AI-generated code could be debugged by either human or AI, parts could only be purchased if Gemini explicitly recommended them, and libraries could only be included if Gemini specified them. This strict adherence to AI guidance, summarized as "no thoughts, just vibes," set the stage for a compelling, and often exasperating, exploration of AI's current capabilities in a domain demanding precision and deep technical understanding.

Key Findings

▶ Watch: The ambitious plan: Get AI to build a weather station (2:20)

The "Vibe School" experiment yielded several critical findings regarding the current state of AI's utility in complex hardware hacking and embedded systems development. Far from being a seamless "vibe coding" experience, the project highlighted significant limitations and unexpected challenges.

Firstly, AI, specifically Gemini, demonstrated a profound struggle with the granular, context-specific details of hardware interaction and configuration. While it could provide high-level conceptual guidance—such as correctly recommending radio frequency (RF) analysis over SPI line sniffing for data capture—its ability to translate this into functional, debugged code and configurations was severely lacking. This was most evident in its repeated failures to correctly generate YAML configurations for ESPHome, a task that consumed four to five hours of prompting and ultimately led to the discovery of hardware incompatibility.

Secondly, debugging AI-generated code proved to be a major hurdle. Gemini frequently "forgot" previous working solutions when new modifications were introduced, necessitating a complete regeneration of code from scratch. This lack of persistent context and self-correction made iterative development incredibly frustrating. The AI also struggled with subtle, real-world issues like timing and signal-to-noise ratios, often declaring problems "extremely hard to solve reliably on a microcontroller," a statement that the speaker ultimately proved wrong through accidental human intervention.

A surprising discovery was the unintended consequence of Serial.print statements. The AI's generated code only "worked" when debug print statements were active, as the act of printing slowed down the microcontroller's loop just enough to allow for successful packet capture. This indicated a fundamental gap in the AI's understanding of real-time embedded system behavior and the side effects of its own debugging tools.

Furthermore, the "deep research" feature of Gemini, intended to provide comprehensive insights, often proved inefficient and overly verbose, taking "so long" and often being less effective than a quick, targeted human Google search. This challenged the notion that AI can autonomously conduct efficient, relevant research for highly specialized technical problems.

Finally, the project underscored the critical importance of human expertise and intervention. Despite the strict "AI-only" rules, Dr. Paxton Fear repeatedly found herself correcting Gemini's errors, interpreting its responses, and ultimately diagnosing fundamental hardware compatibility issues (e.g., the ESP32C6's struggles with ESPHome) that the AI could not identify. The final, rudimentary hardware setup, while technically "working" to some degree, produced inaccurate data, reinforcing the initial warning: "It's expensive to be bad at this, huh?" The speaker eventually had to swap out the microcontroller for a different ESP32 model, a human decision based on existing parts, which finally allowed the ESPHome integration to proceed. The talk concluded that while AI excels at API-driven tasks, its current capabilities fall short when dealing with the nuanced, low-level world of embedded hardware.

Technical Deep Dive

▶ Watch: Customizing Gemini AI with books and strict rules (3:20)

The technical journey of "Vibe School" began with the aspiration to intercept and decode radio signals from a domestic weather station, then relay that data via an ESP32C6 microcontroller to a Home Assistant instance. This involved several distinct technical phases, each guided (or misguided) by Gemini.

The initial plan presented to Gemini involved sniffing SPI lines from the weather station's base unit. However, Gemini, acting as the "expert," correctly advised against this, instead recommending radio frequency (RF) analysis. This was a crucial piece of guidance, steering the project towards a more practical and less invasive approach for a novice. Gemini's "deep research" feature then produced a verbose report, which was subsequently condensed into a five-step action plan: capture data with an SDR dongle, set up a receiver, work with Visual Studio Code, integrate Zigbee, and connect to Home Assistant. A human correction was needed early on, as the ESP32 required configuration before a receiver could be added.

The first practical step involved capturing weather data. Gemini recommended using an SDR dongle (Software Defined Radio) and the RTL_433 software. An SDR dongle, such as the widely available RTL-SDR, allows a computer to receive a wide range of radio frequencies. RTL_433 is a generic data receiver for ISM band devices, designed to decode signals from various 433.92MHz devices like weather sensors, doorbells, and tire pressure monitors. Dr. Paxton Fear successfully used this setup to identify and capture the raw radio signals, and RTL_433 was able to decode some packets, providing initial validation that the radio signal analysis approach was viable.

The next significant hurdle was extracting and understanding the packet structure of the weather station's transmissions. Gemini suggested consulting GitHub for existing RTL_433 implementations, but this was deemed "cheating" under the strict rules. Instead, the raw decoded data and related source files were fed back to Gemini, tasking it with reverse-engineering the packet structure. This proved to be an "absolute nightmare," requiring "a lot of prompting" due to Gemini's inability to consistently decode the data, often failing with messages like "not enough data for the packet." Despite numerous attempts and direct human suggestions for debugging, Gemini frequently responded with "sass," asserting the human was "wrong." Eventually, after extensive iteration, Gemini produced a C-like sketch for the Arduino IDE that was able to decode the weather station's packets. Intriguingly, this "final decode algorithm" was essentially a direct port of the RTL_433 C code into an Arduino sketch, demonstrating that while Gemini could eventually reproduce existing solutions, it struggled with independent derivation.

For the microcontroller implementation, the initial attempt involved Visual Studio Code, which Gemini recommended. However, this environment proved problematic, leading Dr. Paxton Fear to revert to the more familiar Arduino IDE. Gemini then provided instructions for wiring the radio receiver to the ESP32 dev board. A comical moment arose when Gemini specified an antenna, which hadn't been purchased. The speaker improvised by stripping a piece of wire and bending it, demonstrating a pragmatic, if not ideal, solution.

A critical, yet accidental, technical discovery occurred during debugging: the code only functioned correctly when Serial.print statements were active. Gemini had struggled with "difficult timing and signal to noise issues," effectively giving up. However, the serial output inadvertently slowed down the ESP32's processing loop, providing enough time between packet captures for successful decoding. This highlighted a fundamental limitation of AI's understanding of real-time embedded systems: it lacked the intuitive grasp of how execution speed and I/O operations can impact timing-critical tasks.

The final phase involved integrating the weather data into Home Assistant. The original plan was to use Zigbee on the ESP32C6 with an existing Sky Connect dongle. However, Gemini "talked me out of it," recommending ESPHome instead. ESPHome is an open-source framework that allows users to create custom firmware for ESP-based microcontrollers, enabling seamless integration with Home Assistant via YAML configuration files. This transition proved to be the most frustrating technical challenge. Gemini exhibited extreme difficulty generating correct YAML, a declarative configuration language known for its strict syntax. This "took four or five hours of prompting," with Dr. Paxton Fear resorting to telling the AI, "can you read the documentation, please?" The struggle was attributed to a likely lack of specific ESPHome YAML training data, leading Gemini to "hallucinate" incorrect configurations. The ultimate roadblock was a hardware incompatibility: the ESP32C6 was not fully supported or compatible with the generated ESPHome configurations, a fact Gemini failed to identify. The solution required a human decision: swapping the ESP32C6 for another ESP32 model that Dr. Paxton Fear had "lying around," which finally allowed ESPHome to function. The finished hardware, though comprising only "eight wires," represented "several days" of painstaking effort, with the irony that the temperature readings were still "completely wrong."

Demo / Proof of Concept

▶ Watch: Shopping for parts based only on AI's recommendations (7:00)

The "Vibe School" presentation itself served as a comprehensive demonstration of the AI-driven hardware hacking process, showcasing the journey from initial concept to a flawed, yet functional, proof of concept. Rather than a polished, perfectly working device, the "demo" highlighted the trials, tribulations, and unexpected successes encountered when relying on AI for such a complex undertaking.

Key aspects of the demonstration included:

• Visuals of AI Interaction: Dr. Paxton Fear provided numerous screenshots and even a video clip of her chat interface with Gemini. These visuals vividly illustrated the extensive prompting required, Gemini's lengthy and often unhelpful "deep research" reports, its "sass" when debugging, and its repeated struggles, particularly with the ESPHome YAML configuration. The video of Gemini generating incorrect YAML over and over was a particularly compelling illustration of the frustration involved.
• Hardware Setup: A photograph of the final hardware configuration was displayed, featuring an ESP32 microcontroller connected to the radio receiver of the weather station with a mere "eight wires." This simple physical setup belied the days of effort required to reach that point.
• Data Capture and Decoding: Screenshots of tools like SDR# (or similar SDR software) showed the raw radio signals being captured, followed by examples of RTL_433 successfully decoding initial packets. This visually confirmed that the fundamental approach of RF analysis was sound.
• Serial Monitor Output: The presentation included a screenshot of the Arduino IDE's serial monitor, displaying the decoded weather data from the custom ESP32 firmware. While the data was being outputted, Dr. Paxton Fear pointed out the critical flaw: the temperature readings were wildly inaccurate and inconsistent between channels, despite the sensors being physically adjacent. This served as the ultimate "proof" that while the system was technically processing data, its practical utility was compromised.
• The "Working" Moment: The speaker described the moment when Gemini's code finally "worked" as a significant breakthrough, only to later reveal that this success was an unintended side effect of Serial.print slowing down the loop. This narrative arc was a crucial part of the "demo," illustrating the often-fragile nature of AI-generated solutions in embedded systems.

The core "proof of concept" was not a perfectly engineered weather station, but rather the feasibility of using an AI to guide a novice through complex hardware integration, even if the result was imperfect and the process arduous. It was a demonstration of AI's current limits, illustrating that while it can provide building blocks and general direction, the nuanced understanding and persistent debugging required for a truly robust system still demand significant human cognitive effort and problem-solving.

Defensive Implications

▶ Watch: First practical step: Capturing weather data with SDR dongle (7:20)

While "Vibe School" primarily focused on the challenges of using AI for hardware development, its narrative implicitly highlights several crucial defensive implications for IoT security and embedded systems. The struggles encountered by Dr. Paxton Fear offer valuable insights for both developers and security practitioners.

Firstly, the reliance on AI for generating code and configurations, especially for low-level hardware, introduces potential supply chain security risks. If AI recommends specific components or libraries without fully understanding their security posture or compatibility, developers might inadvertently incorporate vulnerable elements. Gemini's struggles with selecting a compatible ESP32 for ESPHome, for instance, could lead to developers using unsupported hardware that lacks critical security updates or features.

Secondly, the quality and reliability of AI-generated code are significant concerns. The "vibe coding" approach, where a human blindly trusts AI output, can lead to subtle, hard-to-diagnose bugs. The accidental dependence on Serial.print for correct timing illustrates how AI-generated code might function under specific, non-obvious conditions, potentially failing in production environments where debug output is removed. Such latent bugs could manifest as instability, unexpected behavior, or even introduce exploitable vulnerabilities. For instance, timing-sensitive operations in security protocols or data handling could be compromised by unoptimized or poorly understood AI-generated code, leading to race conditions or data corruption that attackers might exploit.

Thirdly, the challenges Gemini faced with YAML configurations for ESPHome underscore the importance of precise and secure configuration management. Misconfigurations are a leading cause of security breaches in IoT devices. If AI struggles to generate correct and secure configurations, human oversight and deep understanding of specific platform documentation (like ESPHome's) become paramount. Relying on AI for complex configuration without human validation could result in open ports, weak authentication, or unintended data exposure. Defenders must assume that AI-generated configurations will require rigorous auditing.

Fourthly, the talk emphasizes the necessity of understanding underlying protocols and hardware interactions. While Gemini correctly suggested RF analysis, dissecting packet structures and understanding data formats is a foundational skill. For defenders, this knowledge is vital for threat hunting, anomaly detection, and incident response. If a device is compromised, understanding its expected communication patterns and data structures, even at the radio frequency level, is critical for identifying malicious activity. Over-reliance on AI to abstract away these details could leave defenders ill-equipped to analyze sophisticated attacks.

Finally, the overall theme of "it's expensive to be bad at this" extends to security. Shortcuts taken due to lack of expertise or over-reliance on AI can lead to insecure products that are costly to fix post-deployment. The "vibe coding" mindset, while entertaining, highlights the danger of deploying systems without a thorough understanding of their inner workings, security implications, and potential failure modes. Defenders should advocate for robust testing, code review (even for AI-generated code), and a deep understanding of the technologies being deployed, rather than simply trusting an AI to deliver secure solutions.

Key Takeaways

• AI for Hardware Hacking is Nascent and Challenging: While AI can offer high-level guidance (e.g., suggesting RF analysis over SPI sniffing), it currently struggles significantly with the specific, low-level technicalities of hardware interaction, precise code generation for embedded systems, and complex configuration languages like YAML.
• Debugging AI-Generated Code is a Major Hurdle: AI often lacks persistent context, "forgets" working solutions, and struggles with subtle, real-world issues like timing. Debugging requires extensive human intervention, prompting, and sometimes accidental discoveries (like Serial.print slowing down a loop) to achieve even partial functionality.
• AI's "Deep Research" Can Be Inefficient: Gemini's "deep research" feature often produced lengthy, generalized reports that were less effective and more time-consuming than targeted human searches or consulting specific documentation for niche technical problems.
• Hardware Compatibility and Specific Configurations Remain Human Domains: AI struggled profoundly with generating correct ESPHome YAML and identifying hardware incompatibilities (e.g., ESP32C6 with ESPHome), indicating that detailed knowledge of specific hardware limitations and configuration syntax is still a critical human skill.
• "It's Expensive to Be Bad at This" Applies to AI-Assisted Projects: Despite the promise of AI, the project ultimately cost significant time and money, resulting in an unreliable and inaccurate device, reinforcing the speaker's partner's adage. The perceived efficiency of AI can be negated by the extensive debugging and troubleshooting required.
• AI Excels with APIs, Struggles with Low-Level Embedded Systems: The speaker's prior success with AI in web development and API-driven projects contrasted sharply with its struggles in low-level embedded hardware, suggesting that AI's utility is currently higher in structured, well-documented API environments than in the less forgiving, real-time world of microcontrollers.

About the Speaker(s)

Dr. Katie Paxton Fear, widely recognized by her online persona Insider PhD, is a prominent voice in the cybersecurity community. She serves as a Security Advocate at Smrat, a role she humorously describes as "going to companies and telling them, 'By the way, do you know that security is important?'" Her background includes a degree in computer science, and she is also a popular YouTuber, creating content that often blends technical insights with engaging, accessible explanations. While her professional experience leans towards cybersecurity and web development, Dr. Paxton Fear openly admits to being relatively new to the intricacies of hardware hacking. Her "Vibe School" talk at DEF CON exemplifies her adventurous spirit and willingness to explore complex technical challenges, even when they venture outside her primary areas of expertise, all while maintaining a refreshingly "deeply unserious" approach to learning and sharing.