A World Where We Trust Hard-Won Lessons in Security Research, Technology, and People

A World Where We Trust Hard-Won Lessons in Security Research, Technology, and People

Speakers: David Brumley

Conference: USENIX Security '24

YouTube: https://www.youtube.com/watch?v=svmuTsEZ6hQ

Overview

In this compelling keynote address at USENIX Security '24, David Brumley, a distinguished professor at Carnegie Mellon University and founder of ForAllSecure (creators of Mayhem), delivered a deeply personal and insightful talk on the challenges and realities of translating cutting-edge security research into practical, trustworthy software. Brumley, drawing from two decades of academic work and his journey into commercialization, highlighted the often-overlooked human and organizational factors that dictate the adoption and impact of security technologies. His central thesis revolved around the idea that creating truly trustworthy software is an achievable goal, but one that requires understanding and addressing the motivations, processes, and diverse needs of real-world developers and organizations.

The talk diverged from typical research presentations by focusing not on novel technical advancements, but on the "non-obvious problems" encountered when bringing robust security tools to market. Brumley shared four poignant stories from his experience, illustrating how academic assumptions frequently clash with industry realities. He emphasized that for security to be effective, it must be integrated seamlessly into existing workflows and, critically, align with the immediate goals and incentives of the engineers tasked with building software. This shift in perspective, from pushing technology to solving user problems, forms the bedrock of his vision for a more secure software ecosystem.

Brumley's address serves as a vital bridge between the theoretical rigor of security research and the practical demands of its application. It offers a critical reflection for both academics and industry professionals on how to foster greater trust in software by first building trust with the people who create and depend on it. By sharing his hard-won lessons, Brumley not only provided a roadmap for successful technology transfer but also an inspiring message of perseverance in the face of significant personal and professional hurdles.

Background

▶ Watch: Introduction: Building a world of trustworthy software (0:00)

The journey to building trustworthy software is fraught with challenges, a reality David Brumley and his team at ForAllSecure learned firsthand. Their work on Mayhem, an automated security analysis platform, stemmed from two decades of academic research, primarily in symbolic execution and fuzzing. This foundational research was rigorously tested and validated in high-stakes environments, most notably the DARPA Cyber Grand Challenge, where automated systems competed to find and fix vulnerabilities in real-time. The success in such a demanding environment instilled a strong conviction that their technology represented the "latest and greatest" in practical security research.

However, the transition from academic validation to widespread market adoption revealed a chasm between theoretical efficacy and practical integration. Brumley referenced a seminal paper by the Stanford research team behind City, an early and highly successful commercial static analysis engine. This paper candidly noted, "the problems we encountered were not the obvious ones," a sentiment that deeply resonated with Brumley's own experiences. The academic pursuit often prioritizes technical soundness, completeness, and novel algorithms, sometimes overlooking the nuanced operational realities of development teams.

The problem, as Brumley elucidated, is multi-faceted. Developers, particularly in large enterprises or safety-critical domains, operate within established frameworks, use specific toolchains, and are driven by metrics that may not directly align with a security researcher's definition of "success." For instance, asking an aerospace engineer to containerize their PowerPC/VXWorks application in Docker—a concept entirely foreign to their environment—before demonstrating any value, immediately creates a barrier. This highlights a fundamental disconnect: the security community's focus on vulnerability detection often fails to consider the user's primary goal, which might be achieving specific code coverage for a safety standard, meeting a deadline, or simply getting their job done efficiently. The talk aimed to dissect these "non-obvious problems" and offer insights into bridging this critical gap.

Key Findings

▶ Watch: Influential paper and the challenge of non-obvious problems (2:15)

David Brumley's keynote distilled his experiences into four core "stories," each revealing crucial lessons for bridging the gap between security research and real-world adoption:

1. Understanding Their Wins: The most significant finding was that security solutions gain traction when they align with the user's existing goals, rather than imposing new ones. In the aerospace example, the engineering team was motivated by achieving high assembly code coverage for safety standards, a task that was budgeted at $10 million for manual effort. Mayhem, through its automated fuzzing and symbolic execution, generated an 80% assembly coverage test suite "for free" in a day. By reframing vulnerability detection as a means to achieve a critical safety metric (coverage), Brumley transformed a skeptical user into an adopter. This demonstrated that "untested code is risky code" can be translated into tangible benefits beyond just finding "bugs."

1. Building Community: Cultivating a pipeline of security talent is essential for creating trustworthy software. Brumley's experience with PicoCTF highlighted the power of gamified learning and the "law of large numbers" in talent identification. The key to PicoCTF's massive scale (over 400,000 players) wasn't just building a great game, but finding "communicators"—specifically, high school teachers. By offering free, zero-install curriculum and a way to record class participation, PicoCTF solved a problem for teachers (providing engaging content with minimal effort) which, in turn, opened a funnel to thousands of students interested in cybersecurity. This approach proved far more effective than traditional outreach, demonstrating that solving a connector's problem can unlock access to a vast audience.

1. Segments Matter: Brumley challenged the common security narrative that "70% of vulnerabilities are memory safety issues" in C/C++. He presented data from SlashData showing that 87% of developers don't primarily use C/C++. This revealed a critical missegmentation in the security community's focus. The world is primarily building software in JavaScript, Python, and other languages, often for web, API, and cloud services. These segments have different security concerns (e.g., SQL injection, XSS for web vs. memory safety for embedded systems) and different perceptions of attack surface (public-facing APIs are higher priority than a single car's infotainment system). Effective security solutions must be tailored to these diverse language choices, project types, and attack surface priorities. Furthermore, the industry prioritizes known vulnerabilities (addressed by SBOM/SCA) before investing heavily in finding novel vulnerabilities.

1. Keep Moving: Beyond the technical and strategic lessons, Brumley shared a deeply personal finding: the importance of perseverance. Drawing from his own journey—dropping out of high school, being rejected from a Stanford PhD program despite winning a best paper award, and battling cancer—he underscored that setbacks and roadblocks are inevitable. The ability to "keep falling down but keep getting up" is crucial for long-term impact in research, entrepreneurship, and life. This meta-lesson emphasizes resilience as a fundamental component for any ambitious endeavor, including the grand challenge of building a world of trustworthy software.

Technical Deep Dive

▶ Watch: First story: Collaborating with an aerospace company (4:10)

Brumley's talk, while focused on adoption, provided significant technical context for both his company's solution, Mayhem, and the broader landscape of software security.

Mayhem's Core Analysis Engine

Mayhem's foundation is built upon two decades of academic research, primarily in program analysis, specifically symbolic execution and fuzzing. Brumley highlighted that while he initially championed symbolic execution, practical experience demonstrated that pairing it with fuzzing yields the best results, citing the Driller paper from ASU as validation. This combined approach is central to Mayhem's effectiveness:

• Symbolic Execution: This technique works by analyzing program paths, modeling path constraints as mathematical formulas, and then using a solver to generate inputs that force the program down specific, previously untraversed paths. This is powerful for exploring deep logic branches.
• Fuzzing: This involves feeding semi-random or mutated inputs to a program to trigger crashes or unexpected behavior. Modern fuzzers often use heuristics (like coverage guidance) to maximize the exploration of new code paths.
• Iterative Process: Mayhem continuously runs these analyses, identifying untested code paths, generating new test inputs, and iterating to find defects.

Beyond just finding an input that triggers an issue (Proof of Vulnerability or PoV), Mayhem includes a critical diagnosis step. This is crucial for real-world adoption because simply reporting a crash isn't enough. Users need to know:

• Location: Where in the code the defect occurs.
• Type: What kind of bug it is (e.g., out-of-bounds read, out-of-bounds write). Brumley noted that an out-of-bounds read might be a "medium" severity, while an out-of-bounds write is typically "high."
• Severity: A qualitative assessment of impact.
• Mitigation Analysis: Mayhem checks if common defenses like ASLR (Address Space Layout Randomization) and DEP (Data Execution Prevention) are compiled into the binary. In some safety-critical systems, the presence of these mitigations can make fielding a program with a known defect acceptable, as exploitation becomes significantly harder.

A key output of this process, often overlooked by security researchers focused solely on CVEs, is an expanded test suite. By systematically exploring code paths, Mayhem generates a rich set of test cases that increase code coverage. This output proved invaluable to Brumley's aerospace customer, as it directly contributed to their safety standards.

Embedded Systems Development and the V-Model

Brumley offered an insightful look into the often-opaque world of safety-critical embedded systems development, which follows a rigorous V-model (a variation of the waterfall model):

1. Requirements: Detailed engineering specifications, often traced back to customer needs and documented in Word.
2. Model-Based Design: Engineers build models (e.g., using Simulink for speed control in a car) that represent the system's behavior. These models are then model-checked against the requirements.
3. Code Generation: A button click in the modeling tool automatically generates source code (often C). Brumley described this generated code as "evil vile thing" – full of global variables and deeply nested conditionals – but implicitly trusted because it's machine-generated from a verified model. This presents a potential insider attack vector if the code generation tools themselves were compromised (e.g., MathWorks, Visual Studio).
4. Hardware-in-the-Loop (HIL) Testing: The generated code is compiled for specific architectures (e.g., PowerPC running VXWorks) and tested on actual hardware. This is critical because runtime behaviors, like divide-by-zero errors, can vary significantly across architectures (e.g., x86 throws an exception, ARM does not).
5. Real-World Testing: The final product is tested in its operational environment.

Brumley noted that targeting the few code generation tools used in this industry to produce safer languages (e.g., Rust) could have an enormous impact, as developers are essentially "programming in this UI, not at the code level."

Known vs. Unknown Vulnerabilities: SBOM and SCA

A pivotal technical distinction Brumley made was between finding previously known vulnerabilities and novel vulnerabilities. He argued that industry's first priority is almost always the former. This is addressed by:

• Software Bill of Materials (SBOM): A simple list of all open-source and third-party dependencies used in a software project (e.g., requirements.txt for Python, package-lock.json for JavaScript).
• Software Component Analysis (SCA): Tools that take an SBOM and identify which specific versions of those dependencies have known vulnerabilities (e.g., Fast API version 0.11.0 has a known CVE).

Brumley highlighted Snyk as a $7.4 billion company built on this premise, demonstrating the immense market value in accurately detecting known vulnerabilities. Mayhem, recognizing this market need, integrated an SBOM tool. A key technical challenge they addressed was reducing "useless but accurate results"—false positives in the sense of actionable insights. For example, a Docker image might contain a package manager like APT with a CVE-10.0 vulnerability. While technically present, if APT isn't on the public attack surface, it's not an immediate concern for the customer. Mayhem's solution involves analyzing the image to determine what components are actually exposed on the attack surface, thereby prioritizing truly exploitable known vulnerabilities.

Attack Surface and Segments

The concept of attack surface profoundly influences how companies prioritize security. For web and API applications (often Python/JavaScript), the attack surface is vast and public, making vulnerabilities like SQL injection and XSS critical. For embedded systems, while a car compromise is severe, it affects one unit, whereas a backend cloud API compromise affects all users. This segmentation dictates tool choice and security investment. Brumley also noted that stateful attacks (e.g., putting an item in a database, removing it, then performing XSS) are particularly challenging for traditional fuzzing.

Ultimately, the technical deep dive revealed that sophisticated program analysis techniques like symbolic execution and fuzzing are powerful, but their real-world impact is amplified when integrated into existing development paradigms, tailored to specific language ecosystems and project types, and, crucially, aligned with the operational goals and attack surface concerns of the users.

Demo / Proof of Concept

▶ Watch: Conflict: Modern tools vs. legacy embedded systems (5:20)

As a keynote address, this talk did not feature a live technical demonstration or proof of concept in the traditional sense. Instead, David Brumley shared compelling real-world "proofs of concept" through the successful application of his technologies and methodologies, illustrating the tangible impact of his approach.

One significant example came from the first story, involving a large aerospace company. Brumley recounted how Mayhem, his company's automated security analysis tool, was deployed to analyze critical components of aerospace software. Despite initial resistance from the development team regarding integration challenges (like containerizing PowerPC/VXWorks code), Mayhem successfully identified "defects" that indicated potential cyberattack vectors in critical systems. More importantly, Mayley provided an invaluable side-effect: it generated a comprehensive test suite that achieved 80% assembly code coverage in just one day. This was a critical win for the customer, who had budgeted $10 million and thousands of hours for manual analysis to reach a similar coverage level for their safety standards. This demonstrated that security tools can deliver immense value by aligning with existing, non-security-specific goals.

Another powerful "proof of concept" was the PicoCTF platform. This gamified cybersecurity competition, built on the principle of a broad "funnel" for talent acquisition, has engaged over 400,000 active players to date. The success of PicoCTF is evident in the caliber of talent it has produced. Brumley proudly showcased a picture of his former students, many of whom are now "black badge" winners at Defcon CTF (earning lifetime admission to the conference). This group includes the head of Google Threat Intelligence for China, a Microsoft Research scientist, a Pwn2Own browser hacker (who earned $60,000 in bug bounties by exploiting Safari and Chrome), the individual behind the first Tesla hack, a UC Berkeley HTI Professor, and key figures within the DOD and the new Cyber Challenge. This clearly demonstrates PicoCTF's ability to identify and cultivate world-class cybersecurity talent from a broad base.

Finally, Brumley detailed a community-driven initiative where ForAllSecure offered a $100 bounty to college students for taking an open-source project, containerizing it (using Docker), and setting up a CI/CD pipeline to run Mayhem and other security analyses. This program successfully resulted in 2,000 open-source packages being dockerized and instrumented. This initiative not only made these projects more accessible for security analysis but also created a valuable dataset for researchers, proving that small incentives can yield significant community resources and solve a practical problem of "dockerizing" code. These examples, though not live demos, served as robust evidence for the talk's central arguments.

Defensive Implications

▶ Watch: Understanding the V-model for embedded systems development (7:15)

David Brumley's keynote offers several critical implications for cybersecurity defenders, shifting the focus from purely technical solutions to a more holistic, user-centric approach:

• Align Security with Developer Goals: Defenders must understand the primary motivations and metrics of developers. Instead of simply demanding "secure code" or pointing out vulnerabilities, security teams should articulate how their recommendations and tools help developers achieve their existing goals, such as meeting code coverage requirements for safety certifications, improving code quality, or accelerating release cycles. This transforms security from a roadblock into an enabler.
• Embrace a Multi-faceted Approach to Analysis: The "static vs. dynamic" debate is too narrow. Defenders need both. For known vulnerabilities, robust SBOM and SCA tools are non-negotiable and should be the first line of defense. For novel vulnerabilities, advanced techniques like combined symbolic execution and fuzzing (as in Mayhem) are essential. Furthermore, the analysis must go beyond mere detection to provide actionable diagnoses, including bug type, severity, and mitigation status (e.g., presence of ASLR/DEP).
• Prioritize Based on Attack Surface and Impact: Not all vulnerabilities are created equal. Defenders should focus resources on the most exposed components, such as public-facing web applications and APIs, which represent the highest risk attack surface. While critical embedded systems are important, the internet-facing backend infrastructure often presents a broader and more immediate threat.
• Segment Security Strategies by Technology Stack: A "one-size-fits-all" security strategy is ineffective. Python and JavaScript developers building web APIs have different concerns (e.g., SQL injection, XSS, business logic flaws) than C/C++ developers working on embedded systems (e.g., memory safety, race conditions). Security tools and guidance must be tailored to these specific language ecosystems, project types, and their associated threat models.
• Invest in Talent Development and Community Building: The future of trustworthy software relies on a skilled workforce. Initiatives like PicoCTF are vital for identifying and nurturing cybersecurity talent from a broad base. Defenders should support or initiate programs that make cybersecurity education accessible and engaging, focusing on finding "communicators" (like teachers) to amplify their reach.
• Adopt a Pragmatic Language for Findings: In sensitive industries, referring to discoveries as "defects" rather than "vulnerabilities" or "exploits" can prevent immediate legal or recall processes. This allows for a more constructive and less combative dialogue with engineering and management, enabling issues to be addressed within existing quality assurance frameworks.
• Recognize the Distinction Between Safety and Security: Especially in safety-critical domains, defenders must understand that safety standards (focused on random failures) are distinct from security standards (focused on malicious attackers). While overlapping, the presence of an intelligent adversary fundamentally changes the problem. Security efforts should explicitly address attacker models.
• Leverage Automated Coverage for Dual Benefits: Tools that automatically generate test cases and boost code coverage (a byproduct of many advanced security analyses) offer a dual benefit: they improve software quality and reliability (a safety goal) while simultaneously increasing the likelihood of finding security defects. Defenders should highlight this synergy to gain buy-in from engineering teams.

Key Takeaways

• Tie Security to Their Goals: For security tools and practices to be adopted, they must address the immediate, measurable objectives of developers and organizations, rather than imposing new security-centric mandates.
• Find and Empower Communicators: To scale security education and adoption, identify individuals or groups (like high school teachers) who can act as conduits to broader audiences, solving their problems to reach your target users.
• Segment Your Approach by Use Case: Recognize that different programming languages, project types (e.g., web API vs. embedded), and attack surfaces demand distinct security tools, strategies, and priorities. The world is not just C/C++.
• Prioritize Known Vulnerabilities First: Companies will always address easily identifiable, known vulnerabilities (via SBOM/SCA) before investing in finding novel, unknown ones. This should be the foundational layer of any security strategy.
• Perseverance is Paramount: The journey to building trustworthy software and successful security initiatives is filled with roadblocks and setbacks, both technical and personal. Continuous effort and resilience are essential for long-term impact.
• **Focus on Solving Their Problems**: The overarching lesson is to shift the mindset from "how can my technology solve security problems?" to "what problems do my users have, and how can security help them solve those?"

About the Speaker(s)

David Brumley is a distinguished figure in the cybersecurity community, known for his contributions as both an academic researcher and an entrepreneur. He is a Professor at Carnegie Mellon University (CMU), where he has led significant research in program analysis, particularly in symbolic execution and fuzzing.

Brumley is the founder of ForAllSecure, the company behind Mayhem, an automated security analysis platform that emerged from two decades of his academic work. His team famously won the DARPA Cyber Grand Challenge, a competition designed to test the efficacy of automated vulnerability discovery and patching systems at scale.

Beyond his technical achievements, Brumley is a passionate advocate for cybersecurity education and community building. He is the driving force behind PicoCTF, a highly successful gamified Capture the Flag (CTF) competition that has introduced hundreds of thousands of students to cybersecurity. His personal journey, marked by dropping out of high school, being rejected from a Stanford PhD program (despite winning a best paper award), and overcoming a cancer diagnosis, underscores his message of perseverance and dedication to his passion. He credits his advisor, Don Song, with profound guidance throughout his career.