How Computers Kill People: Marine Systems

How Computers Kill People: Marine Systems

Speakers: Michael DeVolld (Director of Maritime Cyber Security, ABS Consulting); Austin Reid (Senior Consultant, ABS)

Conference: DEF CON

YouTube: https://www.youtube.com/watch?v=s1-4KoND6wM

Overview

In an era dominated by discussions of nation-state hackers, ransomware, and AI-driven threats, Michael DeVolld and Austin Reid from ABS Consulting, joined by Chris Stein, delivered a sobering talk at DEF CON that reframed the most critical cyber risk facing the maritime industry: not external adversaries, but the insidious threat of poor software quality and engineering practices. Titled "How Computers Kill People: Marine Systems," the presentation starkly illustrates how flawed code, misunderstood design, or inadequate testing in increasingly automated and digitized maritime systems can lead to real-world physical harm, environmental disasters, and even loss of life. This isn't a futuristic dystopia but a present danger, a "Trojan horse" already embedded within the critical infrastructure that underpins global trade.

The core premise of the talk challenges the conventional perception of cyber threats, urging the audience to look beyond the sensational headlines and focus on the fundamental vulnerabilities introduced by the ubiquitous integration of software into operational technology (OT). DeVolld, with his background in maritime operations and incident investigation, highlighted how the industry's rapid adoption of digitalization for efficiency has inadvertently imported significant risk. The speakers argue that when code controls physical processes—from engine speed to navigation—mistakes don't just crash programs; they crash ships, cause blackouts, and can have catastrophic consequences that are often more immediate and tangible than data breaches.

This presentation serves as a critical wake-up call, emphasizing that any system running code, from a sophisticated navigation suite to a simple coffee machine on the bridge, is a computer capable of failure, compromise, or dangerous misbehavior if its software is poorly conceived or implemented. By dissecting historical incidents and showcasing a compelling real-world simulation, the speakers underscore the urgent need for a paradigm shift in how the maritime sector approaches software development, cybersecurity, and engineering resilience in an increasingly connected world.

Background

▶ Watch: Introduction: Software quality as the 'Trojan horse' (0:00)

The journey into understanding how computers can kill people in marine systems begins with a fundamental redefinition of what constitutes a "computer." For the layman, a computer might be a desktop PC, but in the context of critical infrastructure, it's anything with a processor, memory, and the ability to execute code. This expansive definition encompasses everything from a key fob and smart lights to a ship's navigation system, engine controls, ballast systems, and even the coffee machine on the bridge. If it runs code, it's a computer, and thus, it can fail, be hacked, or be programmed dangerously from the outset.

Historically, many critical systems relied on "dumb but safe" mechanical components. The speakers contrasted an old-school wax pellet thermostat—a purely mechanical device with no IP address, firmware, or attack surface—with a modern PLC-based control system. While the latter offers brilliant precision, remote control, and extensive logging capabilities, it simultaneously transforms a simple component into a cyber attack surface, often complete with default passwords like "admin" on its web interface. This transition from purely mechanical to networked systems represents not just an upgrade in capability but a significant import of risk.

This shift is emblematic of the Fourth Industrial Revolution, where the physical world is inextricably linked to code and networks through cyber-physical systems. In this environment, a change in logic, a bug in the code, or a compromised sensor doesn't merely cause a screen to flash red; it can make real-world equipment move, fail, or break in dangerous ways. The problem is often not elite hackers or exotic malware, but rather poor software quality, developers lacking understanding of process limits, control systems failing to protect themselves (or the user), and un-accounted hardware failures.

The consequences of such failures are not theoretical. The talk cited numerous high-profile incidents from various sectors:

• Boeing 787s grounded due to a power bug.
• An F-35 lost because of bad code.
• NASA's $327 million Mars Climate Orbiter destroyed due to a metric-imperial unit mix-up in software.
• Toyota's "killer firmware" incidents, where unintended acceleration was linked to software defects.
• The Chinook helicopter crash attributed to unsafe code.
• A particularly relevant maritime example: the USS Yorktown incident in the late 1990s. The Navy's ambitious digitization efforts, using Windows NT for shipboard systems, led to a catastrophic failure when a crew member set a valve set point to zero. A divide-by-zero error in the system's code crashed the entire ship's control system, leaving the vessel's systems offline for approximately two weeks. This incident powerfully demonstrated how a seemingly minor input, combined with flawed software, could paralyze a critical naval asset.

These examples collectively underscore a critical point: the system will execute exactly what it's told, even if that command is fundamentally wrong or dangerous. When code governs the physical world, mistakes are no longer confined to the digital realm; they manifest as real-world catastrophes.

Key Findings

▶ Watch: Redefining 'computer' for critical maritime systems (3:20)

The central finding of this talk is a stark re-prioritization of cyber risk in critical infrastructure, particularly in the maritime domain. The speakers emphatically argue that the greatest threat is not the external hacker, but the "Trojan horse" of poor software quality and engineering practices embedded within operational technology. This internal fragility, often overlooked in the pursuit of efficiency and advanced features, poses a more pervasive and immediate danger than sophisticated nation-state attacks.

A key revelation is the direct and often rapid translation of software flaws into physical harm. Unlike IT systems where a bug might lead to data loss or system downtime, in cyber-physical systems, a programming error or an unvalidated input can trigger mechanical failures, system shutdowns, and even loss of life. The talk demonstrated that the "initification" (a term borrowed from Cory Doctorow) of codebases—where complex, often fragile, and poorly understood software is layered onto legacy systems—creates inherent fragility that cannot simply be "uninstalled" or reverted like a consumer application. This makes critical systems vulnerable to cascading failures from seemingly minor triggers.

Furthermore, the speakers highlighted how the modern development culture, characterized by rushing products to market and a reliance on abstract tools (including emerging AI code generation), often results in developers lacking a deep understanding of the critical process limits and physical consequences of their code. This disconnect means that control systems are frequently designed without adequate self-protection mechanisms or robust input validation, leaving them susceptible to dangerous manipulation, whether accidental or malicious. The sheer volume of alarms triggered during a critical incident (600-800 alarms in the demo) further impedes effective human response, turning what should be an aid into an overwhelming barrier to diagnosis and recovery.

The regulatory environment, while evolving with standards like IACS E26, E27, and E22 (which apply IEC 62443 SL1 control bases to the maritime sector), is still playing catch-up. These regulations aim to enforce "secure by design" principles for new builds, but the legacy burden and the industry's inherent resistance to rapid change mean that many existing vessels and systems remain vulnerable. The confluence of regulatory lag, vendor "magic sauce" (proprietary, often opaque solutions), and the constant layering of emerging technology onto already complex systems creates a perilous environment where a single flawed variable or sensor reading can indeed take down an entire vessel.

Technical Deep Dive

▶ Watch: Dumb safe vs. smart hackable: thermostat example (4:20)

The technical core of the talk delved into the architecture and vulnerabilities of modern marine systems. At the highest level, a ship is a complex ecosystem of interconnected cyber-physical systems. Drawing from IACS Standard E22, the speakers illustrated how a vessel comprises various systems, subsystems, and programmable devices, each with embedded software modules. Key examples include PLCs (Programmable Logic Controllers) for automation, variable frequency drives (VFDs) for motor speed control, flow meters for critical fluid measurement, and engine telegraphs that dictate main engine RPM. Every one of these relies on software, and if that code is wrong, misunderstood, or compromised, the system will execute it precisely, even if the outcome is dangerous.

The evolution from mechanical to digital systems was a critical point. The shift from a simple wax pellet thermostat to a PLC-based control system for engine cooling exemplifies the introduction of a vast new attack surface. While the mechanical thermostat offered robust, air-gapped functionality ("heat in, valve out"), the modern digital equivalent incorporates sensors, logic controllers, motorized valves, and often a network interface—potentially with insecure default credentials. This transformation, while enabling precision and remote control, renders a previously "dumb but safe" component into a "smart but hackable" one, part of the Internet of Targets.

The presentation highlighted how this digitalization fuels the Fourth Industrial Revolution, where changes in code logic or compromised sensors directly impact physical equipment. This means a software bug is no longer an abstract error but a direct command to move, fail, or break physical machinery. The underlying technical issues often stem from:

1. Poor Software Quality: Inherent bugs, race conditions, or logic errors within the code itself.
2. Lack of Understanding of Process Limits: Developers may not fully grasp the physical constraints or potential catastrophic outcomes of certain software states in an operational technology (OT) environment.
3. Inadequate Control System Protection: Systems are often designed without robust input validation, authentication, or authorization mechanisms, making them vulnerable to both accidental and malicious manipulation.
4. Unaccounted Hardware Failures: Software may not be designed to gracefully handle or compensate for sensor malfunctions, actuator failures, or other hardware issues.

The USS Yorktown incident serves as a quintessential technical illustration. Its Windows NT-based shipboard control system, designed to automate and streamline operations, contained a critical flaw. When a crew member entered a zero as a set point for a valve, the system's underlying code executed a divide-by-zero operation. This fundamental arithmetic error, a classic programming bug, was not adequately handled, leading to an immediate crash of the entire control system. The cascading failure rendered the ship's critical systems inoperable for an extended period, demonstrating the profound impact of a basic software defect on complex physical machinery.

Austin Reid further introduced the concept of "vibe coding" and Cory Doctorow's "initification" to describe the modern development culture. This refers to the rushed development cycles, the layering of code by developers who might lack deep domain expertise in critical systems (e.g., a WinZip developer moving to propulsion control systems), and an over-reliance on emerging tools (including AI for code generation) without rigorous understanding or testing. This approach leads to codebases that are fundamentally fragile, difficult to debug, and inherently risky when deployed in life-critical contexts. Unlike an IT application that can be uninstalled, the fragility introduced into OT systems through "initification" is deeply embedded, posing persistent risks that are challenging to mitigate once deployed.

Demo / Proof of Concept

▶ Watch: Cyber-physical systems: code directly affects the physical world (6:00)

The most compelling segment of the talk was a simulation video, presented by Chris Stein, demonstrating a catastrophic failure scenario on a maritime vessel due to a software manipulation. The simulation was conducted on a Kongsberg simulator, a leading machinery automation system, specifically modeled after the Viking Grace, a RoPax (roll-on/roll-off passenger) ship that operates in the archipelagos between Stockholm and Turku, Finland. The Viking Grace typically runs four engines at 50% power to ensure spare capacity in case of a blackout, crucial for navigating treacherous, rock-strewn waters.

The demonstration focused on the ship's power management system and, more precisely, the freshwater cooling system for the diesel generators. These generators are cooled by seawater, which exchanges heat with a freshwater cooling system, which in turn cools the engine jacket. A critical component in this loop is a digitally controlled valve that diverts water into the heat exchanger, with a normal set point of around 70°C.

In the simulation, Stein manually manipulated this set point for the cooling valve. Instead of a realistic temperature, the set point was changed to 700°C. The objective was to illustrate what happens when a software failure (or malicious manipulation) causes this valve to effectively "stick" in a position that prevents adequate cooling, not because of a mechanical fault but due to software instruction. Crucially, because the software controlling this valve was identical across all four engines and part of one large automation system, the issue would propagate.

The results were swift and dramatic:

• The simulation began with the ship running full ahead.
• Within seconds, the first warnings appeared: high temperature on the outlet of the freshwater jacket, indicating the engine water was overheating.
• A "shutdown pre-warning" quickly followed.
• In rapid succession, all four engines began to trip. The first engine tripped, then the second, then the fourth, and finally the third, with breakers disengaging.
• The entire sequence, from the set point manipulation to a complete ship blackout and loss of propulsion, occurred in less than 90 seconds. This meant, as Stein grimly noted, the ship had technically "sailed into the rock."

A critical secondary outcome of this failure was the overwhelming alarm flood. Stein reported that approximately 600 to 800 alarms would pop up simultaneously on the ship's control systems. This deluge of alerts makes it nearly impossible for human operators to quickly diagnose the root cause, forcing them to scroll through hundreds of warnings to identify the specific manipulated set point amidst the chaos. He cited incidents like the Viking Sky where such alarm floods have severely impeded live investigations and contributed to catastrophic scenarios.

The demo's key takeaway was profound: it doesn't matter if the set point was manipulated by a hacker, an accidental user input, or a software bug. The physical consequence—engine overheating, blackout, loss of propulsion—remains the same. This illustrates the extreme fragility introduced by digital control when not coupled with robust engineering and safety limits.

Defensive Implications

▶ Watch: Headlines: Real-world examples of fatal software bugs (6:50)

The insights from "How Computers Kill People: Marine Systems" demand a significant re-evaluation of defensive strategies in the maritime sector and critical infrastructure at large. The primary implication is a shift in focus: defenders must prioritize software quality and robust engineering practices as a foundational cybersecurity measure, rather than solely concentrating on external adversarial threats.

Here are specific defensive implications:

1. Embrace "Secure by Design" Principles: The industry must move towards embedding security and resilience from the earliest stages of system design and development. This is precisely what emerging regulations like IACS E26, E27, and E22 aim to achieve, by applying the industrial cybersecurity framework IEC 62443 SL1 to the maritime sector. For new builds, this means mandating rigorous security requirements and testing throughout the lifecycle.
2. Implement Robust Input Validation and Error Handling: The USS Yorktown divide-by-zero incident and the Viking Grace simulation highlight the catastrophic potential of unvalidated user input or out-of-bounds parameters. All software, especially in critical control systems, must include comprehensive checks to prevent dangerous values (e.g., zero, excessively high temperatures like 700°C) from being processed, and robust error handling to prevent system crashes.
3. Thorough Software Testing and Validation: Beyond functional testing, software for OT systems requires extensive testing in realistic simulation environments that account for process limits, hardware failures, and various operational scenarios. This includes testing for cascading failures and the impact of identical software modules across redundant systems.
4. Architectural Resilience and Isolation: To prevent issues like the Viking Grace blackout, where a single software flaw propagated across all engines due to identical code and system integration, designs should incorporate greater isolation and redundancy. Critical systems should be designed to fail gracefully, with independent backups that are not susceptible to the same software flaws.
5. Intelligent Alarm Management: The problem of 600-800 alarms during an incident is a critical issue. Defensive strategies must include improvements in alarm philosophy, focusing on context-aware, prioritized, and actionable alerts that help operators quickly identify root causes rather than overwhelming them.
6. Developer Training and Domain Expertise: Software developers working on critical OT systems must receive specialized training that goes beyond traditional IT development. They need to understand the unique constraints, physical consequences, and safety implications of the processes their code controls. This helps bridge the gap between abstract code and real-world physical limits.
7. Scrutinize Vendor Practices: Given the "vendor magic sauce" and the "vibe coding" culture, organizations must rigorously vet their technology providers. This includes demanding transparency on software quality, security testing, and adherence to industry standards. Avoid blindly layering new, untested technologies onto legacy systems.
8. Resilience Against "Initification": Recognize that the inherent fragility of poorly designed or rapidly deployed software cannot be easily undone. Focus on building architectural resilience, implementing fail-safes, and developing robust incident response plans that account for software-induced physical failures, rather than just external attacks.

Ultimately, defending against "how computers kill people" requires a holistic approach that integrates cybersecurity with fundamental engineering principles, prioritizing safety, quality, and resilience above all else in the maritime domain.

Key Takeaways

• Software quality and poor engineering practices are a greater, often overlooked, threat in critical maritime infrastructure than traditional external cyberattacks. Bad code is a "Trojan horse" already inside, capable of causing real physical harm.
• Digitalization and automation introduce significant cyber-physical risks, where software flaws or manipulated inputs can directly lead to catastrophic physical outcomes like ship blackouts, loss of propulsion, and environmental disasters.
• Seemingly minor software flaws or unvalidated inputs can have rapid and devastating consequences. The demo showed a manipulated temperature set point causing a complete ship blackout in less than 90 seconds, highlighting the extreme fragility of interconnected systems.
• The "initification" of codebases, driven by rushed development and lack of domain expertise, creates inherent fragility in critical systems that is difficult to mitigate post-deployment and leads to cascading failures.
• Defenders must prioritize "secure by design" principles, robust input validation, and comprehensive software testing in operational technology, moving beyond a purely IT-centric cybersecurity mindset.
• Effective incident response requires intelligent alarm management, as overwhelming floods of 600-800 alarms during a crisis can severely impede diagnosis and recovery, as seen in past maritime casualties.

About the Speaker(s)

Michael DeVolld is the Director of Maritime Cyber Security at ABS Consulting. With a career spanning his entire professional life in the maritime industry, DeVolld started on the deck plates as a technician, working directly on ship systems and navigating vessels. He transitioned into inspecting commercial ships and has been involved in investigations of major maritime casualties, including the infamous Costa Concordia cruise ship sinking. Approximately 10 years ago, he moved into the cyber domain, now working at the critical nexus of marine operations, engineering, compliance, policy, and cyber risk. His extensive experience provides a deep understanding of the real-world implications of system failures.

Austin Reid is a Senior Consultant with ABS Consulting, bringing 10 years of experience in the maritime industry. His background includes working in cargo operations as a stevedore superintendent and later moving into automated container terminal operations. This diverse experience has given him insight into both traditional and bleeding-edge aspects of maritime logistics, fueling his passion for securing control systems. This was his fourth appearance at DEF CON, demonstrating his commitment to addressing these critical security challenges.

Chris Stein, acknowledged by Michael DeVolld as one of the most brilliant engineers he has known, also contributed to the talk. Stein has direct experience in developing and maintaining some of the very systems discussed during the presentation. He was responsible for producing the compelling simulation video of the Viking Grace incident, lending significant technical credibility and practical insight to the talk's core arguments.