RadSee: See Your Handwriting Through Walls Using FMCW Radar

RadSee: See Your Handwriting Through Walls Using FMCW Radar

Speakers: Shichen Zhang, PhD Student, Michigan State University

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=gWMSltzXlPA

Overview

In an era where digital interactions dominate, the seemingly anachronistic act of writing by hand might appear to be a bastion of privacy. However, the NDSS Symposium talk "RadSee: See Your Handwriting Through Walls Using FMCW Radar" by Shichen Zhang from Michigan State University shatters this perception, unveiling a groundbreaking system capable of discerning handwritten content from behind physical barriers. This research introduces RadSee, a sophisticated, custom-designed 6 GHz Frequency-Modulated Continuous Wave (FMCW) radar system paired with a tailored deep neural network, demonstrating the chilling feasibility of remote, through-wall surveillance of even subtle hand movements.

The implications of RadSee are profound, extending far beyond academic curiosity. It represents a significant leap in RF sensing capabilities, transforming what was once science fiction into a tangible privacy threat. The ability to "see" and interpret millimeter-level hand motions through walls opens new avenues for espionage, intellectual property theft, and personal data breaches, challenging conventional notions of physical security and privacy in enclosed spaces. This work underscores the urgent need for heightened awareness and innovative defensive strategies against increasingly sophisticated non-line-of-sight attack vectors.

Background

▶ Watch: Introduction: Seeing handwriting through walls (0:00)

The concept of "seeing through walls" has long captivated researchers in the RF sensing domain. Prior work, notably from institutions like the MIT CSAIL lab, has demonstrated the ability to detect human body skeletons and even track their poses behind walls using RF signals. These advancements laid foundational groundwork, proving that RF signals could indeed penetrate opaque barriers to extract information about human presence and gross movements. However, a critical gap remained: the resolution required to discern fine-grained, millimeter-level motions, such as those involved in handwriting.

Existing solutions for handwriting recognition largely fall into two categories: vision-based and millimeter-wave (mmWave) based systems. While both offer high accuracy, their fundamental limitation is a reliance on line-of-sight, rendering them ineffective when a physical barrier like a wall obstructs the view. On the other hand, through-wall solutions, such as those leveraging Wi-Fi signals, typically suffer from insufficient resolution. Wi-Fi signals, operating at lower frequencies, have longer wavelengths, making them inadequate for capturing the minute, sub-centimeter movements inherent in handwriting. This presented a significant challenge: developing a system that could both penetrate walls and provide the high spatial and temporal resolution necessary for handwriting recognition.

The RadSee attack model operates under several reasonable and achievable assumptions. The attacker is presumed to have physical access to the space behind the target wall (e.g., an adjacent room in a hotel, apartment, or office building). Knowledge of the room's layout and the approximate location of the victim's writing hand is also assumed, which can often be obtained from standard building plans or preliminary scans. Critically, the model assumes the absence of specialized shielding devices between the rooms, allowing RF signals to propagate. These assumptions highlight a realistic threat scenario, particularly in public or semi-public environments where adjacent room access and layout information are often obtainable.

Key Findings

▶ Watch: Why existing through-wall solutions fail (2:50)

The RadSee project successfully demonstrates the feasibility of accurately detecting and interpreting millimeter-level hand movements through walls to reconstruct handwritten content. The core innovation lies in its unique integration of a custom-designed 6 GHz FMCW radar system with a specialized deep learning model, overcoming the long-standing challenges of resolution and penetration in through-wall sensing.

One of the most significant findings is the system's overall letter recognition accuracy, which averaged approximately 75% across 12 diverse test participants. This substantial accuracy, achieved under realistic through-wall conditions, validates the system's capability to differentiate between various handwritten characters. The research also revealed nuances in performance, noting that cursive writing styles presented a greater challenge, resulting in lower accuracy compared to printed styles due to their increased diversity and complexity of motion patterns.

Furthermore, RadSee exhibited remarkable robustness to different writing media, maintaining similar accuracy whether participants wrote on an iPad or traditional Post-it notes. This indicates that the system primarily tracks the hand's motion rather than relying on specific interactions with the writing surface. The system also demonstrated strong resilience to interference, with its performance remaining unaffected as long as a distracting person was more than one meter away from the writer. This suggests practical applicability in environments with some ambient movement, provided the target motion is sufficiently isolated. These key findings collectively establish RadSee as a potent and practical tool for through-wall handwriting surveillance, highlighting both its technical prowess and the emerging privacy challenges it poses.

Technical Deep Dive

▶ Watch: RadSee system overview: Hardware and software (3:30)

The technical prowess of RadSee stems from a meticulous co-design of custom hardware and a sophisticated software pipeline, specifically engineered to address the unique challenges of through-wall millimeter-level motion detection.

At the heart of the system is its custom-designed 6 GHz FMCW radar. The choice of operating frequency at 6 GHz is a critical engineering decision. Lower frequencies, while offering superior wall penetration, lack the resolution needed for minute hand movements. Conversely, higher frequencies (e.g., millimeter-wave bands) provide excellent resolution but suffer from significant signal attenuation when passing through walls, rendering them impractical for through-wall applications. The 6 GHz band strikes an optimal balance, providing sufficient penetration while maintaining a wavelength of approximately 5 centimeters, which is ideal for observing hand motion patterns that are typically smaller than this wavelength. The FMCW modulation technique is employed for its ability to function as a distance filter. By measuring the frequency difference between the continuously transmitted and received chirped signals, the system can accurately determine the distance to targets and effectively separate them based on their range, isolating the target hand from other objects in the environment.

Complementing the radar board are custom-designed patch antennas. These antennas are crucial for generating a strong, focused signal. They boast an impressive 18 dBi antenna gain, ensuring that the transmitted signals are powerful enough to penetrate most common wall materials and that the reflected signals are strong enough for detection. Unlike conventional omnidirectional antennas, these patch antennas are designed to concentrate energy within a narrow angular range. This directional focus serves as an angular filter, significantly reducing interference from reflections originating from other directions, thereby enhancing the signal-to-noise ratio for the target hand. The total hardware cost for the RadSee system is remarkably low, approximately $500, making it an accessible technology. The PCB board integrates essential RF components, including a power supply, RF mixer, filter, low-noise amplifier (LNA), RF coupler, power amplifier, and a voltage-controlled oscillator.

The software component of RadSee is equally sophisticated, featuring a dedicated signal processing pipeline and a specialized deep neural network. The raw analog signals from the radar are first subjected to analog filtering, followed by digital signal processing. A key step involves extracting phase information from the processed signals. As the speaker demonstrated, even small, back-and-forth hand movements on paper produce distinctive, observable patterns in the signal's phase rotation. This phase sequence, which directly correlates with the hand's minute motions, is then segmented into individual letter units.

These segmented phase sequences are then fed into a deep neural network, specifically a Bi-directional Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) units. The choice of a Bi-directional LSTM is strategic; it allows the model to process the signal sequence in both forward and backward directions. This dual-direction processing is critical for accurately identifying turning points within the handwritten strokes, which are fundamental features for distinguishing between different letters. To further enhance the model's ability to focus on these crucial elements, an attention layer is incorporated, assigning higher weights to these significant turning points in the signal. Finally, the processed features are passed through a fully connected neural network for the ultimate letter recognition and classification. The system was trained on a dataset encompassing 62 categories (capital letters, lowercase letters, and numbers), collected from 18 participants, and evaluated on 12 distinct participants.

Demo / Proof of Concept

▶ Watch: Detecting millimeter-level hand motion using phase (6:00)

While the talk did not feature a live, real-time demonstration, the capabilities of RadSee were thoroughly validated through extensive empirical evaluation, serving as a robust proof of concept for its through-wall handwriting detection system. The evaluation methodology was comprehensive, designed to assess the system's accuracy under various conditions and participant demographics.

Data was meticulously collected from a diverse group of participants across different countries, encompassing a range of writing styles (both printed and cursive) and hand dominance (right or left-handed). The training dataset comprised data from 18 participants, while an independent set of 12 participants provided test data, ensuring unbiased evaluation. Participants were positioned behind a wall, writing on either traditional paper (e.g., Post-it notes) or digital devices like iPads, while the RadSee system captured their hand movements from the adjacent room.

The primary metric for evaluation was letter recognition accuracy. The system achieved an impressive average accuracy of approximately 75% across the 12 test participants. A deeper dive into individual performance revealed interesting insights: accuracy tended to be lower for cursive writing styles compared to printed letters. This was attributed to the greater variability and complexity inherent in cursive movements, making it more challenging for the model to extract consistent, common features for classification.

To assess the system's versatility, researchers evaluated its performance across different writing media. RadSee demonstrated similar accuracies whether participants were writing on an iPad or on physical Post-it notes. This finding confirms that the system is effectively tracking the subtle hand movements themselves, rather than being influenced by the specific interaction with the writing surface, broadening its potential applicability.

Finally, the resilience of RadSee to environmental interference was tested. In these scenarios, another person was instructed to walk around the writer at varying distances. The evaluation showed that the system's performance remained unaffected as long as the interfering person was positioned at a distance greater than one meter from the writer. This highlights the effectiveness of the system's directional antennas and FMCW-based distance filtering in mitigating ambient noise and focusing on the target hand, making it robust enough for practical deployment in certain dynamic environments.

Defensive Implications

▶ Watch: Bi-directional LSTM for handwriting recognition (7:40)

The emergence of technologies like RadSee presents significant challenges to privacy and security, necessitating a re-evaluation of defensive strategies. The ability to covertly "see" sensitive information being handwritten through walls transforms the traditional understanding of secure physical spaces.

Firstly, individuals and organizations must be acutely aware of this novel surveillance vector. Confidential meetings, sensitive document handling, or even personal journaling conducted in what were previously considered private rooms are now potentially vulnerable. The assumption that opaque walls provide inherent privacy against visual or motion-based surveillance is fundamentally challenged.

From a technical defense perspective, the most direct countermeasure against RF-based through-wall sensing is RF shielding. The RadSee attack model explicitly assumes "no shielding devices" between rooms. Implementing materials that attenuate or block 6 GHz RF signals, such as specialized paint, wallpapers, or building materials containing metallic mesh, could effectively disrupt RadSee's operation. However, comprehensive shielding can be costly and may interfere with legitimate wireless communications within the protected space.

Operationally, organizations should consider implementing stricter physical security protocols for rooms where sensitive information is handled. This might include designating "RF-secure" zones, conducting regular sweeps for unauthorized RF devices, or even altering standard room layouts to prevent an attacker from gaining easy access to an adjacent room. Personnel should also be educated about the risks, encouraging them to minimize handwriting sensitive information in vulnerable locations or to use alternative, more secure methods for recording data.

The speaker's mention of future work involving distributed antenna arrays for increased accuracy suggests that future iterations of such systems could become even more potent and harder to evade. As these technologies mature, passive defense mechanisms like physical distance or slight angular displacement (which the current system can tolerate up to 15 degrees) may become less effective. This necessitates a proactive approach to research and development of countermeasures, potentially involving active jamming or signal obfuscation techniques that introduce controlled noise into the RF environment specifically targeting the operating frequencies of such surveillance systems. Ultimately, RadSee underscores the continuous arms race between surveillance capabilities and privacy-enhancing technologies in the evolving landscape of cyber-physical security.

Key Takeaways

• Novel Through-Wall Surveillance: RadSee introduces a groundbreaking system capable of detecting and interpreting millimeter-level hand movements through walls to discern handwritten content, posing a new privacy threat.
• Integrated Hardware & Software: The system combines a custom-designed 6 GHz FMCW radar with high-gain patch antennas and a specialized Bi-directional LSTM deep neural network with an attention layer.
• Achieved Accuracy: RadSee demonstrated an average letter recognition accuracy of approximately 75% across diverse participants in through-wall scenarios.
• Robustness: The system maintains similar accuracy across different writing media (iPad, Post-it notes) and exhibits strong resilience to interference, as long as the distracting source is more than one meter away.
• Cost-Effective Design: The hardware components of the RadSee system have a total cost of around $500, making the technology relatively accessible.
• Significant Privacy Implications: This research highlights a critical vulnerability in physical security, emphasizing the need for increased awareness and advanced defensive strategies against sophisticated non-line-of-sight surveillance.

About the Speaker(s)

Shichen Zhang is a PhD student in the Computer Science and Engineering Department at Michigan State University. His research, as presented in the RadSee talk, focuses on innovative applications of RF sensing technologies, particularly in areas concerning through-wall detection and human-computer interaction.