Abusability of Automation Apps in Intimate Partner Violence

Abusability of Automation Apps in Intimate Partner Violence

Speakers: Shirley Zhang

Conference: USENIX Security

YouTube: https://www.youtube.com/watch?v=UkmheMvupRA

Overview

This talk, presented by Shirley Zhang at USENIX Security, unveils a critical and often overlooked vector for intimate partner violence (IPV): the weaponization of readily available mobile automation applications. While much research in tech-enabled abuse focuses on overt spyware, this work highlights how powerful, built-in, or easily downloadable apps like iOS Shortcuts, Android Tasker, IFTTT, and Samsung Bixby Routines can be silently reconfigured by abusers to conduct surveillance, impersonation, overloading, and local control attacks against victims. The presentation details the extensive capabilities of these apps, the specific attack patterns they enable, and introduces a novel detection pipeline utilizing large language models (LLMs) to identify malicious automation "recipes" within public repositories.

The significance of this research lies in exposing a pervasive threat vector that leverages legitimate, high-privilege system functionalities for malicious purposes. Unlike traditional malware, these automation apps are often pre-installed or easily accessible, making them less suspicious and harder for victims to detect. The study's focus on an IPV threat model, where an abuser might gain temporary physical access to a victim's device, underscores the practical relevance of these findings for domestic abuse scenarios.

Ultimately, this work serves as a crucial call to action for both app developers and the security community. It not only quantifies the prevalence of potentially abusive automation recipes but also proposes concrete mitigations, from enhanced system-level warnings to per-recipe runtime permissions, aiming to protect vulnerable individuals from digital exploitation.

Background

▶ Watch: Introduction: Automation apps weaponized in IPV (0:00)

The landscape of intimate partner violence (IPV) has increasingly intertwined with technological advancements. Abusers now frequently leverage modern technology, from overt spyware to seemingly innocuous communication channels, to control, harass, and monitor their victims. While existing research has extensively documented the use of commercial spyware, stalkerware, and social engineering tactics, a significant blind spot has remained: the potential for automation applications to be weaponized for abuse.

Automation apps are designed to empower users by allowing them to define custom triggers and actions, creating automated sequences known as "recipes." Examples include setting a reminder on a smart speaker at a specific time or turning off lights when leaving home. However, the powerful access these apps have to device functionalities and sensitive data presents a severe risk when co-opted for malicious intent. Previous discussions, even years prior, have hinted at the potential for tools like Tasker to be used for spying, and more recent online forums show users actively creating and sharing spyware using platforms like iOS Shortcuts. This talk formalizes and deeply investigates this emerging threat.

The research operates under an IPV threat model, which assumes that an attacker (abuser) possesses the capability to at least once unlock the victim's mobile device. This initial physical access allows the abuser to install or configure malicious automation recipes. Furthermore, the model acknowledges that abusers can communicate with victims through various channels, such as phone calls and text messages, which can serve as external triggers for these automation sequences. Crucially, this work focuses exclusively on mobile automation apps, deliberately excluding IoT devices, to narrow the scope and provide a focused analysis of smartphone-based abuse. The core problem lies in the high privileges and extensive device control these apps are granted, often without sufficient user awareness or granular control over specific recipe actions.

Key Findings

▶ Watch: Analyzing automation app capabilities: triggers & actions (3:20)

The research yielded several critical findings that underscore the hidden dangers of mobile automation apps in the context of IPV:

1. Extensive Capabilities of Automation Apps: The study found that popular automation apps like iOS Shortcuts and Android Tasker possess exceptionally high privileges, including Turing-complete logic, allowing them to perform complex sequences of actions. They can collect and exfiltrate sensitive information, manipulate device hardware (e.g., Wi-Fi, flashlight), and control software functionalities.
2. Four Classes of IPV Attacks Enabled: The identified capabilities can be assembled into four distinct categories of IPV attacks:

• Surveillance: Covertly gathering sensitive information from the victim's device.
• Impersonation: Sending messages or performing actions on behalf of the victim without their knowledge.
• Overloading: Repeatedly sending messages or adjusting device functionalities to harass or disrupt.
• Local Control: Manipulating device settings or content to gaslight or psychologically abuse the victim.

1. Widespread Presence of Exploitable Recipes: A comprehensive analysis of over 12,000 public iOS Shortcut recipes across four domains revealed that 1,040 (approximately 8.7%) were potentially exploitable for IPV attacks. The "Local Control" attack type represented the highest confirmed threat among these.
2. Novel LLM-Based Detection System: The research successfully designed and implemented a detection pipeline for malicious automation recipes. This system leverages large language models (LLMs) to analyze the textual representation of shortcuts, identify "attack operators" (e.g., data collection, exfiltration), and reason about potential abusive patterns, demonstrating high efficacy in identifying risky recipes.
3. Inadequate Developer Response and Mitigation Gaps: Initial outreach to app developers like Apple and Tasker revealed a general lack of mechanisms or responsibility for preventing such misuse. This highlights a significant gap in current security models, where the power of automation is recognized but its potential for abuse is largely unaddressed.

These findings collectively illuminate a critical, under-researched area of tech-enabled abuse and provide both a detailed understanding of the threat and a practical approach to its detection.

Technical Deep Dive

▶ Watch: Four types of IPV attacks enabled by apps (5:00)

The technical foundation of this research begins with a comprehensive analysis of the capabilities inherent in popular mobile automation applications. The study focused on four prominent platforms: iOS Shortcuts, Android Tasker, IFTTT (If This Then That), and Samsung Bixby Routines. Through a detailed UI step-through and interaction with these applications, the researchers cataloged an extensive set of triggers and actions available to users.

Triggers were categorized into four main types:

• Internal Event: Device-centric occurrences such as connecting to a charger, changes in location, or specific app launches.
• External Event: Interactions originating outside the device, primarily receiving text messages or phone calls from other entities.
• Preset Time: Schedule-based activations, like a specific time of day or day of the week.
• User Actions: Direct user interactions such as taking a photo or starting a recording.

Apps like iOS Shortcuts and Android Tasker demonstrated the broadest support across all trigger types, while IFTTT and Bixby Routines offered a more limited selection, particularly in user action triggers.

Actions were found to be even more diverse and powerful, especially in Tasker and Shortcuts, which support Turing-complete logic, enabling highly complex and conditional automation. These actions were categorized as:

• Device Control: Manipulating device settings or functionalities, such as turning off Wi-Fi, adjusting volume, or activating the flashlight.
• Data Management: Operations on local files or data, including adding, editing, or deleting photos, contacts, or other stored information.
• Flow Control: Implementing conditional logic (if statements) and loops, allowing for sophisticated, dynamic, and repetitive actions.
• Communication: Utilizing various protocols and channels to interact with third parties, including SSH, email, text messages, and HTTP requests.

The combination of these high-privilege triggers and actions means that automation apps are inherently capable of:

1. Collecting and Exfiltrating Sensitive Information: Triggers like location changes or photo captures, coupled with communication actions (e.g., sending data via HTTP POST or email), allow for covert surveillance.
2. Manipulating Device Hardware and Software Functionality: Actions can remotely adjust critical device settings, affecting user experience or creating a sense of confusion.

These core capabilities were then mapped to the four identified types of IPV attacks:

• Surveillance: Achieved by combining data collection actions (e.g., "Get Last Photo," "Get Current Location") with data exfiltration actions (e.g., "Send Message," "Make HTTP Request"). The inclusion of "trace hiding" actions (e.g., deleting logs) further enhances stealth.
• Impersonation: Involves data insertion (e.g., composing a message) and data exfiltration (e.g., sending that message on the victim's behalf).
• Overloading: Typically uses flow control (loops) to repeatedly perform data exfiltration (e.g., sending numerous messages) or resource control (e.g., toggling Wi-Fi on/off rapidly).
• Local Control: Primarily relies on resource control actions to manipulate device functionalities or content (e.g., sending self-emails with unsettling content, changing wallpapers).

The inherent data flow within these automation recipes is crucial for understanding their potential for abuse. For example, a "Get Last Photo" action producing an output that then becomes the input for an "Encode Base64" action, which then feeds into an "HTTP POST Request" action, establishes a clear chain of data collection and exfiltration. This chain, even when embedded within the XML source code of the recipes, is what the detection system later leverages.

Demo / Proof of Concept

▶ Watch: Detecting malicious automation recipes: pipeline overview (8:00)

The talk included a compelling demonstration of how an automation app could be weaponized for surveillance on an iOS device. The proof-of-concept showcased a scenario where an attacker, having gained temporary physical access to the victim's iPhone, configured a malicious iOS Shortcut recipe.

The demo highlighted the following steps:

1. Victim Action: The victim takes a photo, which becomes the "last photo" on their device (in the demo, a picture of a cat).
2. Attacker Configuration: The attacker pre-configures an iOS Shortcut. This shortcut's sequence of actions is designed to:

• Fetch the Last Photo: Utilize the "Get Last Photo" action, which accesses the device's camera roll.
• Process the Photo: The transcript mentions actions like resizing, converting, and encoding the photo into a Base64 string. This prepares the image data for transmission.
• Exfiltrate Data: The encoded photo data is then sent via an HTTP POST request to a server controlled by the attacker.

1. Remote Activation: Crucially, the speaker emphasized that while the demo manually triggered the shortcut, this step could be automated remotely through various triggers (e.g., receiving a specific text message, entering a geofenced area, or at a preset time). This illustrates the stealthy and hands-off nature of such an attack once configured.
2. Data Reception: Upon the shortcut's execution, the HTTP POST request transmits the Base64 encoded photo data to the attacker's server.
3. Attacker Verification: The attacker downloads the received data, decodes it, and the original cat photo appears on their end, demonstrating successful covert surveillance.

This demonstration effectively illustrated the feasibility and stealth of using legitimate automation features for malicious data exfiltration. The fact that such a powerful capability resides within a seemingly benign, system-level application like iOS Shortcuts underscores the novel threat vector identified by this research.

Defensive Implications

▶ Watch: Using LLMs and attack operators for detection (9:00)

The findings of this research carry significant defensive implications for users, app developers, and the broader security community. The inherent design of automation apps, particularly their notification mechanisms, often makes it challenging for victims to detect malicious recipes. For instance, many actions can be configured to run silently or with minimal, easily dismissed alerts.

To address this, the researchers developed a robust detection pipeline specifically for iOS Shortcuts, which could be adapted for other platforms. This pipeline operates on publicly available shortcut recipes and involves several stages:

1. Data Collection: Scraping over 12,000 shortcut links from four public domains.
2. Conversion: Translating the .shortcut files into an XML source code representation.
3. Text Representation: Converting the XML into a human-readable text format, highlighting key actions and their data flow through UIDs (Unique Identifiers) that link outputs of one action to inputs of the next. For example, a shortcut might "Get Last Photo," then "Resize Image," then "Convert Image," then "Encode Base64," and finally "Send Message," with the UID ensuring the flow is preserved.
4. Initial Filter: A rule-based filter identifies recipes containing potentially exploitable actions, such as sending data via text message or accessing user location.
5. Large Language Model (LLM) Analysis: Filtered recipes are then fed into a fine-tuned LLM. The LLM is trained to identify specific attack operators abstracted from the four IPV attack types:

• Surveillance: Data collection, data exfiltration, trace hiding.
• Impersonation: Data insertion, data exfiltration.
• Overloading: Repeated data exfiltration, resource control.
• Local Control: Resource control.

The LLM reasons about the presence of these operators, the existence of execution paths between data collection and exfiltration, and indicators of stealth (e.g., actions with "Show When Run" toggled off). It then makes a reasoned decision on whether a recipe poses a threat.

The results of applying this detector to public domains were stark: 1,040 (approximately 8.7%) of the 12,000 collected shortcuts were deemed exploitable for IPV attacks. While "Local Control" represented the highest confirmed threat, the presence of surveillance and impersonation capabilities was also significant. The detector is open-sourced and available as a Docker image.

When the findings were presented to automation app developers, the responses highlighted the existing challenges. Tasker acknowledged the potential for misuse but struggled to identify concrete countermeasures. Apple's representative, while receptive, emphasized user responsibility in preventing unauthorized physical access, which, while true, doesn't fully address the inherent risks once access is gained.

Based on these insights, the researchers proposed several crucial mitigations:

• System-Level Identification and Warnings: Devices should proactively identify potentially problematic installed recipes and warn users about abusive execution patterns, perhaps through a lightweight local model.
• Per-Recipe Runtime Permissions: Automation apps should implement granular, per-recipe runtime permissions, allowing users to approve or deny specific actions (e.g., "Allow this shortcut to send messages?" rather than blanket app permissions).
• Persistent Notifications: When sensitive information or sensors are accessed by an automation recipe, persistent and clear notifications should be displayed, making covert operations more difficult.
• Collaboration with Tech Clinics: Security researchers and app developers should collaborate with local tech clinics and domestic violence support organizations to offer direct assistance and resources to victims.
• Online Self-Check Tools: Provide an online platform where users can upload or analyze their installed shortcuts for potential abuse, empowering victims with self-assessment tools.

These mitigations represent a multi-layered defense strategy, combining technological interventions with user education and support infrastructure to counteract the growing threat posed by the weaponization of automation apps.

Key Takeaways

• Automation Apps are a Hidden IPV Threat: Beyond traditional spyware, legitimate mobile automation applications like iOS Shortcuts and Android Tasker possess high privileges that can be weaponized for intimate partner violence (IPV).
• Four Distinct Attack Vectors: Abusers can leverage these apps for Surveillance, Impersonation, Overloading, and Local Control, exploiting their extensive capabilities to collect data, send messages, and manipulate device functions.
• Widespread Vulnerability in Public Recipes: A significant number of publicly shared automation recipes (1,040 out of 12,000 analyzed iOS Shortcuts) contain functionalities exploitable for IPV attacks, indicating a broad, unaddressed risk.
• LLMs as a Detection Tool: Large Language Models (LLMs) can effectively identify and analyze malicious patterns in automation recipes by understanding data flow and "attack operators," offering a novel approach to automated threat detection.
• Urgent Need for Developer Action: App developers must implement stronger security measures, including granular runtime permissions for recipes, persistent notifications for sensitive actions, and proactive identification of potentially abusive configurations.
• Empowering Users and Supporting Victims: Users need better tools for self-checking their devices, and collaboration with domestic violence support organizations is crucial to provide help for victims of tech-enabled abuse.

About the Speaker(s)

Shirley Zhang is the presenter of this work, focusing on the investigation into how automation apps can be weaponized for intimate partner violence. The research is a collaborative effort with Pochan, Jacob Orvald, Nishant Kolapati, Rahul Chadi, and Kasam Fas. While specific titles or affiliations for Shirley Zhang were not detailed in the transcript, her presentation at USENIX Security indicates expertise in cybersecurity research, particularly in the intersection of technology and societal issues such as abuse.