SoK: Safer Digital-Safety Research Involving At-Risk Users

SoK: Safer Digital-Safety Research Involving At-Risk Users

Speakers: Rosanna Bellini, Emily Tseng, Noel Warford, Alaa Daffalla, Tara Matthews, Sunny Consolvo

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=kcQt5zcMCKQ

Overview

In an increasingly interconnected world, all technology users face a myriad of digital safety threats, ranging from online harassment and privacy invasions to sophisticated security attacks. However, a specific demographic, termed at-risk users—including activists, disabled persons, and children—confront a disproportionately higher likelihood of technology-facilitated abuse and experience more severe harms when such incidents occur. Understanding the unique needs and vulnerabilities of these groups is paramount for developing effective protective solutions. This understanding necessitates empirical research, yet the very act of conducting such research introduces significant safety challenges for both the participants being studied and the researchers themselves.

Despite a growing body of published studies focusing on at-risk populations, there remains a critical void: a lack of consolidated, replicable guidance for researchers navigating these complex ethical and practical landscapes. This talk, presented by Rosanna Bellini on behalf of her co-authors, addresses this pressing issue head-on. It synthesizes existing knowledge and expert insights to present a robust framework of six actionable strategies designed to guide safer digital safety research, thereby empowering researchers to conduct impactful studies without inadvertently exposing vulnerable individuals or themselves to undue risk. The work emphasizes the need for a community-wide shift towards more transparent and methodologically sound safety protocols.

Background

▶ Watch: Introduction: Digital safety threats for at-risk users (0:00)

The premise of this research acknowledges that while digital safety interventions are crucial for all users, the stakes are considerably higher for at-risk groups. These groups are not only more frequently targeted by digital threats but also suffer more profound consequences, which can include physical harm, severe psychological distress, and significant socio-economic repercussions. Empirical research is indispensable for identifying these groups' specific needs, understanding adversary tactics, and developing tailored defenses. However, the sensitive nature of this work inherently generates a complex web of risks that demand careful consideration.

The authors meticulously categorized the types of risks encountered in digital safety research involving at-risk users. Participant risks are multifaceted, encompassing the potential for unauthorized data access—for instance, through hostile subpoenas from state adversaries—that could compromise identities or sensitive information. Direct engagement with participants, particularly through interviews or surveys, carries the risk of re-traumatization, forcing individuals to relive distressing experiences. Furthermore, the very act of publication of findings can inadvertently provide adversaries with actionable intelligence, potentially exposing participants or others in their network to harm if not handled with extreme care. Researchers themselves are not immune to these dangers. They face vicarious trauma and burnout from immersion in stories of abuse and hardship. Moreover, researchers investigating at-risk groups, especially those targeted by sophisticated adversaries, can become subjects of surveillance by the same actors, extending the threat landscape to the research team.

To address this intricate problem, the research began with a foundational dataset compiled by Warford et al., comprising 170 to 270 peer-reviewed papers on at-risk groups published in top security and privacy (S&P) venues. The current team expanded upon this by reviewing an additional 3,900+ papers published between 2020 and 2022, resulting in a comprehensive corpus of 196 relevant studies. Utilizing a rapid evidence review approach, they systematically extracted methodological and ethical information from these papers. The review revealed that while many authors were keenly aware of the risks and implemented various digital safety practices—such as seeking external or non-institutional ethical review, encrypting collected data, and refusing to report participant demographics to preserve anonymity—these practices were often reported inconsistently or lacked sufficient detail for replication. This inconsistency underscored a significant gap in the research community's approach to safety protocols.

Key Findings

▶ Watch: Identified risks for participants and researchers (2:00)

The comprehensive review of 196 papers yielded a critical finding that illuminated the core problem: a staggering 83.7% of works lacked replicable descriptions of safety practices. This pervasive inconsistency meant that even when researchers took significant precautions, the details of what was done, when, and why were often omitted or vaguely described, making it impossible for other researchers to learn from or build upon these efforts. This lack of transparency not only hinders cumulative knowledge but also perpetuates the risk of researchers inadvertently overlooking crucial safety measures in future studies.

In response to this identified gap, the research team embarked on a rigorous process to develop practical, consolidated guidance. They recruited a panel of 12 experts with extensive experience working with at-risk groups across industry, academia, and non-profit sectors. Over a period of nine months, these experts engaged in iterative rounds of informal discussions and oral history elicitations, sharing their lived experiences and insights into effective safety practices. Through a process of focused meetings and consensus building, the expert panel collaboratively established six core strategies for safer digital safety research. These strategies are designed to be flexible and adaptable, recognizing the diverse contexts and specific needs of various at-risk populations.

The six strategies are:

1. Engage experts early in the research process: Leveraging the knowledge of those with direct experience can preemptively identify risks and inform safer methodologies.
2. Assess and mitigate risks with threat modeling: Systematically identifying potential adversaries, their capabilities, and goals, and then designing research to counter these threats.
3. Select the lowest risk method that addresses the research goals: Prioritizing research approaches that minimize direct engagement with at-risk individuals when possible, or employing less intrusive methods.
4. Respect that at-risk users self-manage risks: Acknowledging and integrating participants' own security practices and risk assessments into the research design.
5. Be an advocate for at-risk user needs: Ensuring that the research actively contributes to the well-being and empowerment of the communities studied, and that findings are disseminated responsibly.
6. Handle data and publications carefully: Implementing robust data security protocols and exercising extreme caution in how research findings are presented to prevent harm.

These strategies collectively form a comprehensive framework aimed at elevating the standards of digital safety research, promoting ethical conduct, and ensuring the well-being of both participants and researchers.

Technical Deep Dive

▶ Watch: Problem: Inconsistencies in reporting safety practices (4:00)

The core contribution of this work lies in the detailed articulation and contextualization of its six proposed strategies, two of which were explored in depth during the presentation: threat modeling and selecting the lowest risk method. These strategies represent a significant step towards formalizing and standardizing safety protocols in digital safety research.

Strategy 2: Assess and mitigate risks with threat modeling.

In the broader security and privacy domain, threat modeling is a well-established practice for identifying potential adversaries, evaluating their capabilities, and understanding their motivations and goals. This strategy advocates for applying this rigorous methodology directly to the research process itself. This means viewing the research environment, data collection, storage, analysis, and dissemination as potential targets for adversaries, particularly when dealing with at-risk populations.

The presentation provided a compelling example from the work of co-author Alaa Daffalla and colleagues, who studied the privacy practices of political activists campaigning against their government.

• Adversaries Identified: The primary adversaries were nation-state actors who possessed significant resources and control over the country's telecommunications infrastructure.
• Capabilities and Goals: These actors had the capability to surveil, arrest, and potentially harm activists. Their goal was to suppress dissent and identify organizers.
• Evolving Threat Model: The research team recognized that the political climate and, consequently, the adversaries' tactics, could evolve rapidly. This necessitated an adaptive threat model.
• Mitigation Strategies:
• Expert Consultation: The team consulted with an expert with prior experience in the specific country to gain an accurate understanding of the local threat landscape.
• Secure Data Collection Environment: They ensured that interviewers and participants shared a cultural background and spoke the same language, fostering trust and nuanced communication.
• Publication Control: Crucially, recruitment protocols were deliberately omitted from publications to prevent adversaries from reverse-engineering participant identities or recruitment networks. This decision directly addressed the risk of findings being weaponized against the activists.

By proactively applying threat modeling, the research team was able to implement safeguards that effectively minimized potential harm to participants, demonstrating the strategy's practical efficacy.

Strategy 3: Select the lowest risk method that addresses the research goals.

This strategy encourages researchers to exercise extreme caution and creativity in designing studies involving sensitive topics and vulnerable populations. It posits that direct engagement with at-risk participants, while sometimes necessary, should not be the default or immediate choice. Instead, researchers should first explore lower-risk alternatives that can still yield valuable insights.

The talk highlighted two primary lower-risk approaches:

• Indirect Studies (e.g., measurements): These involve analyzing publicly available data or existing datasets that do not require direct interaction with at-risk individuals.
• Proxy Studies: This method involves engaging proxies—individuals who are close to or work directly with at-risk groups (e.g., aid workers, legal advocates, family members)—as stand-ins for direct engagement. Proxies can provide valuable perspectives without exposing the at-risk individuals to the immediate risks of research participation.

An illustrative example was drawn from the work of co-authors Emily Tseng, Nikki Dell, and Thomas Ristenpart, who sought to study the perspectives of abusers in technology-mediated intimate partner violence (IPV) settings.

• High-Risk Engagement: Directly engaging perpetrators of IPV posed significant risks not only to the abusers themselves (who can also be at-risk in various contexts) but, more importantly, to survivors and researchers.
• Strategic Delay: The team wisely delayed direct data collection.
• Measurement Studies: Instead, they conducted measurement studies by analyzing publicly available online communities where individuals discussed tactics for technology abuse. This indirect approach allowed them to:
• Gain initial insights into abuser perspectives and methods.
• Equip the team with a deeper understanding and expertise, enabling them to formulate more informed and safer questions for potential future studies, should direct engagement become necessary and feasible.

This approach exemplified how strategic use of lower-risk methods can lay essential groundwork, build expertise, and fulfill research goals without incurring immediate, high-stakes risks.

While not detailed in the presentation, the other four strategies also contribute significantly to the overall framework:

• Engage experts early: This ensures that research design benefits from practical experience and nuanced understanding of specific at-risk contexts from the outset.
• Respect that at-risk users self-manage risks: This acknowledges the agency and existing coping mechanisms of vulnerable individuals, integrating their self-protection strategies into research design rather than imposing external ones.
• Be an advocate for at-risk user needs: This emphasizes the ethical imperative for research to be beneficial and supportive, ensuring findings are used to empower rather than exploit.
• Handle data and publications carefully: This strategy underpins all others by emphasizing robust data governance, secure storage, and judicious reporting to prevent inadvertent harm.

Together, these six strategies provide a comprehensive and deeply considered guide for researchers navigating the ethical and practical complexities of digital safety research involving at-risk users, moving beyond mere ethical approval to proactive, informed risk mitigation.

Demo / Proof of Concept

▶ Watch: Six expert-derived strategies for safer research (4:50)

While this talk does not feature a traditional software or hardware demonstration, the efficacy and practical application of the proposed strategies are powerfully illustrated through detailed case studies that serve as real-world proofs of concept. These examples, drawn from the speakers' own collaborative research, effectively "demonstrate" how the theoretical strategies translate into actionable research practices that mitigate risk and ensure participant safety.

The first such demonstration involved the application of threat modeling in the study of political activists, led by co-author Alaa Daffalla. This was not a simulated scenario but a description of actual research where the team rigorously identified nation-state adversaries, analyzed their capabilities (e.g., control of telecommunications infrastructure), and proactively implemented safeguards. The decision to omit recruitment protocols from publications is a tangible outcome of this threat modeling process, directly demonstrating how an understanding of adversary tactics can inform publication strategy to prevent harm. This real-world application showcases threat modeling as a dynamic, essential component of safe research design, proving its utility in protecting highly vulnerable participants.

The second "demonstration" highlighted the selection of the lowest risk method through the work of co-authors Emily Tseng, Nikki Dell, and Thomas Ristenpart on technology-mediated intimate partner violence. Instead of directly engaging with abusers—a high-risk endeavor for all involved—they demonstrated the effectiveness of measurement studies using publicly available online communities. This strategic pivot allowed the researchers to gather crucial insights into abuser perspectives and tactics without exposing survivors or the research team to immediate, direct risks. This example serves as a powerful proof of concept for the strategy, illustrating how indirect methods can fulfill research objectives, build expertise, and inform future, potentially more direct, inquiries in a safer, more phased manner. These case studies, therefore, function as vital empirical evidence, validating the practical utility and protective capacity of the proposed digital safety strategies within the complex landscape of at-risk user research.

Defensive Implications

▶ Watch: Strategy 2: Selecting lowest risk methods for abusers (7:05)

The implications of this work are profound, primarily targeting the research community and the institutions that support them. The strategies presented by Bellini et al. provide a much-needed framework for establishing a more robust and ethical approach to digital safety research involving at-risk users. In this context, "defenders" are researchers themselves, tasked with protecting their participants, their research integrity, and their own well-being against the inherent risks of this critical field.

For individual researchers, the immediate implication is a call to action regarding transparency and methodological rigor. The finding that 83.7% of previous works lacked replicable descriptions of safety practices highlights a systemic issue. Researchers are now encouraged to:

• Report Safety Strategies Explicitly: Integrate detailed descriptions of specific safety practices and mitigation strategies into research deliverables, particularly in the methodology sections of papers. This moves beyond mere ethical approval statements to a comprehensive account of how participant and researcher safety was actively managed. This transparency is crucial for peer learning and for enabling future researchers to build upon established best practices.
• Proactive Risk Assessment: Adopt a mindset of proactive risk assessment, integrating threat modeling into the very design of their research projects. This involves systematically identifying potential adversaries, understanding their capabilities and motivations, and designing safeguards specific to the research context and participant vulnerabilities.
• Methodological Prudence: Prioritize the lowest risk methods that can still achieve research goals. This encourages creative problem-solving, exploring indirect studies, measurement studies, or proxy engagement before resorting to direct, high-risk interactions with at-risk individuals.

For the broader research community, the implications extend to establishing collective standards and fostering a supportive environment:

• Standardization of Safety Protocols: The community should work collaboratively towards establishing flexible, context-specific standards for safety protocols. These standards must be adaptable to the wide range of at-risk users and the diverse circumstances in which research is conducted, moving away from a one-size-fits-all approach.
• Community Building: Actively build and nurture a dedicated digital safety research community. This involves creating forums for sharing experiences, best practices, and challenges, thereby fostering a culture of collective responsibility for safety.
• Elevating Publication Standards: Integrating these safety strategies into calls for papers at conferences and journals. By explicitly asking for detailed accounts of safety measures, the community can naturally encourage higher standards and facilitate the development of novel conferences or specific paper tracks dedicated to methodological safety in this domain.

Finally, institutional support is critical for the successful adoption and implementation of these strategies:

• Funding Programs: Institutions and funding bodies should establish dedicated programs that recognize and fund the additional time, expertise, and resources required to develop and enact comprehensive safety plans.
• Training for Researchers: Provide specialized training for researchers on topics such as threat modeling, ethical considerations in sensitive research, trauma-informed approaches, and secure data handling.
• Valuing Quality over Quantity: Shift institutional incentives to value the quality, ethical rigor, and safety of research over sheer publication quantity. This encourages researchers to invest the necessary time in robust safety planning rather than rushing through studies.

By embracing these defensive implications, the digital safety research ecosystem can evolve to better protect its most vulnerable stakeholders, ensuring that the pursuit of knowledge genuinely contributes to a safer digital world without inadvertently creating new avenues for harm.

Key Takeaways

• Digital safety research involving at-risk users (e.g., activists, disabled persons, children) is crucial but inherently poses significant risks to both participants and researchers.
• A comprehensive review of existing literature revealed that 83.7% of studies lacked replicable descriptions of safety practices, highlighting a critical gap in guidance and transparency.
• Through expert consensus, six actionable strategies were developed to guide safer digital safety research, emphasizing proactive risk mitigation and ethical conduct.
• Threat modeling is essential for researchers to systematically identify adversaries, understand their capabilities, and design safeguards specific to the research context.
• Prioritizing the lowest risk method (e.g., indirect studies, proxy studies) before direct engagement is crucial, allowing for valuable insights without incurring immediate, high-stakes risks.
• The research community and institutions must provide robust support, including funding, specialized training, and a shift towards valuing quality and ethical rigor over publication quantity, to effectively implement these safety strategies.

About the Speaker(s)

The presentation was delivered by Rosanna Bellini, who is a key co-author of this significant work. The research itself is a collaborative effort involving a distinguished group of experts: Emily Tseng, Noel Warford, Alaa Daffalla, Tara Matthews, and Sunny Consolvo.

While specific titles and affiliations for all speakers were not detailed in the transcript, the collective expertise of the authors is evident in the depth and rigor of their work. Noel Warford is credited as part of the foundational dataset for the research, indicating prior significant contributions to the field of at-risk user studies. Alaa Daffalla is highlighted as a close collaborator whose work on political activists served as a practical example of applying threat modeling in sensitive research. Similarly, Emily Tseng is recognized for her collaborative work on studying abusers in intimate partner violence settings, providing a key illustration of the "lowest risk method" strategy. The panel of 12 experts recruited for developing the strategies, which included professionals from industry, academia, and non-profit sectors, further underscores the diverse and practical experience informing this critical synthesis. The team's collective background spans the necessary domains to address the complex challenges of digital safety research involving vulnerable populations.