Secrets & Shadows: Leveraging Big Data for Vulnerability Discovery

Secrets & Shadows: Leveraging Big Data for Vulnerability Discovery

Speakers: Bill Demirkapi

Conference: DEF CON 32

YouTube: https://www.youtube.com/watch?v=-KXgcWuv-Ug

Overview

In "Secrets & Shadows: Leveraging Big Data for Vulnerability Discovery," independent security researcher Bill Demirkapi presents a compelling case for shifting traditional perspectives on vulnerability discovery, particularly within cloud environments. The talk aims to introduce a methodology for identifying and exploiting common bug classes at scale, specifically focusing on dangling cloud resources and leaked secrets. Demirkapi emphasizes that while complex low-level software vulnerabilities often dominate security discussions, simpler misconfigurations or missing identity checks in cloud infrastructure can yield a far broader and more significant impact.

Demirkapi, drawing from his diverse background in both low-level software and cloud security, highlights his personal motivation for focusing on the cloud: the ability to achieve substantial security impact without necessarily requiring the same degree of technical complexity associated with deep software exploitation. The core premise of the talk is to challenge existing, often limited, approaches to vulnerability discovery and demonstrate a more expansive, data-driven methodology. This shift in perspective is promised to be illustrated through practical exploitation examples, though these specific examples are not detailed within the provided transcript.

The talk sets the stage by defining fundamental cloud concepts and the inherent risks associated with cloud authentication mechanisms. It underscores the critical difference in impact between a minor API key leak and a full compromise of root cloud credentials, laying the groundwork for understanding why large-scale discovery of such vulnerabilities is paramount for modern security.

Background

▶ Watch: Introduction to talk: Secrets & Shadows, vulnerability discovery (0:00)

Cloud computing has fundamentally transformed how organizations deploy and manage their IT infrastructure. At its core, cloud computing allows users to create infrastructure on demand—including servers, websites, storage, and more—without the need to purchase, maintain, or house physical server racks. This model operates on the principle of borrowing from a shared pool of resources managed by a cloud provider over the internet, offering significant cost savings and scalability, particularly for smaller operations or rapidly evolving projects.

Securing access to these internet-managed cloud environments is critical. For platforms like Amazon Web Services (AWS), access is typically controlled via an access key. An AWS access key is a credential that can be either short-lived or long-term, specifying permissions for an account and enabling the management of AWS resources according to the assigned role. The AWS SDK, for instance, automatically utilizes these credentials to sign API requests, facilitating secure and convenient access for workloads. While using temporary credentials, such as those provided by an AWS session token, is strongly recommended for enhanced security, the use of long-term credentials remains a common and risky practice. Similarly, Google Cloud offers various authentication methods, including service accounts, CLI credentials, application default credentials, impersonated service accounts, and metadata server-based authentication, all relying on different forms of tokens or credentials.

A significant security risk inherent in cloud provider tokens is the default lack of stringent restrictions on their usage. By default, a root access key in AWS, for example, can be used by anyone from any IP address to manage an entire cloud environment. The impact of a leaked cloud credential varies drastically depending on its scope and permissions. A leaked Google Maps API key, while potentially leading to billing overcharges or service abuse, typically has a limited blast radius. In stark contrast, a leaked root AWS access key poses a far more severe threat. If an organization stores its database or computing infrastructure in AWS, an attacker with root credentials could not only access sensitive customer data but also disrupt or interfere with critical production workflows, leading to catastrophic business consequences. This fundamental difference in potential impact underscores the urgency of addressing cloud security vulnerabilities at scale.

The talk specifically highlights two common bug classes: dangling cloud resources and leaked secrets. A dangling cloud resource is defined as a cloud resource that has been deallocated or removed from an environment but is still referenced by a DNS record. This can create opportunities for attackers to take over these resources through techniques like subdomain takeover. The speaker notes that the cloud business model inherently relies on sharing resources, which, while efficient, introduces additional security risks if not managed meticulously. The discussion around dangling resources also touches on distinctions between primitive versus shared resources, and qualifiers such as whether an endpoint is used by only one customer or shared with many. Relevant DNS record types that can lead to dangling resources include A*, CNAME, NS, and MX records, among others. The other primary bug class, leaked secrets, refers to the unauthorized exposure of sensitive credentials, API keys, and other access tokens, which, as previously discussed, can grant attackers broad control over cloud environments.

Key Findings

▶ Watch: Speaker's independent research and security background (0:17)

The provided transcript focuses primarily on introducing the problem space and the speaker's motivation, rather than detailing specific key findings, research results, or novel contributions from an extensive study. While the talk's title, "Leveraging Big Data for Vulnerability Discovery," suggests a methodology that would yield significant findings, the transcript does not delve into the practical application of this big data approach, nor does it present any quantitative results, specific vulnerability discoveries, or new techniques developed by Demirkapi. The "Key Findings" section, as typically understood in a technical article, would involve the outcomes of the research; however, these details are not present in the brief introductory segment provided.

Technical Deep Dive

▶ Watch: Why cloud security offers a much wider impact (1:07)

While the talk promises a deep dive into leveraging big data for vulnerability discovery at scale, the provided transcript largely serves as an introduction and definition of the problem space, rather than a detailed technical exploration of the methodology. The speaker identifies two critical bug classes: dangling cloud resources and leaked secrets.

For leaked secrets, the technical implications revolve around the nature of cloud authentication tokens. As discussed, AWS access keys (comprising an access key ID and a secret access key) and various Google Cloud credentials (such as service accounts or CLI credentials) are the primary means of programmatic access. The danger lies in their default permissive nature, where a leaked credential, especially a root key, can grant unrestricted access to a cloud account from any IP address. The technical challenge in discovering these at scale involves scanning vast amounts of publicly accessible data (e.g., code repositories, public S3 buckets, misconfigured web servers) for patterns matching these sensitive tokens. However, the specific tools, regex patterns, data sources, or big data processing frameworks employed for this "at scale" discovery are not elaborated upon in the transcript. The discussion remains at a conceptual level, highlighting the severity of such leaks rather than the mechanisms of their discovery.

Regarding dangling cloud resources, the technical definition provided is a resource that is "deallocated from your environment while still referenced by a DNS record." This vulnerability typically leads to subdomain takeover, where an attacker can claim the dangling resource (e.g., an S3 bucket, a virtual machine, or a CDN endpoint) and then control the domain or subdomain pointing to it. The process of identifying these involves:

1. Enumerating DNS records: Collecting a comprehensive list of DNS records for target organizations or broad internet scans.
2. Resolving associated cloud resources: Determining the cloud provider and specific resource type (e.g., AWS S3 bucket, Azure Blob Storage, Google Cloud Storage) that a DNS record points to.
3. Checking resource existence/ownership: Verifying if the referenced cloud resource still exists and, if so, whether it is owned by the expected entity or if it has been deallocated.
4. Identifying claimable resources: If a resource is deallocated, checking if it can be registered or claimed by an attacker.

The transcript briefly mentions "primitive resources" versus "shared resources" and qualifiers related to whether an endpoint is used by "one customer" or "many customers," implying different attack surfaces or complexities in identifying dangling resources. It also lists relevant DNS record types such as A*, CNAME, NS, and MX records. However, the detailed technical process for automating this discovery at scale, the specific types of "big data" being leveraged (e.g., passive DNS data, internet-wide scan data), or the architectural components of such a system are not described. The talk outlines the what and why of these vulnerabilities but stops short of the how from a technical implementation perspective.

Demo / Proof of Concept

▶ Watch: Cloud computing basics: borrowing shared resources (2:00)

The provided transcript does not include any description or mention of a demonstration or proof of concept. While the speaker states the goal is to "shift our perspective through practical exploitation," the details of such practical examples or any live demonstrations are not present in this introductory segment of the talk.

Defensive Implications

▶ Watch: Introduction to AWS access keys for managing resources (2:08)

Although the transcript does not explicitly outline a dedicated "Defensive Implications" section, the risks highlighted by Bill Demirkapi provide clear guidance for cloud defenders. The core message is that seemingly simple misconfigurations in cloud environments can have a disproportionately large impact, often exceeding that of complex software vulnerabilities.

For leaked secrets, the primary defensive strategy revolves around stringent credential management:

• Minimize long-term credentials: The speaker explicitly states that using long-term credentials like root AWS access keys is not recommended. Defenders should prioritize short-lived credentials and session tokens (e.g., AWS Session Tokens) which expire automatically, significantly reducing the window of opportunity for attackers if a credential is leaked.
• Principle of Least Privilege: Cloud access keys and tokens should always be granted the minimum necessary permissions required for their function. This limits the blast radius if a credential is compromised.
• Restrict IP access: Where possible, restrict access to cloud resources and credentials by specific IP addresses or ranges. While a root AWS key is dangerous because "anyone from any IP can use that access key," configuring IP restrictions or using services like AWS VPC endpoints can mitigate this risk.
• Secrets Management Solutions: Implement dedicated secrets management solutions (e.g., AWS Secrets Manager, HashiCorp Vault) to securely store, rotate, and access credentials, preventing hardcoding in code or insecure storage.
• Proactive Scanning: Regularly scan public repositories, internal codebases, and other potential exposure points for inadvertently leaked credentials.

For dangling cloud resources and preventing subdomain takeovers:

• Regular DNS Audits: Organizations must perform continuous audits of their DNS records to identify any entries pointing to deallocated or non-existent cloud resources.
• Resource Decommissioning Procedures: Establish robust procedures for decommissioning cloud resources, ensuring that associated DNS records are updated or removed before or concurrently with the deallocation of the resource.
• Monitoring for Takeover Attempts: Implement monitoring to detect suspicious activity related to DNS records or attempts to provision resources that could lead to a takeover.
• Utilize Cloud Provider Features: Leverage cloud provider features that help prevent dangling resources, such as domain validation services or resource tagging for better inventory management.

The overarching defensive implication is to acknowledge the unique attack surface presented by cloud environments and to shift security strategies towards comprehensive identity and access management, rigorous configuration auditing, and continuous monitoring, rather than solely focusing on traditional application-level vulnerabilities. The emphasis on "vulnerability discovery at scale" implies that defenders, too, need scalable approaches to identify and remediate these widespread cloud misconfigurations.

Key Takeaways

• Cloud Security Impact: Simple cloud environment misconfigurations, such as missing ID checks or leaked credentials, can have a broader and more significant impact than complex low-level software vulnerabilities.
• Token Dangers: Cloud provider tokens and access keys (e.g., AWS access keys, Google Cloud CLI credentials) are inherently dangerous due to often permissive default settings, allowing access from any IP address without restriction.
• Impact Varies: The severity of a leaked credential depends heavily on its scope; a Google Maps API key has limited impact, whereas a root AWS access key can lead to full data compromise and production workflow interference.
• Dangling Cloud Resources: A critical vulnerability arises when deallocated cloud resources are still referenced by DNS records, creating opportunities for subdomain takeovers.
• Vulnerability Discovery at Scale: The talk advocates for shifting from traditional, limited vulnerability discovery methods to a big data-driven approach to identify these widespread cloud security issues efficiently.
• Prioritize Temporary Credentials: Defenders should always use short-lived credentials and session tokens, and never rely on long-term root access keys, to minimize the risk window of a compromise.

About the Speaker(s)

Bill Demirkapi is an independent security researcher with a diverse professional background spanning both low-level software security and cloud security. He describes himself as being interested in both offensive and defensive security disciplines, driven by a passion for solving challenging security problems. Demirkapi notes his particular interest in cloud security stems from its potential for wider impact, often achievable through addressing less technically complex, yet highly consequential, vulnerabilities such as missing ID checks. He explicitly states that the work presented in this talk was conducted independently of his employer, using his own resources.