Lessons from OSV: Vulnerability Management for Open Source

Lessons from OSV: Vulnerability Management for Open Source

Speakers: Oliver, Google open source security team

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=qHcpKW2nOqI

Overview

Oliver from Google's open source security team presented a comprehensive talk at VulnCon, detailing the journey and principles behind the Open Source Vulnerability (OSV) schema. This JSON schema, an initiative of the OpenSSF, is designed to describe known vulnerabilities in open source packages in a minimal, consistent, and machine-readable format. Launched four years ago, OSV has rapidly gained traction, with adoption across major language ecosystems, open-source vulnerability databases, and Linux distributions. The core objective of OSV is to empower software developers to accurately identify and remediate vulnerabilities within their open-source dependencies, transforming a historically complex and error-prone process into an actionable and automated workflow.

The talk highlighted the significant impact of OSV, noting its collection of nearly 300,000 advisories spanning 31 ecosystems and 21 vulnerability databases. This extensive reach underscores its role in fostering a community-maintained, user-friendly, and comprehensive database for open-source vulnerabilities. Oliver delved into the six guiding principles that have shaped OSV's design and evolution: a laser focus on core use cases, simplicity, correctness and consistency, prioritizing open source, a federated database model, and unwavering backwards compatibility. These principles collectively ensure that OSV remains relevant, robust, and adaptable to the dynamic landscape of open-source security.

The importance of OSV cannot be overstated in today's software development environment, where open-source components form the backbone of nearly every application. Traditional vulnerability feeds often suffer from inconsistencies and a lack of machine-readability, making it challenging for developers to accurately map vulnerabilities to their specific package versions. OSV addresses these critical pain points by providing a standardized, actionable, and ecosystem-aware format, thereby significantly reducing the burden of vulnerability management and enhancing the overall security posture of open-source software.

Background

▶ Watch: Introduction to OSV schema and project goals (0:25)

The genesis of the Open Source Vulnerability (OSV) schema stemmed from persistent challenges faced by the industry, including Google, in effectively leveraging existing vulnerability feeds such as the National Vulnerability Database (NVD) and the common Vulnerabilities and Exposures (CVE) list. Historically, these feeds, while vital, were often difficult to parse and map to the specific package names and versions found in real-world software applications in a consistent and automated manner. This disconnect created a significant hurdle for software developers and security teams alike, leading to manual effort, potential inaccuracies, and delayed remediation of critical vulnerabilities.

The problem was exacerbated by the unique characteristics of open-source ecosystems. Unlike proprietary software, open source thrives on diverse communities, varied package managers, and often bespoke versioning schemes. A generalized approach to vulnerability data, without accounting for these ecosystem-specific nuances, proved inadequate. The lack of a standardized, machine-automatable format meant that vulnerability scanners struggled to produce accurate and actionable results, frequently leading to high rates of false positives or, worse, missed vulnerabilities.

Recognizing this critical gap, the OpenSSF initiated the OSV project approximately four years ago. The goal was to create a JSON schema that was not only minimal and easy to adopt but also deeply rooted in the realities of open-source development. The project aimed to foster a collaborative environment where vulnerability data could be maintained and consumed by the community, offering a developer-friendly approach to a pervasive security problem. By focusing on the specific needs of open-source software and its users, OSV sought to overcome the limitations of prior work and establish a new standard for open-source vulnerability intelligence.

Key Findings

▶ Watch: OSV's primary goal: actionable vulnerability remediation for developers (2:08)

The development and widespread adoption of the OSV schema are underpinned by a set of meticulously defined guiding principles, which represent the core findings and design philosophy of the project. These principles, refined over years of iteration, ensure the schema's effectiveness, maintainability, and relevance.

1. Core Use Cases Focus: The most crucial principle is a laser focus on enabling software developers to accurately identify and remediate known vulnerabilities in their open-source dependencies. This translates into catering to two primary user groups:

• Vulnerability Databases: The schema must facilitate easy adoption, export, and expression of comprehensive vulnerability information.
• Vulnerability Scanners: Scanners must be able to produce automated, actionable, and accurate results using OSV-formatted feeds. Other use cases, such as research requiring richer data, are considered out of scope if they compromise these core goals.

1. Simplicity: A concise and minimal schema is paramount. Each field must serve a distinct purpose directly linked to the core use cases, making it easy for both producers to supply data and consumers to understand. Aspirational fields that cannot be realistically populated or used are deliberately avoided, preventing unnecessary complexity and bloat.

1. Correctness and Consistency: OSV strives to enforce consistency and correctness, particularly in how vulnerability feeds map to package names and versions. This involves:

• Adhering to upstream rules for relevant ecosystems (e.g., Debian release encoding, PyPI package name normalization).
• Distinguishing between strict SemVer 2.0 ordering and ecosystem-specific version ordering rules.
• Providing explicit pseudocode for implementing version matching algorithms, ensuring predictable evaluation of vulnerable package versions.

1. Prioritizing Open Source: The schema's design is tailored specifically for open-source software. While some proprietary software might fit within the schema, explicit design choices that would increase complexity for broader scope (e.g., adding CPEs for proprietary software identification) are rejected. Decisions are made based on whether they provide actionable steps for open-source vulnerability scanners and users.

1. Federated Database Model: OSV promotes a distributed model where vulnerability databases are maintained by relevant communities or organizations (e.g., GitHub, Go, Rust communities). This aligns with the open-source ethos of distributed contributions. The schema provides extension points (database_specific or ecosystem_specific fields) for community-specific metadata, while enabling cross-referencing with external authoritative sources like EPSS and CISA KEV via the aliases field.

1. Backwards Compatibility: A fundamental commitment is never to introduce breaking changes. This means no removal of fields and no backwards-incompatible modifications. This principle ensures stability for consumers and producers but also places a high bar on adding new fields, as any additions are permanent.

These principles have guided the creation of a robust and widely adopted schema, evidenced by the collection of almost 300,000 advisories across 31 ecosystems and 21 vulnerability databases.

Technical Deep Dive

▶ Watch: Simplicity principle and the practical alias field (4:15)

The OSV schema is a testament to minimalist design, aiming for maximum utility with minimal complexity. Its entire structure can be condensed into a single slide, yet it provides the necessary granularity for accurate vulnerability management in open-source contexts. The core of OSV's technical prowess lies in its meticulous handling of vulnerability identification, relationships, and, critically, version matching across diverse ecosystems.

A fundamental aspect of OSV is its focus on solving the problem of vulnerability deduplication. It's common for a single vulnerability to be tracked under multiple identifiers, such as a GitHub Security Advisory (GHSA- prefix) and a Go vulnerability (GO- prefix). To address this, OSV introduces the aliases field, allowing vulnerability scanners to clearly deduplicate records that refer to the same underlying issue. This prevents redundant reporting and streamlines remediation efforts.

Beyond simple aliases, OSV also accounts for more complex relationships. The upstream field, for instance, was added to capture scenarios where one vulnerability record is derived from another. A prime example is Linux distributions, which often issue their own advisories (e.g., Ubuntu Security Notices, USN-prefixed) that bundle multiple upstream CVEs. In such cases, aliases would be inappropriate as it would incorrectly imply that all bundled CVEs are identical. The upstream field provides a clear, automatable way for scanners to reason about whether a Linux machine or container is affected by a specific upstream CVE, even when bundled within a distribution-specific notice.

A cornerstone of OSV's design is its commitment to correctness and consistency, particularly in how it handles package names and versioning across diverse open-source ecosystems. Historically, mapping generic vulnerability feeds to specific package names and versions used in actual software was fraught with ambiguity. OSV rectifies this by enforcing specific encoding rules and following upstream conventions:

• Debian Releases: Debian has multiple ways to refer to releases (code names, version numbers, ambiguous floating points or integers). OSV enforces a consistent encoding by referencing an authoritative Debian list.
• PyPI Package Names: The Python Package Index (PyPI) allows for an "infinite number of ways" to refer to the same package due to normalization rules (e.g., requests, Requests, requests-toolbelt). OSV mandates that PyPI package names are normalized according to Python's official rules.
• Maven Version Ordering: The Maven ecosystem has notoriously complex rules for version ordering. OSV acknowledges that most open-source ecosystems define their own unique version ordering rules, which often differ significantly from a generalized approach.

This leads to a critical distinction within OSV: between strict SemVer 2.0 ordering and ecosystem-specific version rules. While SemVer provides clear, unambiguous rules for version precedence, few ecosystems fully adhere to it in practice. OSV accommodates this by allowing semver and ecosystem types within its version ranges, ensuring that comparisons are always performed according to the relevant package manager's logic. To further aid implementers, OSV provides pseudocode detailing how to correctly implement version matching algorithms using OSV data, ensuring predictable and accurate results.

The schema's sophistication extends to handling Git versioning trees, moving beyond simple linear version ranges. Open-source projects frequently leverage Git for version control, involving multiple branches and complex merge histories. OSV can encode vulnerabilities in these non-linear structures using introduced and fixed events. For example, if a vulnerability is introduced at a specific commit and fixed in a later commit on a particular branch, OSV can represent this. The semantics dictate that unless an explicit fix event is specified within a branch, that branch is considered vulnerable. For more nuanced scenarios, where only a single branch contains a vulnerability, OSV uses a limit event instead of a fix event to denote the end of the vulnerable range for that specific branch. The same underlying algorithm used for linear version ranges applies, demonstrating the schema's flexibility.

A deliberate design choice for OSV is its narrow focus on open source. This means rejecting features that might serve proprietary software but would add unnecessary complexity or compromise core principles for open source. For instance, the addition of CPEs (Common Platform Enumerations) was rejected because CPEs are not predictable and do not align well with OSV's consistency principles, especially for identifying proprietary software lacking centralized package management. Similarly, a proposal to add a timestamp range type to mark vulnerable periods for services was rejected because it didn't provide actionable steps for open-source vulnerability scanners or users, reinforcing the "actionable" aspect of the schema's core use cases.

The federated database model is another cornerstone of OSV. It recognizes that different open-source communities (e.g., GitHub, Go, Rust) are best positioned to maintain vulnerability data for their respective ecosystems. This distributed approach promotes community ownership and allows for corrections and changes via standard open-source workflows (issues, pull requests). While OSV doesn't include fields like EPSS (Exploit Prediction Scoring System) or CISA KEV (Known Exploited Vulnerabilities) as top-level fields (as they come from single authoritative sources), it facilitates cross-referencing through the aliases field. Furthermore, OSV provides extension points via database_specific or ecosystem_specific fields, allowing home databases to include arbitrary JSON metadata that is relevant only to their specific context, such as Go import paths and symbols, GitHub's internal triage metadata, or Ubuntu's IU priority.

Finally, the principle of backwards compatibility is absolute: OSV will never introduce a breaking change. This means no fields will be removed, and existing fields will not be altered in incompatible ways. This commitment fosters long-term adoption and prevents fragmentation but also imposes rigorous scrutiny on any new field additions, as their consequences are permanent.

Demo / Proof of Concept

▶ Watch: Addressing historical challenges with vulnerability data consistency (6:35)

The efficacy and user-friendliness of the OSV schema are significantly amplified by the reference tooling and services built on top of it. These tools serve not only as practical demonstrations but also as essential platforms for driving adoption and gathering feedback.

The central hub for OSV data is OSV.dev. This platform acts as an aggregator for all OSV home databases, providing a unified interface for vulnerability information. Users can access OSV.dev through a web UI to browse and look up detailed vulnerability records. Crucially, it also offers an API, enabling programmatic access to vulnerability data for integration into automated workflows. OSV.dev goes a step further by directing users to the specific upstream processes for many records. For instance, if a user identifies incorrect information in a GitHub Security Advisory (GHSA) record displayed on OSV.dev, a direct link or button will guide them to the GitHub UI for submitting an advisory improvement, fostering a collaborative and self-correcting ecosystem.

Complementing OSV.dev is the OSV-Scanner. This client-side tool, also available as a library, provides out-of-the-box vulnerability scanning capabilities. It can analyze various inputs, including source code file systems and container images, to extract an SBOM (Software Bill of Materials) or an inventory of open-source dependencies. These identified dependencies are then matched against the comprehensive feeds available on OSV.dev to produce a precise list of vulnerabilities affecting the user's software.

A core objective of the OSV-Scanner is to deliver results that are both actionable and accurate. This is achieved by implementing sophisticated, ecosystem-specific functionality, such as dependency resolution algorithms and, critically, accurate version ordering. As highlighted in the technical deep dive, every open-source ecosystem has its own specialized rules for handling version numbers. The OSV-Scanner goes to great lengths to ensure high accuracy by covering a wide range of major ecosystems and applying their specific versioning logic. This meticulous attention to detail helps minimize false positives and ensures that developers receive reliable information about their security posture.

Together, OSV.dev and OSV-Scanner form an end-to-end platform that demonstrates the practical application of the OSV schema. They allow developers and security teams to easily integrate vulnerability scanning into their workflows, see how OSV data works in practice, and obtain actionable insights. This reference tooling has been instrumental in fostering adoption of the OSV schema, providing tangible benefits and serving as a vital feedback mechanism for both the schema itself and the data quality of upstream home databases.

Defensive Implications

▶ Watch: Enforcing consistent data for Debian and Pypi ecosystems (7:10)

The OSV schema and its associated tooling provide critical capabilities for defenders seeking to enhance their open-source vulnerability management strategies. Adopting OSV-centric approaches can significantly improve the accuracy, automation, and actionability of security efforts.

1. Standardized Vulnerability Data Consumption: Defenders should prioritize consuming vulnerability feeds formatted in the OSV schema. This provides a consistent, machine-readable format that simplifies integration into existing security tools and workflows, moving away from the ambiguities of older, less structured formats like raw CVE lists.
2. Leverage OSV.dev for Intelligence: Integrate the OSV.dev API into continuous integration/continuous deployment (CI/CD) pipelines or custom vulnerability management systems. This allows for programmatic querying of the latest open-source vulnerability data, ensuring that security checks are always based on the most current information. The web UI also serves as an excellent resource for manual lookups and understanding vulnerability details.
3. Implement OSV-Scanner for Automated Scanning: Utilize the OSV-Scanner tool or library to automate vulnerability detection across source code repositories, file systems, and container images. Its ability to generate an accurate SBOM and match it against OSV.dev feeds provides a robust first line of defense. Organizations should ensure the scanner is configured to respect ecosystem-specific versioning rules to minimize false positives and focus on truly impacted components.
4. Understand Ecosystem-Specific Versioning: Developers and security engineers must recognize that version ordering is not universal. Familiarity with how OSV handles semver versus ecosystem versions, and the underlying logic for different package managers (e.g., PyPI, Maven), is crucial for correctly interpreting scan results and assessing actual risk.
5. Prepare for VEX Integration: Looking ahead, the OSV community is actively exploring how Vulnerability Exploitability eXchange (VEX) can be effectively applied to open-source libraries to reduce the overwhelming number of false positives in vulnerability reports. Defenders should monitor these developments, as a distributed VEX model could drastically reduce the investigation time for transitive vulnerabilities. The vision is for package managers or version control systems to store "intermediate VEX files" that can be aggregated across a dependency graph. This would allow an application to automatically determine if it's truly affected by a transitive vulnerability, even if a component deep in its dependency tree is flagged. Adopting such a convention would enable shared burden of VEX investigation across the open-source ecosystem, leading to more precise and actionable vulnerability management. This is a significant future implication that promises to transform how organizations deal with the volume of alerts from open-source dependencies.

Key Takeaways

• Standardized, Actionable Vulnerability Data: The OSV schema provides a minimal, consistent, and machine-readable JSON format for describing open-source vulnerabilities, enabling developers to accurately identify and remediate issues.
• Ecosystem-Awareness is Crucial: OSV's design incorporates deep understanding of diverse open-source ecosystems, including specific rules for package naming, version ordering, and Git versioning, ensuring highly accurate vulnerability matching.
• Federated and Community-Driven: OSV promotes a distributed model for vulnerability databases, aligning with open-source principles and allowing communities to maintain their own vulnerability data, augmented by flexible extension points for ecosystem-specific metadata.
• Strong Backwards Compatibility: A strict policy of never introducing breaking changes ensures stability and long-term usability for both data producers and consumers, fostering widespread and sustained adoption.
• Comprehensive Tooling: OSV.dev acts as a central aggregator and API, while OSV-Scanner provides client-side, ecosystem-aware vulnerability scanning, demonstrating the practical application and benefits of the schema.
• Future of VEX in Open Source: The OSV community is actively exploring how a distributed model for VEX statements can reduce false positives and streamline vulnerability investigation across complex open-source dependency graphs.

About the Speaker(s)

The talk was delivered by Oliver, a member of the Google open source security team. His work focuses on improving security within the open-source ecosystem, particularly through initiatives like the Open Source Vulnerability (OSV) schema.