JBomAudit: Assessing the Landscape, Compliance, and Security Implications of Java SBOMs

JBomAudit: Assessing the Landscape, Compliance, and Security Implications of Java SBOMs

Speakers: Yue Xiao

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=jDvSTm-_w9A

Overview

In an era increasingly defined by complex software supply chain attacks, the talk "JBomAudit: Assessing the Landscape, Compliance, and Security Implications of Java SBOMs" by Yue Xiao at the NDSS Symposium presented critical research on the state of Software Bills of Materials (SBOMs) in the Java ecosystem. This presentation highlighted a pervasive lack of transparency in software components, a vulnerability that high-profile incidents like SolarWinds, Log4j, and XZ Utils have dramatically exposed. The core problem addressed is that while SBOMs are widely advocated as a solution to enhance visibility and mitigate risks, their real-world implementation often falls short of essential compliance and accuracy standards.

The research introduces JBomAudit, an innovative end-to-end tool designed to meticulously detect inconsistencies between generated Java SBOMs and the actual code they purport to describe. Xiao's work not only formalizes the various types of SBOM compliance issues—categorized into missing and incorrect dependencies—but also conducts a large-scale measurement study across over 25,000 Java SBOMs. The findings reveal a significant prevalence of non-compliant SBOMs, directly correlating these inaccuracies with undetected vulnerabilities and heightened security risks, thereby undermining the very purpose of SBOM adoption. This research is crucial for developers, security professionals, and regulatory bodies striving to secure the software supply chain against increasingly sophisticated threats.

Background

▶ Watch: Introduction to SBOMs and supply chain security problem (0:00)

The escalating frequency and severity of software supply chain attacks have thrust the issue of software transparency into the global spotlight. Incidents such as SolarWinds, the Log4j vulnerability (CVE-2021-44228), and the more recent XZ Utils backdoor (CVE-2024-3094) have demonstrated how a single compromised component or a hidden vulnerability within a dependency can have cascading effects, impacting millions of users and organizations worldwide. A fundamental challenge in mitigating these risks is the inherent opacity of modern software, where developers often integrate numerous third-party libraries without a clear understanding of their internal composition or transitive dependencies. This lack of visibility creates fertile ground for vulnerabilities to go undetected, leaving software exposed.

To counter this, governing bodies and regulators globally, including those in the US, UK, and Canada, have increasingly advocated for the widespread adoption of Software Bills of Materials (SBOMs). An SBOM is essentially an inventory list of software components, akin to an ingredient list for food. It details metadata about components, their versions, and their dependencies, providing organizations with crucial visibility into their software's internal structure. This visibility is intended to facilitate the identification and mitigation of vulnerabilities by enabling security teams to cross-reference known vulnerabilities databases with the components listed in an SBOM.

However, the efficacy of SBOMs hinges on their accuracy and completeness. Recognizing this, the US National Telecommunications and Information Administration (NTIA) established minimum requirements for SBOM compliance. These requirements mandate that an SBOM must provide a comprehensive and correct list of all top-layer direct dependencies and their transitive dependencies (second layer). Despite these guidelines, the research presented in this talk reveals a critical failure: a significant majority of real-world SBOMs fail to meet these fundamental requirements. For instance, a dependency actively used in the code, such as a dynamically loaded class, might be entirely omitted from an SBOM. This specific example highlights a failure to meet the NTIA's completeness requirement, rendering the SBOM non-compliant and, more critically, allowing associated vulnerabilities to remain hidden from vulnerability scanners that rely on SBOM data. The study primarily focused on the Java domain due to its mature SBOM adoption landscape, which allowed for the collection of a substantial dataset of over 25,000 SBOMs and their corresponding Java files, unlike other languages where SBOM adoption is still nascent.

Key Findings

▶ Watch: JBomAudit's focus on Java and research methodology (4:00)

The JBomAudit study revealed a concerning landscape of SBOM non-compliance within the Java ecosystem, identifying two broad and critical categories of dependency issues: missing dependencies and incorrect dependencies. Each category carries distinct security implications that undermine the very purpose of SBOMs.

Missing Dependencies are defined as components actively used in the distributed code but entirely absent from the SBOM. These omissions lead to significant security oversights. If a missing dependency harbors a vulnerability, it will not be tracked by standard vulnerability scanners that rely on SBOM data. Consequently, security teams remain unaware of the risk, fail to patch it, and leave the software vulnerable to potential exploits. The study further refined the classification of missing dependencies into:

• Missing direct dependencies: Core components directly integrated but not listed.
• Missing transitive dependencies: Second-layer dependencies that are utilized but not disclosed.
• Missing transitive relationships: Cases where a dependency is listed, but its relationship to a parent component is incorrectly omitted or misrepresented.

A large-scale measurement study conducted using JBomAudit across more than 25,000 Java SBOMs and their associated JAR files confirmed a significant number of these missing dependencies, particularly direct dependencies. This finding is particularly alarming as it directly compromises the reliability of SBOMs and exacerbates software supply chain risks. The research identified Guava, a popular Google library, as the most frequently omitted library by developers. Crucially, every version of Guava released before June 2023 is known to have at least one vulnerability. This means that for countless projects, the absence of Guava from their SBOMs translates directly into untracked, exploitable vulnerabilities.

Incorrect Dependencies, on the other hand, refer to components listed in the SBOM that are not actually utilized by the code. While seemingly less critical than missing dependencies, incorrect entries lead to substantial inefficiencies and potential misdirection for security teams. When an SBOM is bloated with irrelevant dependencies, security analysts may waste valuable time investigating and attempting to mitigate risks that simply do not exist within the deployed software. This unnecessary effort distracts from genuine threats, potentially delaying the identification and resolution of more critical, actual vulnerabilities. The study found that incorrect dependencies are commonly associated with categories such as annotation libraries, logging frameworks, and various utility libraries.

The security implications of both types of non-compliance are severe. A bubble chart presented in the talk vividly illustrated the distribution of vulnerabilities associated with non-compliant dependencies. The majority of these vulnerabilities were rated with high or critical severity scores, underscoring the profound security risks posed by inaccurate SBOMs. The size of the bubbles in the chart indicated the number of non-compliant SBOMs affected by specific vulnerabilities, revealing widespread exposure to significant threats due to these compliance failures. These findings unequivocally demonstrate that current SBOM generation practices are largely insufficient, creating a false sense of security and leaving organizations exposed to substantial, unmanaged risks.

Technical Deep Dive

▶ Watch: JBomAudit tool architecture and detection process (6:30)

The core of this research is the JBomAudit tool, an end-to-end solution meticulously designed to detect inconsistencies between Java SBOMs and the actual compiled code. JBomAudit's methodology is structured around three main components: Data Collection, Dependency Tree Extraction, and Compliance Check Analysis.

The Data Collection phase begins by systematically downloading artifacts hosted on the Maven platform, the central repository for Java libraries. The tool employs a keyword matching strategy to identify and collect SBOMs alongside their corresponding Java Archive (JAR) files. This ensures that for every SBOM analyzed, the actual compiled Java code it claims to represent is available for verification.

The most technically intensive component is Dependency Tree Extraction, which involves analyzing the compiled Java code to construct a true representation of its dependencies. For this purpose, the researchers developed a specialized binary analysis tool named JPK-tax. JPK-tax performs a deep scan of the bytecode within Java class files. Unlike traditional static analysis that might rely on source code or build system configurations, JPK-tax directly inspects the compiled instructions to accurately identify all utilized Java classes. This includes capturing dependencies arising from:

• External classes: Direct references to classes from other libraries.
• Method parameters: Types used as arguments in method calls.
• Annotations: Runtime-retained annotations that often imply dependencies.
• Dynamic features: More complex dependency patterns, such as those involving reflection, Class.forName(), or service loaders, which are often missed by metadata-only approaches.

After identifying the utilized Java classes, JPK-tax then recognizes the specific libraries or components that provide these classes. It iteratively constructs a comprehensive dependency tree directly from the JAR file, mapping out both direct and transitive dependencies as they are actually consumed by the application. Concurrently, JBomAudit parses the provided SBOM to obtain the dependency tree as disclosed by the SBOM generator.

The final stage, Compliance Check Analysis, involves a rigorous comparison between the dependency tree extracted by JPK-tax (representing the ground truth of code utilization) and the dependency tree parsed from the SBOM. This comparison allows JBomAudit to precisely identify any missing dependencies (components found by JPK-tax but absent in the SBOM) and incorrect dependencies (components listed in the SBOM but not identified as used by JPK-tax).

The study also delved into the root causes behind the widespread non-compliance. A primary factor is that most existing SBOM generators, such as the popular CycloneDX Maven plugin, predominantly rely on metadata files like pom.xml to generate SBOMs. These generators typically do not perform actual code analysis. However, pom.xml files are maintained by humans and are inherently error-prone. Any incompleteness or inaccuracy in this metadata directly translates into flawed SBOMs. Furthermore, these metadata-centric generators often prove insufficient when dealing with the complexities of modern Java projects. They may neglect other crucial metadata sources, such as OSGI bundles or various customized class file configurations, leading to an incomplete picture of dependencies. Beyond these technical limitations, other contributing factors include the improper usage of SBOM generators by developers and a general misunderstanding or misinterpretation of SBOM requirements. These systemic issues collectively contribute to the landscape of unreliable SBOMs, exacerbating security risks.

Demo / Proof of Concept

▶ Watch: Overall results: prevalence of non-compliant SBOMs (8:00)

While the talk did not feature a live, interactive demonstration of the JBomAudit tool in action, the presentation thoroughly elucidated its methodology and showcased the extensive results obtained from its large-scale measurement study. The speaker detailed the tool's three core components—data collection, dependency tree extraction via JPK-tax, and compliance checking—providing a conceptual walkthrough of its operational flow. The overall results, including figures illustrating the prevalence of missing and incorrect dependencies, the top 15 non-compliant libraries, and the security implications visualized through a bubble chart, served as a compelling demonstration of JBomAudit's capabilities and the critical insights it provides into SBOM accuracy. The artifact evaluation badges awarded to the project further validate the tool's functionality and the reproducibility of its findings.

Defensive Implications

▶ Watch: Security implications of non-compliant SBOMs (9:30)

The findings from the JBomAudit research carry profound implications for organizations striving to secure their software supply chains. Defenders must move beyond a passive reliance on automatically generated SBOMs and adopt a more proactive, verification-centric approach.

Firstly, organizations should re-evaluate their SBOM generation strategies. The study clearly demonstrates that SBOM generators relying solely on metadata files like pom.xml are fundamentally insufficient and prone to error. While these tools offer convenience, they fail to capture the actual runtime dependencies of complex Java applications, especially those involving dynamic loading or custom configurations (e.g., OSGI bundles). Defenders should advocate for and adopt SBOM generation tools that incorporate binary analysis capabilities, similar to JPK-tax, to inspect compiled bytecode and accurately reflect the utilized dependencies.

Secondly, security teams must recognize that SBOMs are not inherently trustworthy and require independent validation. Implementing processes to compare generated SBOMs against the actual deployed code can help identify discrepancies. Such validation is critical for ensuring the completeness and correctness of dependency lists, aligning with NTIA minimum requirements. Without this validation, vulnerability scanners built upon faulty SBOMs will inevitably miss critical security issues.

Thirdly, the focus on missing dependencies is paramount. The discovery that libraries like Google Guava, known to have high-severity vulnerabilities, are frequently omitted from SBOMs highlights a significant blind spot. Defenders should prioritize identifying and remediating vulnerabilities associated with these commonly missed, yet widely used, components. A robust vulnerability management program must account for the possibility of hidden dependencies and not solely rely on SBOM-driven scanning.

Finally, while the study did not definitively attribute non-compliance to malicious intent, the potential for malicious actors to intentionally hide vulnerable or malicious libraries by omitting them from SBOMs remains a serious concern. This possibility underscores the need for deep code analysis and runtime monitoring capabilities that can detect unexpected or undeclared dependencies, providing an additional layer of defense against sophisticated supply chain attacks. Organizations should consider integrating SBOM validation into their CI/CD pipelines to catch non-compliance issues early in the development lifecycle, preventing vulnerable software from reaching production environments.

Key Takeaways

• Widespread SBOM Non-Compliance: A large-scale study of over 25,000 Java SBOMs revealed a significant number of non-compliant SBOMs, failing to meet basic completeness and correctness requirements set by bodies like the NTIA.
• Two Critical Issue Types: Non-compliance manifests as either missing dependencies (components used in code but absent from SBOM) or incorrect dependencies (components listed in SBOM but not used in code). Both lead to significant security risks or wasted security efforts.
• Direct Security Implications: Missing dependencies, such as the frequently omitted Google Guava library (known for high-severity vulnerabilities), directly lead to undetected vulnerabilities and increased exposure to software supply chain attacks. The majority of vulnerabilities associated with non-compliant SBOMs are rated high or critical.
• Root Cause: Metadata Over-reliance: The primary reason for inaccurate SBOMs is that most generators solely rely on human-maintained metadata files (e.g., pom.xml) rather than performing actual binary code analysis. This approach is prone to human error and insufficient for complex Java projects.
• Need for Binary Analysis: Tools like JBomAudit, which employ binary analysis (e.g., JPK-tax) to scan bytecode for actual dependency utilization, are essential for generating accurate and compliant SBOMs.
• Defensive Imperatives: Organizations must move beyond metadata-only SBOM generation, validate their SBOMs against actual code, prioritize addressing commonly missed dependencies, and consider advanced code analysis to mitigate risks from non-compliant SBOMs.

About the Speaker(s)

Yue Xiao is a researcher whose work focuses on enhancing the transparency and security of software systems, particularly within the context of software supply chain security. As presented at the NDSS Symposium, their research on JBomAudit demonstrates a deep technical understanding of Java's dependency management intricacies and the challenges associated with generating accurate Software Bills of Materials. Xiao's contributions include formalizing SBOM compliance issues, developing sophisticated binary analysis tools like JPK-tax, and conducting extensive measurement studies to quantify the prevalence and implications of non-compliant SBOMs in real-world applications. Their work is pivotal in guiding the development of more reliable SBOM generation practices and strengthening defenses against supply chain attacks.