Building a PSIRT for a Standards Organization

Building a PSIRT for a Standards Organization

Speakers: Jim (details inferred from transcript)

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=HzSkFOKggso

Overview

This talk, delivered at VulnCon, provides a detailed account of the speaker's experience in establishing a Product Security Incident Response Team (PSIRT) capability within a standards organization, specifically the Trusted Computing Group (TCG). The speaker, identified as Jim, aims to present a template that other groups can adapt for their own vulnerability response needs. The core challenge addressed is how to build a robust vulnerability response process in a collaborative environment where member companies are often competitors, intellectual property is paramount, and traditional incident response paradigms may not directly apply.

The establishment of such a PSIRT is critical because standards organizations, like the TCG, develop foundational protocols and specifications that underpin widely used technologies, such as the Trusted Platform Module (TPM) chips found in virtually every modern phone and laptop. Flaws in these standards or their reference implementations can have widespread security implications, affecting numerous vendors and end-users. The talk emphasizes that while the specific context is the TCG, the principles and lessons learned are broadly applicable to any standards body or consortium grappling with the complexities of multi-party vulnerability coordination and disclosure.

The speaker's extensive background in incident response, including being the first full-time PERT member at Cisco and early involvement with FIRST, lends significant weight to the presented framework. The initiative to create a formal vulnerability response process at TCG stemmed from observed informal and potentially risky "hallway conversations" about flaws among competitors, highlighting the urgent need for a structured and secure mechanism for handling security issues. This article will delve into the specific challenges, solutions, and key considerations involved in building such a specialized PSIRT.

Background

▶ Watch: Talk goal: PSIRT template for standards organizations (0:25)

The genesis of the TCG's vulnerability response capability was rooted in the inherent challenges of security in a collaborative, competitive standards environment. The speaker personally witnessed informal discussions among member company representatives regarding potential flaws in specifications or implementations. These "hallway conversations" often involved speculation about whether a competitor might share the same flaw and whether to disclose it, raising significant alarm bells about the potential for mishandling, uneven disclosure, and other detrimental outcomes. This lack of a formal process created a high risk of uncoordinated, ineffective, or even harmful responses to security vulnerabilities.

The speaker, with support from his employer (Juniper Networks at the time) and other like-minded board members, initiated the effort to formalize vulnerability response within the TCG. The TCG itself is a worldwide, highly active membership group responsible for developing protocols and supporting integrated circuits around trusted computing, with the TPM standard being a prime example. A crucial aspect of the TCG's operational environment is that its members are often direct competitors working together on common goals, a dynamic that heavily influences and constrains any vulnerability response framework.

The initial phase involved forming a Vulnerability Response Subcommittee (VRS), which met for ten months to solicit feedback. This process yielded over 150 substantive comments, which were instrumental in shaping what became the Vulnerability Response Framework. The term "framework" was deliberately chosen to emphasize a guidelines-based, flexible approach rather than a rigid, prescriptive one. Within this framework, a Vulnerability Response Team (VRT) was recommended for establishment. A key design decision was to insulate the VRT from the TCG board via the VRS, creating a crucial buffer. The VRS acts in a managerial capacity, providing oversight, while the VRT handles the hands-on incident response work.

Several critical distinctions and challenges emerged during the framework's development. First, the team anticipated that most incoming reports would involve resolving a report of an issue that the TCG itself couldn't directly fix, but rather had to hand off to member companies. Only a minority of cases would involve a vulnerability directly owned by TCG (e.g., in a specification or reference code). Second, a significant misalignment was identified regarding confidentiality expectations: while standards group members understood confidentiality, their interpretation differed from the stringent requirements of a product security incident response team. Finally, intellectual property (IP) was recognized as an existential concern for a standards group, becoming the ultimate "buck stops here" factor in vulnerability reporting and resolution. These foundational insights shaped every aspect of the PSIRT's design and operational protocols.

Key Findings

▶ Watch: TCG context: Competitors collaborating on standards (2:08)

The development and implementation of the TCG's VRT yielded several key findings and design principles crucial for effective vulnerability response in a standards organization:

1. Framework over Rigidity: The adoption of a "vulnerability response framework" rather than a rigid policy was vital. This allows for flexibility and adaptation, which is essential given the diverse nature of potential vulnerabilities and the competitive yet collaborative environment of a standards body.
2. Separation of Oversight and Operations: The deliberate separation of the managerial Vulnerability Response Subcommittee (VRS) from the operational Vulnerability Response Team (VRT) provided a critical buffer. The VRS, composed of board-nominated members, provides oversight and strategic direction, while the VRT, comprising product security incident response experts, handles the day-to-day technical work. This prevents undue influence or conflicts of interest from board-level politics or competitive pressures.
3. Confidentiality Misalignment: A significant discovery was the difference in understanding of confidentiality between typical standards group members and PSIRT professionals. The latter require a much higher, more specific level of confidentiality, particularly regarding sensitive vulnerability details and pre-disclosure information. This necessitated explicit protocols for information sharing and handling.
4. Intellectual Property is Existential: For a standards organization, intellectual property is not just important; it's existential. The framework had to meticulously address concerns about relinquishing control over sensitive IP. An unforeseen obstacle was the intense alarm among some board members who feared losing control of their companies' IP by participating in a shared vulnerability response process. This required a "charm offensive" to allay concerns and demonstrate that the framework protected, rather than compromised, member IP.
5. The "Careful Dance" with Subject Matter Experts (SMEs): VRT members, while experts in incident response, often lack the deep technical knowledge required for highly specialized TCG topics like cryptography. Engaging Subject Matter Experts (SMEs) from working groups is necessary, but fraught with risk. The "careful dance" involves initial, constrained communication: VRT members must explicitly state information sharing constraints to SMEs before revealing sensitive details. SMEs are asked to only reply to the VRT member and not to share information with others if they cannot assist, preventing inadvertent disclosure to potentially problematic individuals (e.g., competitors or those with a history of mishandling incidents).
6. Resolving a "Report" vs. a "Vulnerability": The VRT primarily focuses on resolving a report of an issue, meaning they identify the affected parties (e.g., member companies, specific working groups) and facilitate the handoff for remediation. Only in specific cases (e.g., flaws in TCG specifications or reference code) does the VRT directly resolve a vulnerability that TCG itself "owns." This distinction is crucial for setting expectations and defining the scope of the VRT's responsibilities.
7. Multi-Party Coordination is Inevitable: Issues can often extend beyond the TCG's direct control, impacting the broader industry. The framework explicitly accounts for escalation to larger coordinating bodies like CISA (implied by "Vince" or generic policy to go to multi-party teams in Europe, Asia, and North America). Conversely, external multi-party issues might affect TCG members, requiring established contact points. The potential nightmare scenario of misaligned information sharing when an external coordinator works with TCG members who are also internally reporting issues highlights the need for extreme flexibility and common sense.

These findings underscore the unique complexities of establishing a PSIRT in a multi-stakeholder, competitive, and IP-sensitive environment, providing valuable lessons for similar organizations.

Technical Deep Dive

▶ Watch: Vulnerability Response Framework and VRT structure (3:40)

The operational mechanics of the TCG VRT are designed to be efficient, secure, and respectful of the unique organizational context.

VRT Staffing and Operations:

The ideal size for the VRT was determined to be four to six incident responders, with five currently serving. Members are nominated by the TCG board and approved by the VRS. Crucially, their VRT membership and participation outlasts any changes in board membership, providing continuity and insulating the team from political shifts. However, if a member's employer leaves the TCG, their VRT participation ends due to the dissolved membership relationship.

The VRT functions much like any other TCG work group, handling administrative tasks. Members internally select a chair and co-chair. For incident allocation, a simple round-robin scheme is employed: a plain text file lists VRT members, and for each new incident, the TCG admin assigns it to the next person on the list. If that person cannot handle the issue, it rotates to the next.

Issue Types and Handling:

The VRT categorizes incoming issues into several types, each with specific handling procedures:

1. Vendor Product Flaw: This is the most common type, where a member company's system has a vulnerability. The VRT's role is to notify the affected member(s) and identify all other potentially affected members to ensure comprehensive remediation. This falls under "resolving a report," as the TCG itself does not fix the vendor's product.
2. Specification or Reference Document Flaw: These are defects, typos, or logical flaws in the TCG's own official documentation that could lead to a vulnerable condition. Since TCG owns these documents, the VRT handles these as typical product security incidents, similar to a regular PSIRT dealing with its own product flaws.
3. TCG Reference Code Flaw: While reference code is not intended for production use, it sometimes is. If a vulnerability is found in TCG's reference code (which is based on reference documents), it's treated as a typical PERT case. This is a rarer occurrence but has happened at least once.
4. Non-TCG Issues: This broad category includes implementation or specification questions that might hint at a potential vulnerability but aren't actual flaws, or completely unrelated vulnerabilities reported to the TCG's public security contact. For TCG-related questions, the VRT hands them off to the appropriate work group chair. For entirely unrelated issues, if the responsible party can be identified, the report is forwarded; otherwise, it's dismissed. This again highlights the "resolving a report" function.

Multi-Party Coordination:

The framework acknowledges that some issues may transcend TCG's scope, becoming industry-wide. In such cases, the VRT policy allows for escalation to major coordinating bodies (e.g., CISA, or other national/regional CERTs like JPCERT/CC in Japan). The policy is deliberately generic to allow flexibility in choosing the appropriate external coordinator. Conversely, external multi-party issues might impact TCG, prompting inbound contact. A dedicated incident manager is selected for these cases. The speaker specifically warns of a "potential nightmare" scenario where an external coordinator engages individual TCG members, while the internal VRT is also working on the same issue, leading to potential misalignments in communication and disclosure. This demands significant flexibility and common sense to navigate.

Administrative Support and Tooling:

The VRT receives dedicated administrative support from the TCG admin team. Instead of the entire admin staff, one specific individual is assigned to the VRT. This admin is trained not only in general TCG procedures but also in incident response tooling and management. They manage secure communication channels, such as a Signal group, and ensure the availability of PGP keys for encrypted communication.

For the VRT members themselves, the tooling is standard for a modern PSIRT:

• A public security page detailing how to engage the group and report issues.
• Secure reporting channels monitored by the TCG admin.
• Proficiency in PGP for secure email communication.
• Knowledge of CVSS (Common Vulnerability Scoring System) for scoring vulnerability severity.
• Understanding and application of TLP (Traffic Light Protocol) for indicating information sharing restrictions.

Notably, the TCG operations have no need to become a CVE Numbering Authority (CNA). Given the low incident rate (1-3 annually) and the fact that most VRT members' employers are already CNAs, TCG can leverage existing CNAs or go directly to the CVE program for numbering if needed. However, in practice, members often prefer to avoid "owning" a TCG issue CVE, leading to direct requests to the CVE program.

Communications, Approvals, and Legal:

Given the IP-centric nature of TCG, publishing security advisories or external communications is highly sensitive. The general rule is that nothing is published without full board sign-off, a process incompatible with rapid security disclosure. To address this, a crucial process innovation was implemented: any two TCG officers (instead of the full board) can now approve external communications or publications in an emergency, significantly accelerating response times.

Furthermore, an anticipated "outsized dependence on legal counsel consultation" was identified. To avoid delays and costs associated with seeking board approval for every legal query, the board pre-allocated a specific number of legal consultation hours (e.g., $800 worth, or four hours) for emergency VRT use. This "eliminates a hurdle" for rapid response.

Demo / Proof of Concept

▶ Watch: Distinction: Resolving a report vs. a vulnerability (4:48)

This talk focuses on the development and operationalization of a Product Security Incident Response Team (PSIRT) and its associated processes within a standards organization. As such, it describes a framework, policies, and operational procedures rather than a specific technical exploit or vulnerability.

The speaker did not present a live demonstration or a proof of concept of a vulnerability or exploit. Instead, the "demonstration" of the framework's success is implied by its successful establishment in late summer 2019, its continued operation, and the positive feedback received from TCG members, with an average of one to three incidents handled annually. The talk is a blueprint for building a capability, not a showcase of its immediate technical output in terms of vulnerability exploitation.

Defensive Implications

▶ Watch: Strategy for careful interaction with SMEs (7:40)

The TCG's journey in building its PSIRT offers profound defensive implications, particularly for other standards organizations, industry consortia, or any multi-stakeholder environment where shared security responsibility is paramount. The speaker explicitly presents this as a "template" for others to use.

1. Proactive PSIRT Establishment is Essential: The primary takeaway is that standards groups need a formal vulnerability process. Relying on informal "hallway conversations" is a recipe for disaster. A structured PSIRT capability is fundamental for responsible stewardship of shared specifications and technologies, especially those as critical as TPM.
2. Tailor to Organizational Context: The framework approach, rather than a rigid policy, is crucial. Organizations must adapt their PSIRT structure to their unique environment, considering factors like competitive dynamics, IP sensitivity, and existing governance structures. The TCG's model of separating oversight (VRS) from operations (VRT) and insulating the VRT membership is a powerful pattern for maintaining neutrality and effectiveness.
3. Prioritize IP Protection and Legal Alignment: For IP-driven organizations, meticulously addressing intellectual property concerns is non-negotiable. This involves proactive engagement with legal counsel, pre-allocating legal consultation hours, and establishing expedited approval processes for security advisories. Failure to do so can stall critical disclosures and erode trust.
4. Master SME Engagement: Defenders in complex technical fields must recognize that their incident response teams may not possess all necessary deep technical expertise. The "careful dance" of engaging Subject Matter Experts (SMEs) with explicit confidentiality constraints is a vital defensive tactic to prevent inadvertent information leakage and ensure the right expertise is brought to bear securely.
5. Understand "Report vs. Vulnerability": Defenders should clarify whether their role is to "resolve a report" (coordinate remediation by others) or "resolve a vulnerability" (directly fix a flaw they own). This distinction helps define scope, manage expectations, and allocate resources appropriately, preventing overreach or under-delivery.
6. Prepare for Multi-Party Coordination: Security incidents rarely stay within organizational boundaries. Implementing a generic policy for escalating to and coordinating with larger multi-party incident response teams (e.g., CISA, national CERTs) is crucial. Furthermore, anticipating the complexities of simultaneous internal and external coordination on the same issue requires high degrees of flexibility and common sense from incident managers.
7. Leverage Standardized Tooling and Protocols: Even with unique organizational constraints, standard PSIRT tooling and protocols like PGP, CVSS, and TLP remain essential for secure communication, consistent severity assessment, and controlled information sharing. There's no need to reinvent the wheel for these core functions.
8. Review Membership Requirements and History: Regular review of organizational membership requirements is important to identify potential information sharing obligations or conflicts. Additionally, researching the organization's history for older, unaddressed flaws can uncover latent risks that need to be addressed, preventing future exploits based on legacy vulnerabilities.
9. Continuity and Insulation of Response Team: Protecting the membership of the response team by having their participation outlast board changes ensures stability and expertise retention, preventing political interference from disrupting critical security operations.

By adopting these principles, organizations can build resilient and effective PSIRT capabilities that protect their stakeholders, maintain trust, and uphold the integrity of their standards and technologies.

Key Takeaways

• Standards groups unequivocally need a formal vulnerability process, as informal discussions are insufficient and risky.
• It's crucial to distinguish between "resolving a report" (coordinating a fix by others) and "resolving a vulnerability" (fixing a flaw the organization owns directly).
• Engage with Subject Matter Experts (SMEs) one-on-one with extreme constraint, clearly outlining information sharing boundaries to prevent inadvertent disclosure.
• Anticipate and plan for multi-party coordination scenarios, both for escalating issues and responding to external industry-wide incidents.
• Recognize that intellectual property is existential for a standards group; all vulnerability response processes must meticulously protect it.
• Negotiate and establish express, expedited processes for publishing security advisories and making external communications decisions (e.g., two officers instead of a full board).
• Regularly review membership requirements of the organization for potential information sharing obligations or conflicts.
• Keep the oversight group (e.g., VRS) separate from the operational response team (e.g., VRT) to maintain independence and focus.
• Protect the membership of the response team, ensuring their participation provides continuity and is insulated from board-level changes.

About the Speaker(s)

The speaker, identified as Jim, brings a wealth of experience in product security and incident response. He has a long-standing career in the field, having worked on the original Mars worm and developed the first classes for "eenics" before SANS existed. Notably, he was the first full-time PERT (Product Emergency Response Team) member at Cisco and has been involved with FIRST (Forum of Incident Response and Security Teams) since 1990. While employed at Juniper Networks, Jim served as the alternate for Juniper's board seat at the Trusted Computing Group (TCG), which is where his initiative to develop the TCG's vulnerability response capability originated. His deep historical and practical expertise in incident response makes him a highly credible authority on establishing and operating PSIRTs in complex organizational environments.