Architecting the Modern SOC: The Evolving AI Reality for Blue Teams

Architecting the Modern SOC: The Evolving AI Reality for Blue Teams

Speakers: Nicole Grinstead, Senior Director of Platform Enterprise and Application Security, Roblox (Moderator); Bryan Fite, AI Sherpa, Worldwide Technology; Swathi Joshi, SVP of Cyber Defense Engineering and Operations, TransUnion; Dean De Beer, Co-founder and CTO, Command Zero

Conference: BSides SF

YouTube: https://www.youtube.com/watch?v=ENNEMVKKo1g

Overview

This panel discussion, "Architecting the Modern SOC: The Evolving AI Reality for Blue Teams," delves into the transformative impact of artificial intelligence on Security Operations Centers (SOCs). Featuring a panel of industry experts, the talk explores the long-held aspiration of a "sockless" future, where manual alert handling is minimized, and examines how the advent of advanced AI and machine learning capabilities might finally bring this vision closer to reality. The discussion highlights the critical evolution of SOC analyst roles, shifting from rote pattern recognition and low-value triage to more strategic functions like decision-making, crisis management, and context engineering.

The panel confronts the challenges and opportunities presented by AI, addressing concerns around job displacement, the nuances of integrating AI into existing security architectures, and the imperative of building trustworthy and responsible AI systems. It underscores that while AI promises significant productivity gains and expanded coverage, it also necessitates a fundamental rethinking of data management, human-in-the-loop processes, and inter-departmental collaboration. For blue teams, understanding these shifts is paramount to effectively leverage AI, enhance defensive posture, and navigate the complex, rapidly evolving cybersecurity landscape.

Background

▶ Watch: Panel introduction: Embracing an agentic future in SOC (0:00)

The concept of a "sockless" future, where security operations are so automated and efficient that a traditional SOC becomes unnecessary, has been an industry aspiration for years. Nicole Grinstead and Swathi Joshi recalled their experience at Netflix circa 2017, where a vision of high-fidelity, true-positive detections eliminating the need for human analysts was pursued. Despite a homogeneous environment and an in-house detection platform designed to focus on high-loss scenarios, the reality was that the alert queue grew, driven by an expanding attack surface, increasing content spending, and growing employee numbers. This led to the eventual establishment of a SOC, demonstrating the limitations of automation without the advanced AI capabilities available today.

For the past two decades, CSOs (Chief Information Security Officers) have sought to "replace Tier 1" analysts, often outsourcing these functions to MSPs or MDRs, only to bring them back in-house. The underlying desire was not to eliminate humans, but to elevate Tier 1 analysts to function more like Tier 2 or Tier 3 professionals, focusing on higher-order tasks rather than repetitive pattern recognition. This historical context highlights a persistent problem: the sheer volume of alerts and the cognitive burden placed on analysts. The panel argues that the current generation of AI, particularly large language models (LLMs) and specialized AI agents, offers a new paradigm for addressing these challenges, potentially fulfilling the long-standing desire to make SOC operations more efficient and strategic.

Key Findings

▶ Watch: Core question: Is the SOC analyst role disappearing? (4:00)

The panel converged on several key findings regarding the future of the SOC in an AI-driven world:

• Evolution, Not Elimination, of SOC Roles: The traditional Tier 1, Tier 2, Tier 3 model is becoming outdated. While AI will significantly reduce low-value triage and manual alert handling, the human analyst role will evolve towards authority decision-making, crisis response, detection life cycle management, and agent life cycle management. Entry-level jobs may change, requiring analysts to be more agile and possess business acumen beyond technical screen-staring.
• The "Qless and Alertless" SOC: The new vision for security operations is one where manual alert handling is a thing of the past, replaced by an agentic future where AI systems manage and process alerts autonomously. However, investigation as a function will not disappear; it will be human-augmented, focusing on complex scenarios.
• Data as the Foundation for AI Success: The effectiveness of AI in the SOC hinges on the quality and structure of data. Ontology engineering – the process of building structured knowledge about an organization's data – becomes critical. AI models need not only security data but also operational data (GitHub, AWS, SharePoint, HR) to build comprehensive context. Organizations must codify their unique business context to allow AI to differentiate between a critical incident and a routine policy violation.
• Human in the Loop Shifts from Gatekeeper to Collaborator: Initially, humans will act as gatekeepers for AI actions, especially for high-impact activities like containment and remediation, ensuring transparency and building trust boundaries. As AI systems mature and prove reliable, the human role will transition to that of a collaborator, working alongside AI agents to investigate and resolve complex threats, rather than merely validating their outputs.
• Measuring AI's Impact Beyond Simple Metrics: Success metrics for AI in the SOC should go beyond traditional alert closure rates. Instead, focus should be on false positive reduction, deduplication rates, alert clustering, and reducing the cognitive load on analysts. Measuring the efficiency of AI tools involves assessing their ability to present comprehensive decision-making, provide investigation memory, and offer all necessary information for human-assisted decisions.
• Architectural Shifts are Imperative: To truly enable an AI-powered SOC, organizations must adopt API-first injection, streaming pipelines, and near real-time detections. The output from AI systems must also be machine readable to feed into other automated components. This enables multi-source correlation and reduces reliance on single telemetry sources like EDR.
• Responsible AI Integration: While AI offers significant acceleration, particularly for simple rule generation and enrichment, it's not a silver bullet. Organizations must select the right model for the task (e.g., small, deterministic language models for specific, redundant tasks rather than creative LLMs). Guard rails, effective governance, and a deep understanding of AI pitfalls like hallucination, bias, and prompt injection are crucial. Training models with localized business context takes significant time and effort (e.g., 10-15 months for one TransUnion example).

Technical Deep Dive

▶ Watch: Dean: SOC roles evolve, Tier 1-3 structure is outdated. (4:30)

The integration of AI into the modern SOC necessitates significant architectural and operational shifts, moving towards an agentic future where intelligent agents play a central role in security operations. This evolution is predicated on several technical pillars.

Firstly, the concept of ontology engineering emerges as paramount. Dean De Beer emphasized that for AI systems to be effective, especially LLMs, they require highly structured and contextually rich data. This extends beyond traditional security telemetry (e.g., SIEM data) to include operational sources like GitHub, AWS logs, SharePoint content, and even HR data. The goal is to build a comprehensive understanding of the organization's unique environment, codifying what constitutes a "critical incident" versus a "policy violation" within specific business contexts. This upfront work, performed by human "context engineers," is essential because "products don't have context; language models don't have context" of a specific organization's nuances.

The panel discussed the shift in the human in the loop paradigm. Initially, humans serve as gatekeepers, validating AI's outputs and actions, particularly for high-impact operations like containment or remediation. This is crucial for establishing trust boundaries with new, non-deterministic systems. As trust grows, the role evolves into a collaborator, where humans and AI agents work together, with AI surfacing important investigations that humans then refine or act upon. This speeds up triage by allowing AI to perform preliminary analysis, determine true/false positives, and assess the breadth and correctness of investigations.

Measuring the efficacy of AI is not straightforward. Swathy Joshi highlighted metrics beyond simple alert closure rates, advocating for false positive reduction, deduplication rates, and alert clustering as indicators of productivity gains. The aim is to reduce the cognitive load on analysts and expand coverage. Tool efficiency is measured by the AI's ability to present all its decision-making, maintain investigation memory, and provide clustered alerts, enabling analysts to make informed "yes" or "no" decisions quickly. Brian Fite added that a focus on risk-reward optimization and continuous improvement, rather than "big bangs," is key. He suggested measuring success by reducing "clicks" (e.g., from 20 clicks to 2 clicks for a workflow) and token consumption, comparing these against traditional methods.

Architecturally, enabling the AI-powered SOC means moving towards API-first injection and streaming pipelines for near real-time detections. The output from AI systems must be machine readable, as agents will increasingly act as consumers of this output, feeding it into subsequent automated processes. This shift from relying solely on structured telemetry like EDR to multi-source correlation significantly enhances reasoning capabilities. Swathy noted that while vendors provide global threat intelligence, organizations must invest in training models with their localized business context. This process is time-intensive, with one example taking 10-15 months to achieve human-assisted and eventually fully AI-led decisions.

Addressing the pitfalls of LLMs, such as hallucination risk and prompt injection, Brian Fite emphasized the importance of picking the right model. He suggested using small language models that are more deterministic and focused on specific tasks (e.g., an IP tracker that understands network transversals but nothing else) rather than general, creative LLMs. Guard rails are essential to make "the right thing easy to do and the wrong thing hard to do." The discussion also touched upon total telemetry and the concept of a federated data fabric, where data is available on demand without being "data hoarders" in a "land of 10,000 data lakes." This enables the creation of "digital doppelgangers" or "digital twins" for testing permutations of changes, moving towards a more proactive security posture. The panel also warned against API sprawl and the risk of mass hallucinations if threat catalogs are poisoned, or if prompts are strung together indiscriminately. Bryan cited NIST 600-1, "the dirty dozen," referring to 12 areas of harm that AI systems can exhibit, which need to be considered in every production environment. He also introduced the concept of cognitive security, noting that attacks that work on humans (like social engineering) can also work on "weird machines."

Dean suggested that if an AI system misses something, similar to how human analysts are fallible, the incident should lead to an after-action report and lessons learned. This data can then be used to "teach" the AI system that context, ideally preventing the same miss from happening again—a distinct advantage over human memory.

Demo / Proof of Concept

▶ Watch: Evolving analyst skills: ontology engineering, data, agent management. (5:50)

The panel discussion did not include a specific live demonstration or proof of concept of an AI-powered SOC system. Instead, the speakers focused on theoretical frameworks, architectural considerations, and practical experiences regarding the implementation and impact of AI in security operations, drawing from their respective roles and organizational contexts.

Defensive Implications

▶ Watch: Swathy: SOC function shifting to Q-less, alert-less investigations. (8:00)

The insights from the panel provide a clear roadmap for blue teams looking to leverage AI effectively and enhance their defensive capabilities:

• Embrace Role Evolution: Defenders must recognize that SOC analyst roles are evolving. Instead of fearing job displacement, analysts should focus on developing skills in decision-making, crisis management, context engineering, and managing the lifecycle of AI agents. Training programs should adapt to cultivate these higher-order analytical and strategic skills.
• Prioritize Data Strategy: The foundation of an effective AI-powered SOC is high-quality, structured, and contextualized data. Blue teams need to invest heavily in ontology engineering and data management practices. This involves not only collecting security telemetry but also integrating data from diverse operational sources (HR, cloud platforms, development tools) to provide comprehensive localized business context to AI models.
• Implement Gradual Automation with Human Oversight: Start with focused, low-risk automation. AI should first target low-value triage, alert enrichment, decoration, and basic, deterministic actions like blocking, sandboxing, and machine segmentation. For high-risk decisions, such as containment or executive reporting, maintain human in the loop oversight, evolving from a gatekeeper to a collaborator role as trust in AI systems grows.
• Architect for AI-Native Operations: Shift security architectures towards API-first injection, streaming pipelines, and near real-time detections. Ensure that both input and output data for AI systems are machine readable to facilitate seamless integration and automation across different security tools and processes. Adopt multi-source correlation to enhance reasoning and reduce reliance on single data sources.
• Experiment Responsibly and Measure Success: Avoid a "big bang" approach to AI adoption. Instead, conduct focused experiments, measure their impact (e.g., false positive reduction, cognitive load reduction, workflow efficiency gains like reducing "clicks"), and iterate based on results. This iterative process allows for continuous improvement and helps build confidence in AI systems.
• Build Trustworthy and Responsible AI: Defenders must actively participate in building trustworthy and responsible AI systems. This includes carefully selecting AI models, implementing strong guard rails to mitigate risks like hallucination and bias, and establishing clear governance frameworks. Understanding and proactively addressing the "dirty dozen" areas of AI harm (NIST 600-1) is critical.
• Foster Cross-Departmental Collaboration: The vision of an AI-powered SOC extends beyond security. Blue teams should work to dissolve traditional departmental boundaries, using AI to minimize the friction between security operations and other functions like HR or IT. Automating routine requests for information (e.g., user activity reports) can enhance overall organizational efficiency and allow security teams to focus on more critical tasks. The goal is to make the SOC more autonomous by reducing dependencies on other teams through clear SOPs and automation.
• Develop a Creative Threat Catalog: Proactively consider how AI systems themselves can be targeted. Develop a "creative threat catalog" that includes scenarios like "lying to robots" (poisoning data) and "hiding from robots" (evading detection). Utilize "digital doppelgangers" or "digital twins" to simulate and test AI systems against various attack permutations, enhancing system confidence.

Key Takeaways

• Evolving Roles, Not Elimination: AI will transform SOC roles, shifting human analysts from low-value triage to higher-order tasks like strategic decision-making, crisis management, and context engineering, rather than replacing them entirely.
• Data is Paramount: The success of AI in the SOC hinges on high-quality, structured, and contextualized data, necessitating robust ontology engineering and integration of diverse operational data sources.
• Iterative & Responsible AI Adoption: Organizations should adopt AI incrementally through focused experiments, rigorously measuring success metrics like false positive reduction and cognitive load reduction, and continuously iterating on human in the loop processes.
• Architectural Transformation: Enabling an AI-powered SOC requires fundamental shifts towards API-first injection, streaming pipelines, near real-time detections, and ensuring machine readable outputs for seamless agent-to-agent communication.
• Mitigating AI Risks: Proactive measures, including selecting specific, deterministic small language models for tasks, implementing strong guard rails, and addressing hallucination risk and bias, are essential for building trustworthy and responsible AI systems.
• Beyond the SOC: AI can dissolve traditional security boundaries, fostering greater autonomy for the SOC by automating routine tasks and improving collaboration with other departments, ultimately enhancing overall organizational security posture.

About the Speaker(s)

• Nicole Grinstead served as the moderator for the panel. She is a Senior Director of Platform Enterprise and Application Security at Roblox.
• Bryan Fite contributed to the discussion as an AI Sherpa at Worldwide Technology.
• Swathi Joshi shared her expertise as the SVP of Cyber Defense Engineering and Operations at TransUnion.
• Dean De Beer provided insights from his role as Co-founder and CTO at Command Zero.