Keynote: Mind the Gap: Bridging Cloud Native Innovation with Real-World Use

Keynote: Mind the Gap: Bridging Cloud Native Innovation with Real-World Use

Speakers: Carlo Physico, Chief Technology Officer, Pepton; Fabio, Head of Engineering, Pepton; Tyson Singer, VP of Technology, Platform (Spotify); Katie Gamanji, Senior Engineer, Apple

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=JqG1wey7-Ao

Overview

This keynote presentation, titled "Mind the Gap: Bridging Cloud Native Innovation with Real-World Use," brought together leaders from diverse industries—finance, biotech, platform development, and consumer electronics—to share their experiences in leveraging cloud-native technologies and open-source principles to tackle complex, real-world challenges. The talks collectively explored how large enterprises and innovative startups are moving beyond theoretical cloud-native concepts to implement practical, scalable, and secure solutions in demanding environments. From managing massive Kubernetes deployments in a highly regulated financial institution to accelerating drug discovery with AI on distributed clusters, fostering developer productivity through open-source platforms, and building privacy-centric AI infrastructure, the speakers highlighted the transformative power and practical considerations of cloud-native adoption.

The presentations underscored a central theme: the "gap" between the rapid pace of cloud-native innovation and the intricate demands of real-world use cases. Each speaker showcased how their respective organizations are actively bridging this divide by strategically adopting, adapting, and contributing to the open-source ecosystem. This includes navigating regulatory hurdles, optimizing for extreme scale and performance, ensuring stringent privacy and security, and cultivating thriving developer experiences. The insights shared provide a comprehensive look at the maturity and versatility of the cloud-native landscape, offering valuable lessons for organizations at every stage of their digital transformation journey.

The keynote featured contributions from a representative from HSBC detailing their enterprise Kubernetes journey, Carlo Physico and Fabio from Pepton on their AI-driven drug discovery for disordered proteins, Tyson Singer from Spotify discussing the impact and evolution of their open-source developer portal Backstage, and Katie Gamanji from Apple outlining the architecture of Private Cloud Compute for Apple Intelligence. These varied perspectives painted a rich picture of how cloud-native technologies are becoming foundational to innovation across the global economy.

Background

▶ Watch: HSBC's Kubernetes journey, scale, and challenges (0:00)

The journey into cloud-native adoption presents unique challenges for organizations, particularly those operating at massive scale or within highly specialized and regulated sectors. This keynote provided a multi-faceted look at these challenges and the innovative solutions being developed.

For a global financial institution like HSBC, their Kubernetes journey, initiated in 2018, quickly escalated to managing hundreds of clusters supporting 7,000 production services across diverse markets and handling 600 million discrete hits daily. This rapid scaling exposed problems not typically seen in smaller deployments, necessitating robust strategies for cost management, blast radius control, and seamless upgrades within a highly regulated environment demanding "hundreds of IT controls." Their operational model remained centralized, diverging from the industry trend towards smaller, devolved clusters, due to the need for high consistency and compliance.

Pepton, a Swiss biotech company, confronts a fundamental challenge in drug discovery: understanding disordered proteins. These proteins, implicated in a vast majority of cancers and neurodegenerative diseases, are notoriously difficult to characterize using standard experimental techniques because of their dynamic, ever-changing structures. Traditional methods often capture only static snapshots, akin to "looking at a single frame" of a video, failing to reveal the crucial dynamics required for effective drug targeting. This gap in understanding necessitated entirely new experimental and computational approaches.

Spotify faced a different kind of scaling problem: managing developer experience across over 700 R&D squads. With a rapidly growing microservices architecture, developers grappled with context switching, fragmentation, and cognitive load, hindering productivity and innovation. To address this, Spotify developed an internal developer portal (IDP) called Backstage. The decision to open-source Backstage in March 2020 was not merely about sharing but driven by an ambitious vision for it to become "the standard for IDPs," a move that would introduce its own set of complexities related to balancing internal priorities with community needs.

Finally, Apple introduced Apple Intelligence, a personalized AI system designed to bring powerful generative models to its devices. A core tenet for Apple is user privacy, meaning that while many AI tasks are processed locally on the device, more sophisticated requests demand greater computational capacity. This led to the development of Private Cloud Compute (PCC), a cloud-based extension that had to meet Apple's stringent privacy and security standards, effectively extending the "privacy and security of your iPhone from the silicon on up" into the cloud environment. This required careful selection and integration of open-source technologies to ensure both performance and an uncompromising privacy posture.

Key Findings

▶ Watch: Pepton's introduction and focus on disordered proteins (3:20)

The keynote delivered a compelling set of findings from each organization, showcasing practical applications and strategic advancements in cloud-native adoption.

HSBC demonstrated that enterprise-scale Kubernetes deployments, even in highly regulated financial sectors, are not only feasible but can be optimized for extreme resilience and cost efficiency. Their key finding was the necessity of sophisticated operational strategies such as workload sharding across markets to manage blast radius and implementing blue-green cluster configurations for upgrades, ensuring stability and minimal disruption. They also highlighted the critical role of a shared responsibility model and proactive cost accountability tools in managing cloud expenditure at scale. Their roadmap findings included the move to a hot-hot cluster model, externalizing data like etcd for enhanced resilience, adopting a declarative GitOps-based model for infrastructure, and the anticipated "hundreds of thousands of dollars" in annual cloud bill savings through the adoption of ambient mode Istio with Solo.io.

Pepton presented a groundbreaking finding in biotech: the development of a novel hydrogen deuterium exchange mass spectrometry (HDX-MS) protocol that achieves "ultra-high resolution and much superior throughput" for characterizing previously intractable disordered proteins. This experimental breakthrough enabled the creation of a unique dataset, which in turn fueled their most significant finding: the Pepron foundational model. This AI model generates dynamic protein ensembles, moving beyond static snapshots to capture the "video" of protein behavior, a critical advancement for drug discovery. The project also validated the efficacy of distributed training on Kubernetes using Nvidia DGX Cloud for computationally intensive scientific models, proving that cloud-native infrastructure can directly accelerate complex scientific research.

Spotify's experience with Backstage yielded a powerful finding about the virtuous cycle of open-sourcing internal developer platforms. They discovered that by open-sourcing Backstage, they not only shared value with over 3,000 adopting companies but also significantly improved the product internally. The community's needs, such as easier plugin integration, prompted Spotify to undertake a year-long, un-roadmapmed rewrite of the entire backend system. This effort, done in collaboration with the community, resulted in "a better Backstage than we could have built on our own," validating their initial ambition for Backstage to become the industry standard for IDPs and eventually leading to a successful commercial SaaS offering, Spotify Portal.

Apple's findings centered on the successful architectural design and implementation of Private Cloud Compute (PCC) to extend the capabilities of Apple Intelligence while maintaining an unwavering commitment to user privacy. They demonstrated that open-source technologies like the Swift programming language and gRPC are fundamental to building such a privacy-centric cloud infrastructure. Swift's "low memory footprint," memory-safe protocol implementations, and ability to "shrink our attack surface" by minimizing unsafe code were identified as crucial for secure and efficient inference on Apple silicon servers. Similarly, gRPC's "high performance and resilience," particularly its bidirectional streaming and robust liveness probing, proved indispensable for secure and reliable communication of encrypted user data within PCC. This showcased that cutting-edge AI in the cloud can be achieved without compromising core privacy principles, by leveraging and contributing to the open-source ecosystem.

Technical Deep Dive

▶ Watch: Pepton's novel technique for resolving protein dynamics (4:30)

The keynote offered several in-depth technical discussions on how these organizations are implementing cloud-native solutions.

HSBC detailed its operational strategies for managing a vast Kubernetes estate. Their Service Hosting Platform handles 600 million hits daily across approximately a dozen clusters, supporting 7,000 production services. To mitigate the challenges of scale, they adopted workload sharding, distributing services across individual markets to effectively manage and reduce the blast radius of potential failures. For critical upgrades, they employ a blue-green configuration, rehydrating new clusters from backups and only cutting over once the new environment is proven stable. Cost management is a significant technical challenge, addressed by providing teams with tools to monitor and manage their financial footprint, encouraging elastic provisioning based on demand. Looking ahead, HSBC plans to evolve to a hot-hot cluster model for enhanced availability and resilience. A key architectural shift involves storing reference data from etcd outside of the cluster, a move designed to improve stability and disaster recovery. The adoption of a more declarative GitOps-based model is also on their roadmap for consistent and automated infrastructure management. Furthermore, they are actively working with Solo.io to integrate ambient mode Istio, projecting "hundreds of thousands of dollars" in annual cloud bill savings by optimizing service mesh operations.

Pepton's technical innovation revolves around tackling the complex problem of disordered proteins. These proteins' dynamic nature makes them difficult to study with traditional methods. Pepton developed a new experimental protocol for hydrogen deuterium exchange mass spectrometry (HDX-MS), achieving "ultra-high resolution and much superior throughput." This advanced technique generates the high-fidelity data necessary to train their proprietary Pepron foundational model. This AI model's purpose is to generate an ensemble of protein states, visualizing the protein's dynamic behavior over time, much like a video captures motion rather than a static image. A significant technical hurdle was ensuring the physical accuracy of these ensembles; as Carlo Physico noted, "a single atom misplaced will push the energy to infinity, making the ensemble useless." This necessitated a complete overhaul of their underlying AI architecture and the development of a "completely new ensemble evaluation metric." The training of Pepron, and indeed their "entire stack that does the analysis for the mass spectrometers," runs on Kubernetes. They leverage Nvidia DGX Cloud as their Kubernetes provider, valuing its "high performance networking, storage, distributed storage and G scheduling." Their distributed Python training scripts are containerized using Docker, simplifying the transition from smaller development environments to full-scale production clusters and utilizing RunAI for orchestration.

Spotify's technical journey with Backstage highlights the iterative development of an open-source platform. Initially a "very thin framework," Backstage has evolved significantly. A major undertaking was the complete rewrite of their backend system, a year-long project driven by the community's need for easier plugin integration. This rewrite simplified plugin development and integration, benefiting both external adopters and Spotify's internal teams. They are currently engaged in a similar rewrite of the frontend. This continuous evolution underscores the technical commitment required to maintain an open-source project while simultaneously building a commercial SaaS product, Spotify Portal, on top of it. The technical architecture is designed to support a vast ecosystem of plugins and integrations, addressing core developer experience issues like context switching, fragmentation, and cognitive load by providing a unified interface.

Apple's Private Cloud Compute (PCC) for Apple Intelligence showcases a sophisticated integration of open-source technologies with custom hardware for privacy-preserving AI. PCC scales computational capacity for complex generative models using servers equipped with Apple silicon, designed to deliver "the privacy and security of your iPhone from the silicon on up." The Swift programming language was chosen for its "security properties," "low memory footprint," and efficiency, enabling optimal use of hardware for inference within PCC. Crucially, Swift's memory-safe protocol implementations minimize "the amount of unsafe code parsing untrusted data," thereby "shrinking our attack surface." For the transportation layer, Apple heavily leverages gRPC, a "mature CNCF project" known for its "high performance and resilience." PCC utilizes gRPC's bidirectional streaming to communicate load information between the PCC gateway and individual Apple silicon servers. Privacy is paramount: response payloads are encrypted by the user device and securely routed via gRPC to the PCC gateway and then to the Apple silicon machines. gRPC's "robust liveness probing" further ensures the availability and integrity of these critical services. The technical stack includes gRPC, Swift, and Swift Protobuf, which is Swift's implementation for gRPC and Protobuf, highlighting a deep commitment to a robust, open-source-driven server ecosystem.

Demo / Proof of Concept

▶ Watch: Pepton's distributed training on Kubernetes with Nvidia DJX (6:00)

The keynote, while rich in technical detail and strategic insights, did not feature live, interactive demonstrations of all the technologies discussed.

For HSBC, the proof of concept lies in their successful operation of hundreds of Kubernetes clusters supporting 7,000 production services and 600 million daily hits. Their ongoing journey and ambitious roadmap, including the adoption of ambient mode Istio for projected cost savings, serve as a testament to their continuous innovation and real-world application of cloud-native principles at an immense scale.

Pepton used a compelling analogy rather than a live demo of their core technology. Carlo Physico showed a "text-to-video" model generating an avocado sitting on an armchair, with the core visibly empty. This visual was intended to illustrate the concept of an ensemble – how seeing a dynamic "video" provides more insight (e.g., the empty core) than a single static "frame." While the video itself encountered technical difficulties during the presentation, the conceptual demonstration effectively conveyed the core problem Pepton is solving: capturing the dynamic nature of disordered proteins. The ultimate proof of concept is Pepton's ability to create a new foundational model, Pepron, based on their novel HDX-MS technique, to accelerate drug discovery.

For Spotify, the Backstage project itself stands as a powerful proof of concept. The speaker highlighted its adoption by "over 3,000 companies" and daily use by "over 700 R&D squads" within Spotify. The tangible success of the platform in solving developer experience problems like context switching and cognitive load, coupled with the backend rewrite driven by community needs, demonstrates the platform's utility and the success of its open-source model. The subsequent launch of Spotify Portal, a commercial SaaS offering built on the open-source Backstage, further validates the project's impact and viability.

Apple's discussion of Private Cloud Compute (PCC) focused on its architectural design and the rationale behind its technology choices. No live demonstration of Apple Intelligence or PCC functionality was presented beyond the detailed explanation of how it extends on-device AI with cloud capabilities while upholding privacy through Apple silicon, Swift, and gRPC. The commitment to "privacy and security of your iPhone from the silicon on up" serves as the foundational proof for their approach.

Defensive Implications

▶ Watch: Avocado video demo and critical precision for proteins (7:00)

The diverse applications of cloud-native technologies discussed in the keynote carry significant defensive implications for security practitioners.

HSBC's large-scale Kubernetes deployment highlights several critical defensive strategies. Their practice of sharding workloads across individual markets directly contributes to blast radius management, limiting the potential impact of a security incident or failure to a specific segment. The implementation of blue-green cluster configurations for upgrades is a robust defensive measure, ensuring that new, potentially vulnerable, versions of the infrastructure are thoroughly validated before being cut over, thereby minimizing exposure to new attack vectors. Furthermore, their focus on providing "better guardrails for our tenants" and making "security and availability of our workloads becomes even easier to test, optimize, and maintain" indicates a proactive approach to embedding security into the platform's design and shared responsibility model. For highly regulated environments, the decision to maintain a centralized model despite industry trends also acts as a defensive strength, ensuring consistent application of "hundreds of IT controls" and compliance standards.

Pepton's work on foundational models for drug discovery underscores the importance of data integrity and precision in scientific computing. The insight that "a single atom misplaced will push the energy to infinity" for protein ensembles emphasizes the need for extremely robust validation mechanisms and anomaly detection in AI models handling critical scientific data. Running the "entire stack" on Kubernetes, including sensitive mass spectrometer analysis, necessitates stringent secure cluster configurations, network segmentation, and data encryption to protect invaluable research data from unauthorized access or tampering.

Spotify's Backstage offers indirect yet powerful defensive benefits. By addressing developer experience issues such as context switching, fragmentation, and cognitive load, a well-implemented Internal Developer Platform (IDP) can significantly improve an organization's security posture. When developers operate within a consistent, well-documented, and streamlined environment, they are less prone to making configuration errors that could introduce vulnerabilities. The standardization promoted by Backstage in how services are built, deployed, and managed inherently leads to a more consistent and therefore more secure infrastructure, making it easier to enforce security policies and conduct audits. The virtuous cycle of open-sourcing also suggests a defensive advantage, as "more eyes" from the community can contribute to identifying and fixing security flaws, leading to a more resilient product.

Apple's Private Cloud Compute (PCC) architecture for Apple Intelligence is a masterclass in privacy by design and attack surface reduction. The primary defensive implication is the architectural commitment to processing tasks locally on the device whenever possible, minimizing data exposure to the cloud. When cloud compute is necessary, PCC runs on Apple silicon servers specifically engineered for "privacy and security... from the silicon on up," indicating a deep hardware-level security integration. The choice of Swift programming language for its memory safety properties is a direct defensive measure, as it minimizes the risk of common vulnerabilities arising from "unsafe code parsing untrusted data," thereby "shrinking our attack surface." Furthermore, leveraging gRPC for its "high performance and resilience" in the transportation layer, coupled with the explicit mention of encrypted response payloads by the user device, ensures data confidentiality and integrity during transit. The robust liveness probing of gRPC also contributes to the continuous availability and health monitoring of these privacy-sensitive services, preventing potential service disruptions that could impact security.

Key Takeaways

• Enterprise-scale Kubernetes: Large organizations like HSBC successfully manage hundreds of clusters and thousands of services by implementing advanced strategies such as workload sharding, blue-green upgrades, externalizing critical data (e.g., etcd), and adopting GitOps for resilience and cost efficiency.
• Open Source as a Virtuous Cycle: Spotify's Backstage demonstrates how open-sourcing internal tools can lead to a superior product through community contributions, fostering a "virtuous cycle" of innovation that benefits both the originating company and a wide ecosystem of adopters.
• AI-Driven Scientific Discovery: Cutting-edge scientific research, such as Pepton's work on disordered proteins, is increasingly reliant on distributed AI models trained on cloud-native infrastructure, enabled by novel experimental techniques and specialized compute platforms like Nvidia DGX Cloud.
• Privacy-First Cloud AI: Apple's Private Cloud Compute exemplifies how privacy-preserving AI systems can be built at scale by strategically combining on-device processing with dedicated cloud compute on custom hardware, leveraging open-source technologies like Swift and gRPC for security, performance, and memory safety.
• Balancing Regulation and Innovation: Highly regulated environments require a nuanced approach to cloud-native adoption, often balancing industry trends (e.g., devolved tenancy) with organizational needs for centralized control, compliance, and robust guardrails to ensure security and accountability.
• Bridging the Cloud-Native Gap: The keynote collectively illustrates that bridging the gap between cloud-native innovation and real-world use involves practical, domain-specific adaptations, strategic contributions to the open-source ecosystem, and a relentless focus on solving complex challenges related to scale, cost, security, and user experience across diverse industries.

About the Speaker(s)

The keynote featured several distinguished speakers:

• Carlo Physico is the Chief Technology Officer at Pepton, a biotech company based in Switzerland. Pepton focuses on pioneering drug discovery for disordered proteins, a peculiar class of proteins involved in numerous cancers and neurodegenerative diseases. Carlo's work at Pepton involves developing innovative experimental techniques and foundational AI models to understand these complex proteins.
• Fabio, whose last name was not provided in the transcript, is the Head of Engineering at Pepton. He plays a crucial role in building the company's models on Kubernetes, leveraging distributed training on platforms like Nvidia DGX Cloud to accelerate their drug discovery efforts.
• Tyson Singer serves as the VP of Technology at Platform, a division of Spotify. He has been instrumental in the development and open-sourcing of Backstage, Spotify's internal developer portal. Tyson's insights highlight the challenges and rewards of fostering a thriving open-source community around a critical internal tool.
• Katie Gamanji is a Senior Engineer at Apple and a respected member of the Technical Oversight Committee (TOC) within the CNCF. At Apple, Katie is involved in developing advanced systems like Apple Intelligence and Private Cloud Compute, focusing on leveraging open-source technologies to build high-performance, privacy-preserving cloud infrastructure.