Lightning Talk: There Is a New Volume Type in Town! - Mario Loriedo, Red Hat

Lightning Talk: There Is a New Volume Type in Town!

Speakers: Mario Loriedo, Software Engineer, Red Hat

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=zXIMJeJrnvI

Overview

In a concise yet impactful lightning talk at KubeCon EU, Mario Loriedo, a software engineer on the Podman team at Red Hat, unveiled a groundbreaking new feature coming to Kubernetes: volumes of type image. This enhancement introduces the ability to specify and mount OCI (Open Container Initiative) images directly as volumes within Kubernetes pods, fundamentally changing how data, models, and auxiliary tools can be distributed and utilized within containerized environments.

This innovation addresses critical challenges, particularly in the burgeoning fields of Artificial Intelligence and Machine Learning, where large models need efficient and secure distribution. By leveraging the robust infrastructure of OCI images, this new volume type enables a clean separation between application logic and its associated data or specialized tools, fostering greater modularity, security, and operational efficiency. Initially introduced as an alpha feature in Kubernetes 1.31, it is slated for graduation to beta in Kubernetes 1.33, signaling its rapid adoption and the strong community interest driving its development.

The significance of this feature extends beyond just AI/ML; it opens up a versatile paradigm for distributing any form of data or executable artifact that benefits from the immutability, versioning, and distribution mechanisms inherent to OCI images. For developers and operators working with Kubernetes, volumes of type image represents a powerful new primitive that streamlines workflows, enhances debugging capabilities, and strengthens the overall supply chain security of containerized applications.

Background

▶ Watch: Introducing new Kubernetes volume type: "image" (0:00)

The Kubernetes ecosystem has long provided a rich array of volume types designed to manage data within pods, ranging from ephemeral storage like emptyDir to persistent options such as PersistentVolumeClaim (PVC), and configuration injection mechanisms like ConfigMap and Secret. Each of these serves distinct purposes, addressing the diverse needs of applications for data persistence, shared storage, or configuration management. However, a gap has historically existed for efficiently distributing and mounting application-specific data bundles, large models, or specialized debugging tools that are too substantial for ConfigMap or Secret and don't fit the typical use cases for persistent storage or host-path mounts.

Traditional approaches for incorporating such assets often involved bundling them directly into the main application container image. While straightforward, this method can lead to bloated images, reduced flexibility, and a tighter coupling between the application and its data/tools. Any update to the data or tools would necessitate rebuilding and redeploying the entire application image, increasing development cycles and resource consumption. Furthermore, hostPath volumes, while allowing access to node-local filesystems, introduce security risks and reduce workload portability across different nodes or clusters.

The problem became particularly acute with the rise of AI/ML workloads. Large language models (LLMs) and other complex AI models can easily span gigabytes or even terabytes. Distributing these models within application images is impractical and inefficient. The need for a standardized, secure, and performant mechanism to deliver these assets into a running container, separate from the core application logic, became a pressing concern. This led to the proposal and rapid development of volumes of type image, leveraging the existing and widely adopted OCI image specification to solve this distribution challenge, bringing its benefits of immutability, content-addressability, and robust registry distribution to a new class of data management within Kubernetes.

Key Findings

▶ Watch: Kubernetes "image" volume feature status: Alpha to Beta (1:35)

The central finding and contribution of this initiative is the introduction of a new volume type in Kubernetes, allowing users to reference an OCI image directly within a pod's volume definition. This innovative approach enables the dynamic mounting of an image's contents into a container's filesystem, providing a flexible and powerful mechanism for data and tool distribution.

Key findings and features highlighted by Mario Loriedo include:

• New Volume Specification: Kubernetes now supports a volume definition with an image field. This field takes a reference to an OCI image, such as image: myregistry/my-model:latest, allowing the contents of that image to be mounted into a container.
• Decoupled Distribution: This feature facilitates the separation of an application's core logic (residing in its primary container image) from its associated data, models, or auxiliary tools. These auxiliary assets can now be packaged and distributed as independent OCI images.
• Primary Use Cases:
• AI/ML Model Distribution: The most significant immediate driver is the efficient distribution of large AI/ML models (e.g., LLMs) as OCI images. These models can then be mounted into inference runtime containers, allowing for independent updates and versioning of models and runtimes.
• Debugging and Observability Tools: Specialized debugging or observability tools can be packaged as OCI images and mounted on demand into running application containers. This provides powerful in-container diagnostics without polluting the main application image.
• Kubernetes Adoption Timeline: The feature was introduced as an alpha feature in Kubernetes 1.31 and is slated to graduate to beta in Kubernetes 1.33, indicating rapid progress and strong community support.
• Runtime Support: This functionality is not limited to Kubernetes alone. Major container runtimes and engines, including CRI-O, containerd, Podman, and Docker, have already implemented support for volumes of type image, enabling local development and testing of this feature outside of a full Kubernetes cluster.
• Generic OCI Artifacts: Beyond standard container images, CRI-O and Podman are extending their support to include generic OCI artifacts. This means that the mounted image does not strictly need to be a runnable container image; it can be any OCI-compliant artifact, such as data bundles, configuration files, or policy documents, further broadening the applicability of this volume type.

These findings collectively demonstrate a significant evolution in how Kubernetes manages application dependencies and data, offering a more modular, secure, and efficient approach to deploying complex workloads.

Technical Deep Dive

▶ Watch: First use case: Distributing AI/LLM models as OCI images (2:00)

The core technical innovation behind volumes of type image lies in its ability to leverage the existing and robust OCI (Open Container Initiative) image specification and distribution mechanisms to serve as a data source for Kubernetes volumes. Instead of pulling an image to run a container, the runtime pulls an image to mount its filesystem contents.

At a high level, the mechanism works as follows:

When a pod is scheduled, and one of its volume definitions specifies image: <image-reference>, the container runtime responsible for creating the pod's containers performs an additional step. It interprets the image reference, pulls the specified OCI image from a configured registry (e.g., Docker Hub, Quay.io, private registries), and then extracts its filesystem layers. Instead of creating a new container process from this image, the runtime makes the extracted filesystem contents available as a volume, which is then mounted into one or more containers within the pod at a specified mount path.

Consider a typical Kubernetes Pod YAML definition. With the new volume type, the volumes array would include an entry structured similarly to this (reconstructed based on the speaker's description):

In this example, the app-container will have the contents of myregistry/my-model-image:v1.0 mounted at /app/models. This allows the application to access the model data as if it were part of its own filesystem, but the model itself is managed and updated independently as a separate OCI image.

Mario Loriedo presented two compelling use cases that illustrate the power of this new volume type:

1. AI/ML Model Distribution:

• Problem: Large Language Models (LLMs) are often very large and frequently updated. Bundling them with inference engines leads to bloated container images and tight coupling.
• Solution: Package the LLM itself as an OCI image. The speaker demonstrated an example where a small LM model was distributed this way.
• Implementation: An inference runtime, such as Ramalama (a tool developed by Red Hat for running LLMs locally or deploying to Kubernetes), runs in its own container. The LLM model image is mounted into the Ramalama pod. This separation means the Ramalama application container can be lightweight and stable, while different LLM models can be swapped in and out by simply changing the image.reference in the volume definition. This significantly streamlines model deployment and versioning.
• The speaker referenced a deployment YAML for this scenario, highlighting how the model image is mounted inside the Ramalama pod.

1. Debugging and Observability Tools:

• Problem: Debugging tools or specialized utilities might be needed temporarily inside a running container, but including them in the production application image adds unnecessary attack surface and size.
• Solution: Package debugging tools, such as the Open VS Code server, into a separate OCI image.
• Implementation: This tool image can then be mounted into any application container. The speaker showed an example (using Podman for local demonstration) where the open VS code server executable was mounted into a container. Once mounted, a user could start the VS Code server inside the container, connect to it via a web browser, and perform real-time debugging or modify files within the container's environment. This provides powerful, on-demand diagnostic capabilities without permanently altering the application's runtime image.

Crucially, the support for volumes of type image extends beyond Kubernetes. Container runtimes like CRI-O and containerd, which are foundational to Kubernetes, have implemented this feature. Furthermore, standalone container engines like Podman and Docker also support it, allowing developers to test and use this volume type locally before deploying to a cluster. The speaker specifically noted that CRI-O and Podman are going a step further by supporting generic OCI artifacts, meaning the mounted "image" doesn't strictly need to conform to the container image specification for runnable applications but can be any OCI-compliant bundle of data. This opens the door to even broader applications for distributing configuration, policies, or other non-executable assets.

Demo / Proof of Concept

▶ Watch: Second use case: Injecting debugging tools like VS Code (3:50)

Mario Loriedo presented two distinct proof-of-concept demonstrations to illustrate the practical applications of the new volumes of type image. While the talk was a lightning presentation, he provided enough detail to understand the mechanics and benefits.

1. LLM Model Deployment with Ramalama:

The first demonstration focused on the distribution of large AI models. The speaker presented a conceptual Kubernetes deployment manifest (though not shown in full detail, its structure was described) designed to serve an LLM model.

• Components: The demonstration involved a pod running Ramalama, a tool developed by Red Hat specifically for easily running LLM models in containers, both locally and on Kubernetes.
• Model Distribution: The core innovation was how the LLM model itself, specifically referred to as a "small LM model," was delivered. Instead of being bundled within the Ramalama container image, the model was packaged as a separate OCI image.
• Mounting Mechanism: This model image was then mounted into the Ramalama pod using the new volumes of type image feature. This setup allows the Ramalama inference engine to access the model files directly from the mounted volume, enabling the application to serve inferences.
• Benefits: This clearly showcased the advantage of decoupling the inference runtime from the model data. Model updates could be deployed by simply changing the image.reference in the volume definition, without requiring a rebuild or redeployment of the Ramalama application. The speaker noted that the source code for this example was available, implying a functional demonstration.

1. In-Container Debugging with Open VS Code Server:

The second demonstration highlighted the utility of volumes of type image for distributing auxiliary tools, specifically for debugging.

• Tool Packaging: In this scenario, the executable for the Open VS Code server was packaged into its own OCI image.
• Local Execution with Podman: To demonstrate this locally, the speaker used Podman, showcasing that this feature is not exclusive to Kubernetes clusters but also supported by standalone container engines. The command involved running a container and mounting the open VS code server image as a volume.
• Debugging Workflow: Once mounted, the open VS code server could be started inside the target container. This allows a developer to connect to the VS Code server via a web browser, gaining a powerful integrated development environment (IDE) directly within the container. From there, they could modify files, inspect processes, or debug applications running inside the container, providing an unprecedented level of insight and control for troubleshooting.
• Benefits: This setup eliminates the need to pre-install debugging tools into production container images, reducing image size and attack surface. Tools can be provisioned on-demand when debugging is required, and then easily removed, ensuring a clean and secure production environment.

Both demonstrations effectively conveyed the versatility and practical benefits of volumes of type image, illustrating how it can streamline workflows for AI/ML model management and enhance the debugging capabilities for containerized applications.

Defensive Implications

▶ Watch: Ecosystem support and generic OCI artifact capabilities (5:00)

The introduction of volumes of type image in Kubernetes has several significant defensive implications for organizations operating containerized workloads, primarily enhancing security, maintainability, and operational flexibility.

1. Enhanced Supply Chain Security: By packaging data, models, or tools as OCI images, organizations can leverage existing container image security practices. This includes signing images with tools like Notary or Cosign, scanning them for vulnerabilities using various image scanners (e.g., Trivy, Clair, Anchore), and enforcing policies for trusted registries. This extends the robust security model of container images to a broader range of application dependencies, ensuring that the data and tools mounted are verified and free from known vulnerabilities.
2. Improved Isolation and Reduced Attack Surface: Decoupling large data assets or debugging tools from the main application container image leads to smaller, more focused application images. Smaller images inherently have a reduced attack surface because they contain fewer dependencies and executables. Furthermore, sensitive tools (like debuggers) can be mounted only when needed, preventing their persistent presence in production environments and minimizing potential misuse.
3. Immutable and Versioned Data/Tooling: OCI images are inherently immutable and versioned. This provides a strong defensive posture by ensuring that the data or tools mounted into a container are precisely what was intended, preventing accidental or malicious modification. Organizations can rely on cryptographic hashes and tags to guarantee the integrity and specific version of any mounted image, facilitating rollbacks and forensic analysis if issues arise.
4. Granular Access Control: Standard Kubernetes Role-Based Access Control (RBAC) can be applied to control who can define pods with image volumes and which image registries they can pull from. Combined with ImagePullSecrets, this ensures that only authorized users and service accounts can access and mount specific OCI images, protecting sensitive models or proprietary tools.
5. Policy Enforcement: Admission controllers can be configured to enforce policies regarding the use of volumes of type image. For instance, policies could dictate that only images from approved internal registries are allowed, that images must be signed, or that certain vulnerability scanning thresholds must be met before an image can be used as a volume. This provides a powerful gatekeeping mechanism for all mounted assets.
6. Simplified Vulnerability Management: While volumes of type image provides a new vector for data, it also centralizes the management of its security. Instead of managing data security through disparate means (e.g., application-specific download mechanisms), all data delivered as OCI images can be subjected to the same rigorous vulnerability scanning and patch management processes as application container images.
7. Resource Management and Efficiency: While not strictly defensive, the efficient distribution of models and tools can reduce network bandwidth and storage requirements, as image layers can be cached and shared. This indirectly contributes to operational resilience by optimizing resource utilization and speeding up deployments.

In summary, volumes of type image provides a structured, secure, and auditable way to manage and deliver application-critical data and tools, extending the proven security benefits of the OCI ecosystem to a new dimension within Kubernetes. Defenders should integrate the scanning, signing, and policy enforcement of these new volume images into their existing container security pipelines.

Key Takeaways

• New Kubernetes Volume Type: Kubernetes introduces volumes of type image, allowing OCI images to be mounted directly as volumes within pods.
• Decouples Application and Data/Tools: This feature enables a clean separation between the application's main container image and large data assets (like AI/ML models) or auxiliary tools (like debuggers), fostering greater modularity and independent lifecycle management.
• Key Use Cases: Primarily driven by the need for efficient distribution of large AI/ML models (e.g., LLMs) and for on-demand provisioning of debugging or observability tools within containers.
• Rapid Adoption & Runtime Support: Introduced as an alpha feature in Kubernetes 1.31, graduating to beta in 1.33. Supported by major container runtimes and engines including CRI-O, containerd, Podman, and Docker.
• Generic OCI Artifacts: CRI-O and Podman extend support to generic OCI artifacts, meaning any OCI-compliant bundle (not just executable container images) can be mounted as a volume.
• Enhanced Security and Operations: Leverages OCI's immutability, versioning, and distribution mechanisms, improving supply chain security, reducing attack surface, and streamlining vulnerability management for data and tools.

About the Speaker(s)

Mario Loriedo is a Software Engineer at Red Hat. His work focuses on the Podman team, a testament to his expertise in container technologies and runtime environments. His contributions are instrumental in advancing features that bridge the gap between container engines and orchestration platforms like Kubernetes, as evidenced by his presentation on volumes of type image.