Cloud PlatformsConfirmedAWS announces GPU sharing capabilities for Amazon ECS using NVIDIA Multi-Process Service (MPS), enabling multiple AI tasks to run on a single GPU, boosting utilization and cutting costs for various AI workloads.
On October 27, 2026, Amazon Elastic Container Service (ECS) announced support for GPU sharing using NVIDIA Multi-Process Service (MPS). This new capability allows multiple Amazon ECS tasks to run concurrently on a single NVIDIA GPU.
The primary objective of this feature is to increase GPU utilization and reduce infrastructure costs. It is specifically beneficial for workloads that do not fully consume a GPU's capacity, such as small-model inference, AV/ADAS simulations, embedding generation, and multi-model serving.
Engineers can implement GPU sharing without modifying their application code. The process involves defining the required GPU memory for each container and optionally setting a maximum share of GPU compute. Amazon ECS then handles the placement of tasks on GPUs with available capacity as services scale and deploy.
GPU sharing is supported on Amazon ECS Managed Instances and Amazon ECS on Amazon EC2. For Amazon ECS Managed Instances, the service includes monitoring of the MPS control daemon. If this daemon becomes unhealthy or stops, Amazon ECS automatically replaces the affected instance and migrates tasks to healthy capacity, ensuring workload continuity.
To begin using this feature, developers can utilize the AWS Management Console, AWS CLI, AWS SDKs, AWS CloudFormation, or other infrastructure-as-code tools. The setup involves creating a capacity provider with NVIDIA GPU instances, configuring the GPU sharing strategy and memory within the container definition of a task definition, and then creating a service on that capacity provider.
This capability is available across all AWS commercial and AWS GovCloud (US) Regions.
The introduction of GPU sharing on Amazon ECS with NVIDIA MPS represents a significant evolution in cloud resource management for compute-intensive workloads, particularly in artificial intelligence and machine learning. Historically, a common challenge with GPU-accelerated workloads in containerized environments has been underutilization of expensive GPU hardware. Many inference tasks, for instance, do not require the full processing power or memory of a dedicated GPU, leading to inefficient resource allocation and increased operational costs. The explicit support for NVIDIA MPS directly addresses this by enabling fine-grained resource partitioning.
NVIDIA MPS is a runtime service designed to allow multiple CUDA applications to run concurrently on a single GPU, reducing the overhead associated with context switching and improving overall throughput. By integrating MPS, Amazon ECS can now manage and orchestrate these shared GPU environments. The mechanism described, where developers define GPU memory per container and optionally a compute share, indicates that ECS is abstracting the underlying MPS configuration. This abstraction simplifies the deployment process for engineers, allowing them to focus on application requirements rather than low-level GPU management. The ability to specify memory and compute shares at the container level within a task definition provides a declarative approach to resource allocation, aligning with typical cloud-native deployment patterns.
From an engineering perspective, the practical consequences are substantial. For organizations deploying numerous small-to-medium sized AI models for inference, embedding generation, or multi-model serving, this feature offers a direct path to significant cost savings. Instead of provisioning an entire GPU instance for each model or task, multiple tasks can now share a single instance, amortizing the hardware cost across more workloads. This is particularly relevant for microservices architectures where individual services might perform specific, less demanding AI functions.
The support for AV/ADAS simulations also highlights the utility for workloads that might have bursty or intermittent GPU requirements, or those that can be broken down into smaller, parallelizable components. By allowing these components to share a GPU, the overall efficiency of simulation environments can improve, potentially accelerating development cycles and reducing the cost of extensive testing.
The integration with Amazon ECS Managed Instances, including the automatic monitoring and replacement of unhealthy MPS control daemons, is a critical operational benefit. This feature offloads the complexity of maintaining the underlying GPU sharing infrastructure from the user to AWS. For production environments, this automatic resilience mechanism helps ensure high availability for GPU-dependent workloads, minimizing downtime and reducing the operational burden on platform engineering teams. Without this, managing the health of the MPS daemon would be an additional responsibility for customers, potentially leading to manual interventions and service disruptions.
For developers, the workflow remains consistent with existing ECS practices, leveraging familiar tools like the AWS Management Console, CLI, SDKs, and CloudFormation. This consistency lowers the adoption barrier, as engineers can integrate GPU sharing into their existing infrastructure-as-code pipelines and deployment strategies without learning entirely new paradigms. The configuration within the task definition's container definition means that GPU resource allocation becomes an integral part of the application's deployment manifest, promoting reproducibility and version control.
While the announcement confirms the ability to define GPU memory and optionally a maximum share of GPU compute, specific details regarding the exact enforcement mechanisms for compute sharing, or how performance isolation is guaranteed under various load conditions, were not disclosed. Similarly, the announcement does not specify which particular NVIDIA GPU architectures or driver versions are required or recommended for this feature, beyond the general mention of NVIDIA GPUs. The scope of support is explicitly stated for Amazon ECS Managed Instances and Amazon ECS on Amazon EC2, indicating that Amazon ECS on AWS Fargate is not included in this initial release for GPU sharing capabilities.