In response to the growing demand for GPUs, especially for tasks like running large language models, Amazon Web Services (AWS) has launched Amazon Elastic Compute Cloud (EC2) Capacity Blocks for ML. This innovative offering allows customers to reserve GPU instances for defined timeframes, addressing the challenge of costly and often scarce GPU resources.
With EC2 Capacity Blocks for ML, users gain access to NVIDIA H100 Tensor Core GPU instances, and they can select cluster sizes ranging from one to 64 instances, each equipped with 8 GPUs. What sets this solution apart is the ability to reserve these instances for specific durations. Customers can plan ahead by reserving GPU capacity for up to 14 days in 1-day increments, with the option to secure resources up to 8 weeks in advance. Once the reserved timeframe elapses, the instances are automatically deactivated.
From the user’s perspective, this system offers a high degree of predictability. They can precisely estimate how long their job will run, how many GPUs will be used, and the associated costs before initiating the task. This transparency empowers users to make informed decisions based on their resource needs and budgets.
For AWS, Capacity Blocks represent an opportunity to efficiently allocate their valuable GPU resources in a manner that is almost akin to an auction. The supply and demand dynamics of these resources will determine their pricing, ensuring that AWS maximizes the utilization of these in-demand GPUs while maintaining revenue streams.
As users sign up for the service, they are presented with the total cost for the selected timeframe and resources. This transparency allows users to adjust their resource allocation based on their specific needs and budget constraints before finalizing their purchase.
By introducing EC2 Capacity Blocks for ML, AWS is providing a solution that not only simplifies the scheduling of GPU resources but also aligns with the requirements of customers who frequently need access to GPUs for AI-related workloads. This service contributes to greater cost predictability and efficient resource allocation, making GPU-based tasks more accessible and manageable for a broader range of customers.
AWS is rolling out this feature, which is now generally available, starting in the AWS US East (Ohio) region. This marks a significant step toward making high-demand GPU resources more accessible and cost-effective for AWS users while enabling more effective resource planning and management.
