AWS Deep Learning Containers for vLLM 0.19.1+amzn2023.6ef1efd5 on EC2, ECS, EKS¶
AWS Deep Learning Containers for EC2, ECS, EKS are now available with vLLM 0.19.1+amzn2023.6ef1efd5.
Announcements¶
-
Initial release of vLLM Server containers on Amazon Linux 2023 for EC2, ECS, EKS
-
Simplified tag format: server-cuda[-vMAJOR[.MINOR[.PATCH]]]
-
Built on Amazon Linux 2023 with Python 3.12 and CUDA 12.9
Core Packages¶
| Package | Version |
|---|---|
| vLLM | 0.19.1+amzn2023.6ef1efd5 |
| PyTorch | 2.10.0 |
| TorchVision | 0.25.0 |
| TorchAudio | 2.10.0 |
| Transformers | 5.5.4 |
| CUDA | 12.9.1 |
| NCCL | 2.27.5 |
| FlashInfer | 0.6.7 |
| EFA | 1.47.0 |
Security Advisory¶
AWS recommends that customers monitor critical security updates in the AWS Security Bulletin.
Reference¶
Docker Image URIs¶
763104351884.dkr.ecr.us-west-2.amazonaws.com/vllm:server-cuda-v1
public.ecr.aws/deep-learning-containers/vllm:server-cuda-v1