User Guide

Choose a framework to get started:

• vLLM — serve large language models on EC2, EKS, or Amazon SageMaker AI
• vLLM-Omni — serve multimodal models (TTS, image, video, audio, omni-chat)
• Ray — deploy any ML model with Ray Serve (NLP, vision, audio, tabular)
• PyTorch — distributed training with EFA, NCCL, flash-attn, and DeepSpeed pre-installed
• Base — lightweight CUDA + Python images for building your own AI/ML container