Changelog¶
Changelog for the AWS Lambda GPU base images (base, cupy, pytorch, vllm, sglang).
v1.0¶
Tags: base:cuda · cupy:cuda · pytorch:cuda · vllm:cuda · sglang:cuda
Highlights¶
- Initial release of the Lambda GPU base image line, for GPU functions on Lambda Managed Instances
- CUDA 13.0.3 on Amazon Linux 2023, Python 3.13,
awslambdaric4.2.2 - Five images from one multi-stage Dockerfile: CUDA-only
base,cupy,pytorch, and thevllmandsglanginference servers - The vLLM and SGLang images include a default handler that serves an OpenAI-compatible API, so you set
MODEL_IDand deploy - A Lambda extension owns the engine on the vLLM and SGLang images: it loads the model during initialization, holds the environment out of
service until
/healthanswers, and recycles it if the engine dies or stops answering - FlashInfer kernels ship prebuilt for sm80, sm89 and sm120f in the vLLM image, removing the per-environment JIT build from cold start
- Weight streaming from Amazon S3 straight into GPU memory via the Run:ai Model Streamer (
*_LOAD_FORMAT=runai_streamer) ands3torchconnector, with no/tmpstaging - FFmpeg 8.0.1 built with NVENC and NVDEC in the
pytorchimage, for GPU transcoding without CUDA code - All caches redirected to
/tmp, the only writable path in a Lambda execution environment
Core Packages¶
| Package | Version |
|---|---|
| CUDA | 13.0.3 |
| Python | 3.13 |
| awslambdaric | 4.2.2 |
| PyTorch | 2.14.1 (cu130) |
| TorchVision | 0.29.1 |
| TorchAudio | 2.11.0 |
| Transformers | 5.17.0 |
| vLLM | 0.33.0 |
| SGLang | 0.5.23 |
| SGLang Kernel | 0.4.10 |
| FlashInfer | 0.7.2.post1 (vLLM) |
| CuPy | 14.0.1 (cupy-cuda13x) |
| s3torchconnector | 1.5.0 |
| FFmpeg | 8.0.1 |
| NumPy | 2.4.2 |