aistaging/smollm2-vllm repository overview

⁠SmolLM 2

Tiny LLM family built for speed, edge devices, and local development.

---

⁠Overview

An OCI model artifact for the SmolLM2 Instruct family (135M · 360M · 1.7B), packaged as Safetensors with a vLLM-oriented layout/config for convenient distribution via Docker Hub.

⁠Contents

• Model weights (*.safetensors)
• Associated configuration files (vLLM layout)

⁠Notes

• DMR support: Docker Model Runner does not yet support safetensors; support is planned soon.
• *-vllm images: These are provided today to support vLLM-compatible safetensor models. Their long-term role is still under discussion — options include continuing them as-is or exploring OCI index–based distribution (to offer --platform-like behavior). The approach may evolve depending on distribution standards and framework compatibility, with the goal of keeping things practical and developer-friendly.

⁠Sources:

• https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct⁠
• https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct⁠
• https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct⁠