ai/mistral-small4-safetensors repository overview

⁠Mistral Small 4

Mistral Small 4 is a powerful hybrid model capable of acting as both a general instruction model and a reasoning model. It unifies the capabilities of three different model families—Instruct, Reasoning (previously called Magistral), and Devstral—into a single, unified model. With 119 billion total parameters and 6.5 billion activated per token, this mixture-of-experts architecture delivers exceptional performance across a wide range of tasks.

With its multimodal capabilities, efficient architecture, and flexible mode switching, Mistral Small 4 is a versatile general-purpose model for any task. In a latency-optimized setup, it achieves a 40% reduction in end-to-end completion time, and in a throughput-optimized setup, it handles 3x more requests per second compared to Mistral Small 3. The model supports both text and image inputs with text outputs, making it suitable for document understanding, coding assistance, agentic workflows, and complex reasoning tasks.

The model features a toggle between fast instant reply mode and reasoning mode, allowing users to optimize for speed or depth depending on the task complexity. Its 256k context window, multilingual support for 24 languages, and native function calling capabilities make it ideal for enterprise applications, software engineering automation, research, and customization through fine-tuning.

⁠Characteristics

⁠Using this model with Docker Model Runner

docker model run mistral-small4-safetensors

For more information, check out the Docker Model Runner docs⁠.

⁠Benchmarks

⁠Internal Benchmarks

Mistral Small 4 supports per-request configuration of reasoning_effort:

• reasoning_effort="none": Fast, lightweight responses for everyday tasks, equivalent to Mistral Small 3.2 24B Instruct
• reasoning_effort="high": Deep, step-by-step reasoning for complex problems, with equivalent verbosity to previous Magistral models

⁠Reasoning Model Comparison

⁠External Benchmarks

Mistral Small 4 with reasoning achieves competitive scores, matching or surpassing GPT-OSS 120B across multiple benchmarks while generating significantly shorter outputs. The model demonstrates exceptional efficiency, particularly in reducing output length while maintaining or exceeding performance.

AA LCR (Accuracy):

Mistral Small 4 scores 0.72 with just 1.6K characters, whereas Qwen models require 3.5-4x more output (5.8-6.1K) for comparable performance.

LiveCodeBench:

Mistral Small 4 outperforms GPT-OSS 120B while producing 20% less output, reducing latency and inference costs.

AIME 2025:

This efficiency reduces latency, inference costs, and improves user experience across all benchmarks.

⁠Key Features

• Mixture of Experts (MoE): 128 experts with 4 active per token for efficient computation
• Reasoning Mode: Toggle between fast instant replies and deep reasoning with test-time compute
• Multimodal Vision: Analyzes images and provides insights based on visual content alongside text
• Multilingual: Native support for 24 languages across multiple language families
• System Prompt Adherence: Strong support for system prompts and instruction following
• Agentic Capabilities: Best-in-class performance with native function calling and JSON output
• Large Context Window: 256k token context length for extensive document processing
• Speed Optimized: 40% reduction in latency and 3x throughput improvement over previous versions
• Apache 2.0 License: Open-source license for commercial and non-commercial use

⁠Use Cases

Mistral Small 4 is designed for a wide range of applications:

• Software Engineering: Coding assistance, SWE automation, and codebase exploration
• Enterprise Applications: General chat assistants, document understanding, and data extraction
• Agentic Workflows: Multi-step task automation with function calling
• Research: Math problem solving and research assistance
• Document Analysis: Parsing and extraction from text and images
• Customization: Fine-tuning for specialized domain tasks

⁠Links

• https://mistral.ai/news/mistral-small-4⁠
• https://huggingface.co/mistralai/Mistral-Small-4-119B-2603⁠

⁠Considerations

• Reasoning Effort Settings: Use reasoning_effort="high" for complex tasks requiring step-by-step reasoning; use reasoning_effort="none" for faster responses on straightforward queries
• Temperature Recommendations: Temperature 0.7 is recommended for reasoning_effort="high", while 0.0-0.7 is suitable for reasoning_effort="none" depending on the task
• Hardware Requirements: With 119B parameters (6.5B active), the model requires significant GPU memory; tensor parallelism across multiple GPUs is recommended for optimal performance
• Quantization Options: For efficiency improvements, consider the FP8 checkpoint (included), NVFP4 quantization, or speculative decoding with the eagle head variant
• Context Window: While supporting 256k tokens, very long contexts may impact latency; consider chunking strategies for extremely long documents
• Production Deployment: For production workloads, vLLM is the recommended inference engine for optimal throughput and latency

⁠Generated by

This model card was automatically generated using cagent-action⁠. Want to learn more about Docker Model Runner? Check out the project repository: https://github.com/docker/model-runner⁠.