aistaging/minicpm-v-4_5 repository overview

⁠MiniCPM-o 4.5

MiniCPM-o 4.5 is a state-of-the-art multimodal language model that delivers Gemini 2.5 Flash-level performance for vision, speech, and full-duplex multimodal live streaming capabilities. Built on an end-to-end architecture combining SigLip2, Whisper-medium, CosyVoice2, and Qwen3-8B, the model totals 9B parameters while surpassing widely used proprietary models like GPT-4o and Gemini 2.0 Pro in vision-language tasks.

The model excels across multiple modalities with leading visual understanding capabilities, achieving an average OpenCompass score of 77.6 across 8 popular benchmarks. It supports both instruct and thinking modes in a single model, offering flexibility for different performance and efficiency trade-offs. MiniCPM-o 4.5 features bilingual real-time speech conversation in English and Chinese with configurable voices, voice cloning, and role-play capabilities. A standout feature is its full-duplex multimodal live streaming, allowing the model to simultaneously process continuous video and audio input streams while generating concurrent text and speech outputs without mutual blocking.

The model demonstrates strong OCR capabilities with state-of-the-art performance for end-to-end English document parsing on OmniDocBench, surpassing specialized tools and proprietary models. It efficiently processes high-resolution images up to 1.8 million pixels and high-FPS videos up to 10fps in any aspect ratio, while supporting multilingual capabilities across more than 30 languages.

⁠Characteristics

⁠Using this model with Docker Model Runner

docker model run minicpm-v-4_5

For more information, check out the Docker Model Runner docs⁠.

⁠Benchmarks

⁠OpenCompass Visual Understanding (Instruct Mode)

⁠Video Understanding

⁠OmniDocBench (Document Parsing)

Note: Lower Edit scores are better; higher TEDS scores are better.

⁠Omni Simplex (Multimodal Understanding)

⁠Text Capabilities

⁠Vision Duplex (Full-Duplex Streaming)

⁠Links

• https://huggingface.co/openbmb/MiniCPM-o-4_5-gguf⁠
• GitHub Repository⁠
• CookBook⁠
• Streaming Demo⁠

⁠Considerations

• The model supports both instruct and thinking modes - instruct mode prioritizes efficiency while thinking mode focuses on enhanced performance for complex reasoning tasks
• Full-duplex multimodal live streaming capabilities require specialized inference frameworks (llama.cpp-omni recommended for local deployment)
• High-resolution image processing (up to 1.8M pixels) and high-FPS video processing (up to 10fps) may require significant computational resources
• Voice cloning and role-play features require reference audio clips for optimal performance
• The model's proactive interaction capabilities in live streaming scenarios are experimental and may require fine-tuning for specific use cases
• While the model supports 30+ languages for text and vision, speech capabilities are currently limited to English and Chinese

⁠Generated by

This model card was automatically generated using cagent-action⁠. Want to learn more about Docker Model Runner? Check out the project repository: https://github.com/docker/model-runner⁠.