ai/nomic-embed-text-v2-moe repository overview

⁠nomic-embed-text-v2-moe

nomic-embed-text-v2-moe is a state-of-the-art multilingual text embedding model built on a Mixture of Experts (MoE) architecture. As the first general-purpose MoE text embedding model, it delivers exceptional multilingual retrieval performance while maintaining efficiency through sparse activation. With support for approximately 100 languages and training on over 1.6 billion text pairs, this model excels at semantic similarity tasks, retrieval-augmented generation (RAG), and cross-lingual information retrieval.

The model achieves competitive performance with embedding models twice its size while only activating 305M of its 475M total parameters during inference. This efficiency makes it particularly well-suited for production environments where both quality and resource constraints matter. The model also incorporates Matryoshka representation learning, allowing embedding dimensions to be truncated from 768 to 256 with minimal performance degradation, enabling up to 3x storage savings.

Developed by Nomic AI and fully open-sourced, the model includes released weights, training code, and evaluation data, making it ideal for researchers and practitioners who need transparency and reproducibility in their embedding pipelines.

⁠Characteristics

⁠Using this model with Docker Model Runner

docker model run nomic-embed-text-v2-moe

For more information, check out the Docker Model Runner docs⁠.

⁠Benchmarks

⁠Performance Comparison with Open-Weights Models

⁠Matryoshka Embeddings Performance

The model supports dimension truncation with minimal performance loss:

⁠Training Details

The model was trained using a comprehensive pipeline that includes:

• Training data: 1.6 billion high-quality text pairs across multiple languages
• Data quality: Consistency filtering to ensure high-quality training examples
• Training methodology: Two-stage approach with weakly-supervised contrastive pretraining followed by supervised finetuning
• Matryoshka learning: Enables flexible embedding dimensions from 768 to 256
• Architecture innovation: First application of sparse MoE to general-purpose text embeddings

⁠Links

• https://www.nomic.ai/blog/posts/nomic-embed-text-v2⁠
• https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe-GGUF⁠
• https://huggingface.co/papers/2502.07972⁠ (Technical Report)
• https://github.com/nomic-ai/contrastors⁠ (Training Code)

⁠Considerations

• Task prefixes required: The model requires task-specific prefixes for optimal performance. Use search_query: for queries and search_document: for documents
• Language variability: While supporting ~100 languages, performance may vary across different languages depending on training data distribution
• Resource requirements: Despite using sparse activation (305M active parameters), the MoE architecture may require more resources than traditional dense models of similar active parameter count
• Trust remote code: When loading this model, you must use trust_remote_code=True to access the custom architecture implementation
• Maximum context: Input text is limited to 512 tokens; longer texts will be truncated
• Matryoshka truncation: For storage-constrained applications, embeddings can be truncated to 256 dimensions with approximately 3x storage savings and minimal quality loss

⁠Generated by

This model card was automatically generated using cagent-action⁠. Want to learn more about Docker Model Runner? Check out the project repository: https://github.com/docker/model-runner⁠.