aitwinkle/llama-3.2-3b-f1-reasoning-instruct-gguf repository overview

⁠Overview

🌏 Website⁠ | 🎮 Discord⁠ | 😺 GitHub⁠ | 🤗 Hugging Face⁠ | 🔗 LinkedIn⁠

This model is a GGUF-packaged Llama-3.2-3B F1 Reasoning Instruct model, fine-tuned with data emphasizing reasoning ability and instruction-following behavior, and published using Docker Model Runner.

The model is distributed as a Docker model image and can be pulled and executed directly via Docker, without manually managing GGUF files, runtimes, or llama.cpp builds.

⁠Requirements

• Docker Desktop (latest recommended)
• Docker Model Runner enabled
  • Docker Desktop → Settings → AI / Model Runner

Verify that Model Runner is running:

docker model status

⁠Pull the Model

Download the model from Docker Hub:

docker model pull aitwinkle/llama-3.2-3b-f1-reasoning-instruct-gguf:latest

Verify that the model is available locally:

docker model list

⁠Run the Model

Run the model with a single prompt:

docker model run aitwinkle/llama-3.2-3b-f1-reasoning-instruct-gguf:latest \
  "Explain step by step why reasoning-tuned models are useful."

The model will generate a response directly in your terminal.

⁠Interactive Mode

Start an interactive session:

docker model run -i aitwinkle/llama-3.2-3b-f1-reasoning-instruct-gguf:latest

Then enter prompts interactively:

> Solve the following logic problem and explain your reasoning.

Exit the session with Ctrl+D.

⁠Notes

• This model is packaged from a GGUF file and runs via Docker Model Runner.
• No manual model download, quantization handling, or runtime configuration is required.
• The published image currently uses Q4_K_M quantization.
• Inference performance depends on local hardware and Docker configuration.