abrarhasinpranto/medgemma-health-chat repository overview

⁠medgemma-health-chat (Q4_K_M)

A 4-bit quantized, GPU-ready build of a MedGemma health-chat fine-tune, packaged for Docker Model Runner⁠.

⁠Model at a glance

⁠Quick start

# Pull
docker model pull abrarhasinpranto/medgemma-health-chat:Q4_K_M

# Run a one-off prompt
docker model run abrarhasinpranto/medgemma-health-chat:Q4_K_M \
  "What are common symptoms of dehydration?"

# Or serve it as an OpenAI-compatible endpoint via Docker Model Runner
# and call it like any chat completion API on the exposed port.

⁠How this build was made

This image is the third generation of a source model — each stage below is a separate, auditable transformation:

┌────────────────────────────────┐
│  google/medgemma-1.5-4b-it      │   base model
│  (Gemma 3, multimodal, HAI-DEF  │
│   license)                      │
└────────────────┬─────────────────┘
                 │ community fine-tune
                 ▼
┌─────────────────────────────────────┐
│ bisonnetworking/medgemma-health-chat │   fp16 safetensors, ~8.6 GB
│ -merged                              │   (text + vision towers)
└────────────────┬───────────────────────┘
                 │ convert_hf_to_gguf.py
                 │ (text tower only)
                 ▼
┌─────────────────────────────────┐
│ medgemma-health-chat-f16.gguf    │   7.76 GB, F16
└────────────────┬──────────────────┘
                 │ llama-quantize → Q4_K_M
                 ▼
┌─────────────────────────────────┐
│ medgemma-health-chat-Q4_K_M.gguf │   2.31 GiB  ◄── this image
└────────────────┬──────────────────┘
                 │ docker model package --gguf
                 ▼
┌─────────────────────────────────┐
│   Docker Model OCI artifact      │
│   (llama.cpp backend, CUDA)      │
└──────────────────────────────────┘

Validation performed before publishing:

• Converted with llama.cpp's convert_hf_to_gguf.py (text-only extraction from the Gemma3 multimodal checkpoint).
• Quantized with the official llama-quantize CLI (Q4_K_M).
• Smoke-tested via llama-server on CPU with a direct chat-completion request — produced a coherent, on-topic clinical answer.
• Re-tested after packaging via docker model run on GPU (CUDA backend) — confirmed working end-to-end.

No additional fine-tuning, RLHF, or safety-tuning was performed by the packager beyond what the upstream fine-tune (bisonnetworking/medgemma-health-chat-merged) already applied. This repo only handles format conversion and quantization — it does not change model weights' learned behavior beyond the precision loss inherent to 4-bit quantization.

⁠Hardware requirements

Lowering --context-size at repackage time (e.g. to 8192) frees several hundred MB more if you need headroom to run another model concurrently (e.g. an ASR model in the same pipeline).

⁠Intended use & license

• The base model (MedGemma) is governed by Google's Health AI Developer Foundations (HAI-DEF) terms of use⁠ and the Gemma license⁠. MedGemma is explicitly described by Google as "intended to be used as a starting point... not intended to be used without appropriate validation, adaptation, and/or meaningful modification."
• This image is a derivative work: base model → community fine-tune → format conversion/quantization. It carries the same license obligations forward.
• Intended for: prototyping healthcare-adjacent chat assistants, education, and research on-device/on-prem, where data never has to leave your infrastructure.
• Not intended for: direct clinical decision-making, diagnosis, treatment planning, or any use without a licensed clinician in the loop and further validation for your specific use case.

⁠Attribution chain

1. google/medgemma-1.5-4b-it⁠ — Google, HAI-DEF license
2. bisonnetworking/medgemma-health-chat-merged⁠ — community fine-tune
3. This image — GGUF conversion + Q4_K_M quantization + Docker Model Runner packaging by @abrarhasinpranto⁠

If you build on top of this image, please preserve this attribution chain and the license terms above.