Nokkela Voice Assistant: STT-NLP-LLM-TTS web app for natural voice interaction
652
Nokkela Voice Assistant – or NVA for short – is a local web service that enables interactive communication in naturally comprehensible language. It is based on advanced OpenAI technology and incorporates three essential components: Speech-to-Text (STT), Natural Language Processing, Large Language Model (LLM) processing, and Text-to-Speech (TTS). With a simple web interface (Start/Stop button), users can record their spoken language and have it converted into a processed answer by the system. In version 1.0, the communication is handled exclusively via the OpenAI infrastructure, which requires a valid OpenAI API key.

The NVA provides two main modes:
Additionally, modern technologies such as Docker are employed to make the installation and operation of the system technically user-friendly and reproducible. A youth protection-friendly system prompt ensures engaging and secure interaction. The use of the “SnakeOil” certificate also enables encrypted communication at all times, which is particularly advantageous in shared networks (e.g., in workshops or training sessions).

Educational Institutions and Training Providers
Benefits:
Companies and Call Centers
Benefits:
Developers and System Integrators
Benefits:
Research Institutions and Start-ups
Benefits:
Public Institutions and Cultural Projects
Benefits:
The NVA demonstrates, as a local Docker-supported web service, a robust solution for integrating modern voice assistant technologies in a variety of environments. The clear division into Assistant and Translator modes enables flexible usage, while the use of OpenAI APIs guarantees project-specific performance and scalability.
NVA offers:
This solution thus opens up new opportunities for both established companies as well as innovative start-ups and educational institutions to leverage the benefits of artificial intelligence and modern voice technologies in their respective markets. Sharing is Caring - by using a SnakeOil certificate, a secure, low-overhead, and versatile approach (e.g., in workshops) is ensured, with all open-source components available under the GPLv3. With the NVA, every target group obtains a powerful tool that not only enables interaction with complex language models but also creates cross-industry added value by solving everyday challenges in communication and information exchange.
The NVA continuously provides the web service with SSL/TLS encryption – both for end users and for communication with the OpenAI infrastructure. Additionally, we recommend using supplementary load balancers, such as HAproxy or Traefik, as well as connecting Ollama servers via SSL/TLS connections to further optimize the security and performance of the solution.

Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh.
While the underlying model was trained on 98 languages, we only list the languages that exceeded <50% word error rate (WER) which is an industry standard benchmark for speech to text model accuracy. The model will return results for languages not listed above but the quality will be low.

1-4. Get OpenAI API Key;
2-4. Docker - Pull the Image
docker pull nokkela/nokkela-voice-assistant:latest
3-4. Docker - Run
docker run \
-d \
-p 20001:20001 \
-e OPENAI_KEY="YOUR-API-KEY" \
-e WEB_AUTH_USER="nva" \
-e WEB_AUTH_PW="nva" \
nokkela-voice-assistant
4-4. Enjoy
1-5. Get OpenAI API Key;
2-5. Clone the Repository
git clone https://github.com/NokkelaAI/nokkela-voice-assistant.git
3-5. Insert your OpenAI API Key into the "docker_build_and_run" script
sed -i 's/YOUR-API-KEY/sk-XXXXXXXXX/g' docker_build_and_run.sh
sed -i '.bak' 's/YOUR-API-KEY/sk-XXXXXXXXX/g' docker_build_and_run.sh
4-5. Run the script
./docker_build_and_run.sh
5-5. Enjoy

For debugging purposes or for future analyses, log files are automatically stored under /app/data. To access this data, a Docker volume mapping can be used.
docker run \
-d \
-p 20001:20001 \
-e OPENAI_KEY="YOUR-API-KEY" \
-e WEB_AUTH_USER="nva" \
-e WEB_AUTH_PW="nva" \
-e OLLAMA_CHAT_URL="http://YOUR-SERVER-IP:11434/v1/chat/completions" \
-v /data/nva:/app/data \
nokkela-voice-assistant
appuser@4cfe3eaa2f2f:/app/data$ cat voice.log
2025-04-07 19:05:59,526 - INFO - Audio received and processed in memory
2025-04-07 19:06:01,228 - INFO - User: Hello, nice to meet you...
2025-04-07 19:06:06,146 - INFO - Bot: Hello! Nice to meet you too. How...
2025-04-07 19:06:07,860 - INFO - TTS audio generated for text: Hello! Nice to meet you too. H...
appuser@4cfe3eaa2f2f:/app/data$

docker run \
--detach \
--publish 8000:8000 \
--name speaches \
--volume /opt/hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
--gpus=all \
-e WHISPER__TTL=-1 \
ghcr.io/speaches-ai/speaches:latest-cuda
docker run -it python:3.12 bash
apt-get update
apt-get install -y jq
pip install uv
git clone https://github.com/speaches-ai/speaches.git
cd speaches
uv venv
source .venv/bin/activate
uv sync --all-extras
export SPEACHES_BASE_URL="http://YOUR-SERVER-IP:8000"
uvx speaches-cli registry ls --task automatic-speech-recognition | jq '.data' > DATA
grep "whisper-large-v3" DATA
uvx speaches-cli model download Systran/faster-whisper-large-v3
uvx speaches-cli model ls --task automatic-speech-recognition | jq '.data' | grep "faster-whisper-large-v3"
exit
docker run \
-d \
-p 20001:20001 \
-e OPENAI_KEY="YOUR-API-KEY" \
-e OPENAI_TRANSCRIBE_URL="http://YOUR-SERVER-IP:8000/v1/audio/transcriptions" \
-e OLLAMA_CHAT_URL="http://YOUR-SERVER-IP:11434/v1/chat/completions" \
-e OPENAI_TTS_URL="Service 1 URL or Service 2 URL or Service 3 URL" \
-e TTS_HD="YES" \
-e TTS_VOICE="shimmer" \
-v /data/nva:/app/data \
nokkela-voice-assistant
http://YOUR-SERVER-IP:11434/v1/chat/completions
http://YOUR-SERVER-IP:11434/v1/chat/completions
http://YOUR-SERVER-IP:11434/v1/chat/completions
http://YOUR-SERVER-IP:11434/v1/chat/completions

#!/bin/sh
export HUGGING_FACE_HUB_TOKEN=XXXXXXXXX
export CUDA_VISIBLE_DEVICES="0,1"
docker run \
--gpus all \
--runtime=nvidia \
--shm-size=8g \
-p 11434:11434 \
--init \
-v /opt/ollama:/root/.ollama \
-e OLLAMA_KEEP_ALIVE=-1m \
ollama/ollama
# -v /etc/passwd:/etc/passwd:ro \
# -v /etc/group:/etc/group:ro \
# -u 1000:1000 \
# EOF
docker ps | egrep ollama
docker exec -it CONTAINERID bash
ollama run qwq
docker ps | egrep ollama
docker exec -it CONTAINERID bash
ollama run cogito:70b
http://host.docker.internal:11434/v1/chat/completions

ollama run cogito:8b
ollama run cogito:3b

docker run \
-d \
-p 20001:20001 \
-e OPENAI_KEY="YOUR-API-KEY" \
-e OLLAMA_CHAT_URL="http://YOUR-SERVER-IP:11434/v1/chat/completions" \
-e OPENAI_TTS_URL="Service 1 URL or Service 2 URL or Service 3 URL" \
-e TTS_HD="YES" \
-e TTS_VOICE="shimmer" \
-v /data/nva:/app/data \
nokkela-voice-assistant
git clone https://github.com/nokkela-ai/NPRF_4_openedai-speech
cp sample.env speech.env
TTS_HOME=voices
HF_HOME=voices
#PRELOAD_MODEL=xtts
PRELOAD_MODEL=xtts_v2.0.2
#EXTRA_ARGS=--log-level DEBUG --unload-timer 300
#USE_ROCM=1
docker compose up
-e OPENAI_TTS_URL="http://YOUR-SERVER-IP:8000/v1/audio/speech" \
root@9fff78efd3d2:/app/voices# grep "key" /app/voices/voices.json
apt-get update; apt-get install git-lfs
mkdir -p /opt/tts_onnx
cd /opt/tts_onnx
git clone https://huggingface.co/csukuangfj/vits-piper-de_DE-thorsten-high
docker cp /opt/tts_onnx/vits-piper-de_DE-thorsten-high/de_DE-thorsten-high.onnx.json CONTAINERID:/app/voices
docker cp /opt/tts_onnx/vits-piper-de_DE-thorsten-high/de_DE-thorsten-high.onnx CONTAINERID:/app/voices
docker exec CONTAINERID bash -c "apt-get update && apt-get install -y ed"
docker exec CONTAINERID bash -c "ed -s /app/config/voice_to_speaker.yaml << 'EOF'
/speaker: 163/
a
thorsten:
model: voices/de_DE-thorsten-high.onnx
speaker: # default speaker
.
w
EOF"
docker stop CONTAINERID
docker start CONTAINERID
... ... ...
server-1 | INFO: XXX.XXX.XXX.XXX:53808 - "POST /v1/audio/speech HTTP/1.1" 200 OK
The text length exceeds the character limit of 253 for language 'de', this might cause truncated audio.
... ... ...
English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Russian, Dutch, Czech, Arabic, Chinese (Simplified), Hungarian, Korean, Japanese, Hindi
docker run \
--detach \
--publish 8000:8000 \
--name speaches \
--volume /opt/hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
--gpus=all \
-e WHISPER__TTL=-1 \
ghcr.io/speaches-ai/speaches:latest-cuda
-e OPENAI_TTS_URL="http://YOUR-SERVER-IP:8000/v1/audio/speech" \
American English, British English, Japanese, Mandarin Chinese, Spanish, French, Hindi, Italian, Brazilian Portuguese

git clone https://github.com/nokkela-ai/NPRF_7_openai-edge-tts
cd openai-edge-tts
cp .env.example .env
API_KEY=your_api_key_here
PORT=5050
DEFAULT_VOICE=shimmer
DEFAULT_RESPONSE_FORMAT=mp3
DEFAULT_SPEED=1.0
DEFAULT_LANGUAGE=en-US
REQUIRE_API_KEY=False
REMOVE_FILTER=False
EXPAND_API=True
docker compose up --build
-e OPENAI_TTS_URL="http://YOUR-SERVER-IP:5050/v1/audio/speech" \
English, Spanish, Arabic, Korean, Thai, Vietnamese, Japanese, French, Portuguese, Indonesian, Hebrew, Italian, Dutch, Malay, Norwegian, Swedish, Greek, German, Afrikaans, Amharic, Azerbaijani, Bulgarian, Bengali, Bosnian, Catalan, Czech, Welsh, Danish, Estonian, Persian, Finnish, Irish, Galician, Gujarati, Hindi, Croatian, Hungarian, Icelandic, Javanese, Georgian, Kazakh, Khmer, Kannada, Lao, Lithuanian, Latvian, Macedonian, Malayalam, Mongolian, Marathi, Maltese, Burmese, Nepali, Polish, Pashto, Romanian, Russian, Sinhala, Slovak, Slovenian, Somali, Albanian, Serbian, Sundanese, Swahili, Tamil, Telugu, Turkish, Ukrainian, Urdu, Uzbek, Mandarin, Zulu

docker run \
-d \
-p 20001:20001 \
-e OPENAI_TRANSCRIBE_URL="http://YOUR-SERVER-IP:8000/v1/audio/transcriptions" \
-e OLLAMA_CHAT_URL="http://YOUR-SERVER-IP:11434/v1/chat/completions" \
-e OPENAI_TTS_URL="http://YOUR-SERVER-IP:8000/v1/audio/speech" \
-e TTS_HD="YES" \
-e TTS_VOICE="fable" \
-e WEB_AUTH_USER="nva" \
-e WEB_AUTH_PW="secret" \
-v /data/nva:/app/data \
nokkela-voice-assistant
docker run \
-d \
-p 20001:20001 \
-e OPENAI_TRANSCRIBE_URL="http://YOUR-SERVER-IP:8000/v1/audio/transcriptions" \
-e OLLAMA_CHAT_URL="http://YOUR-SERVER-IP:11434/v1/chat/completions" \
-e OPENAI_TTS_URL="http://YOUR-SERVER-IP:8000/v1/audio/speech" \
-e TTS_HD="NO" \
-e TTS_VOICE="thorsten" \
-e WEB_AUTH_USER="nva" \
-e WEB_AUTH_PW="secret" \
-v /data/nva:/app/data \
nokkela-voice-assistant
Content type
Image
Digest
sha256:716d8f536…
Size
124.4 MB
Last updated
about 1 year ago
docker pull nokkela/nokkela-voice-assistant:2.5.4