An offline voice assistant on a Raspberry Pi or Jetson

A small robot should be able to hear a sentence, answer it, and speak back without calling a cloud model. The offline voice assistant repository is that pipeline: voice activity detection, speech to text, a local language model, text to speech, and a latency log for each stage.

It is a demo. It does not ship weights, it does not drive servos, and it does not publish timing numbers for a Raspberry Pi or an NVIDIA Jetson. Those numbers belong to the board you measure. The README keeps the latency table blank on purpose. If you see a filled-in millisecond in that file, treat it as a mistake, not a result.

Python 3.11 or newer is enough to start. From a clone of the repo:

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

The console script is offline-voice. The default config path is config/demo.yaml, which uses fake speech, language, and voice backends. You can run the control flow, the tool calls, and the latency logger before you download anything.

What runs on each turn

Both modes cut audio with voice activity detection and run speech to text on a finished utterance. Streaming applies to the language model and to sentence-sized Piper calls. The faster-whisper adapter uses Whisper's segment API, so this demo does not claim streaming recognition.

The stages and the backends they accept:

Audio is 16 kHz in the YAML files, with 30 ms frames. config/demo.yaml uses energy VAD with a threshold of 0.02, 400 ms of silence to end an utterance, a 200 ms minimum speech length, and 120 ms of padding. The fake speech-to-text backend ignores what was said and returns the transcript written in the file: what time is it.

The language-model backends share one text protocol. A tool call is a single block:

<tool>{"name":"robot","arguments":{"command":"wave"}}</tool>

After the tool result is appended, the model is asked again for a short spoken answer. max_tool_rounds is 3 in the configs, so a stuck model cannot call tools forever. The fake model follows the same protocol, which is what the tests exercise.

Run the demo path

Pick exactly one of --text, --wav, or --mic. The CLI returns an error if you pass more than one, or none.

offline-voice run --config config/demo.yaml --text "what time is it"
offline-voice run --config config/demo.yaml --mode stream --text "what is the weather"
offline-voice latency --config config/demo.yaml --text "please wave"

--mode turn speaks once after the model finishes. --mode stream synthesizes each sentence as the model emits it. The flag overrides the mode key in YAML.

samples/utterance.wav is a synthetic 440 Hz tone between silence. It is not a person speaking. Energy VAD trims it, then the fake speech-to-text backend still returns the configured transcript. That is the point of the demo path: the audio stages run, and the words come from the fake backend.

offline-voice run --config config/demo.yaml --wav samples/utterance.wav --out out/reply.wav

--out writes the spoken WAV. --json prints one JSON object, including the timings this process measured. --latency-log overrides the JSONL path. An empty string skips the file. Each JSONL row has stage, duration_ms, backend, and measured_by set to script.

Microphone capture is optional. It records a fixed window, default 5 seconds, then runs the same pipeline. Install the extra and pass --mic:

pip install -e ".[mic]"
offline-voice run --config config/demo.yaml --mic --seconds 4 --out out/reply.wav

The mic extra pulls in sounddevice. Without it, the CLI tells you to install that extra instead of failing deep inside the audio stack.

The three tools

Tools stay on the device. None of them is a network client.

time reads the local clock and returns an ISO timestamp. weather is a stub. It records a location string, defaulting to the lab from the config, and states that no forecast was fetched. robot accepts wave, nod, stop, look_left, and look_right. It returns a queued-command line and does not move hardware. Any other command comes back as unknown, with the allowed list.

Turn tools off with tools.enabled: false in YAML if you want a spoken answer with no tool round.

Switch on real models

Install only the extras you need:

pip install -e ".[stt]"
pip install -e ".[llm]"
pip install -e ".[tts]"
pip install -e ".[vad]"
pip install -e ".[silero]"

stt is faster-whisper. llm is llama-cpp-python. tts is piper-tts, used when the piper binary is not already on PATH. vad is webrtcvad. silero pulls in PyTorch.

config/default.yaml points at faster-whisper tiny on CPU with compute_type: int8, a Qwen2.5 0.5B Instruct GGUF at models/qwen2.5-0.5b-instruct-q4_k_m.gguf, and the Piper voice models/en_US-lessac-medium.onnx. Paths are relative to the working directory, so run commands from the repository root. Context length is 2048, temperature is 0.2, and max_tokens is 256. VAD silence is a bit longer than the demo config, at 700 ms, with a 250 ms minimum speech length.

Preview the download plan without touching the network:

python scripts/download_models.py

Download the Piper voice and the GGUF when you are ready. The script checks SHA-256 against the Hugging Face LFS ids recorded in offline_voice.download.

python scripts/download_models.py --piper --llm

faster-whisper downloads its own weights on the first transcription. A small CPU starting point:

python -c "from faster_whisper import WhisperModel; WhisperModel('tiny', device='cpu', compute_type='int8')"

Then run a WAV through the real config:

offline-voice run --config config/default.yaml --wav path/to/utterance.wav --out out/reply.wav

The sample tone is still not a speech corpus. A recognizer may return an empty or useless transcript for it. Use a real recording when you want to judge recognition.

Ollama is the other language-model backend. It uses its own store:

ollama pull qwen2.5:0.5b
offline-voice run --config config/jetson.yaml --text "what time is it"

config/jetson.yaml expects Ollama on 127.0.0.1:11434 with a 120 second timeout. Point llm.backend at llama-cpp instead when you have a local GGUF.

Raspberry Pi

config/pi.yaml is a starting point for a 64-bit Raspberry Pi 4 or Pi 5. It is not a tuned profile.

Prefer energy or webrtcvad. The Pi file uses webrtcvad with aggressiveness 2. Silero needs PyTorch, which is a poor fit for a small Pi image. Stay with faster-whisper tiny or base, device: cpu, and compute_type: int8. A small GGUF, the 0.5B Q4 file from the download script or another model you choose, is the practical language model if the board has 4 GB of RAM. llama-cpp-python needs a compiler and CMake on the Pi. Piper's medium English voice is a reasonable CPU voice. The low-size Piper voices are smaller if medium is tight. Use a USB microphone and 16 kHz mono.

sudo apt-get install -y python3-venv python3-pip build-essential cmake
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[stt,llm,vad,mic]"
python scripts/download_models.py --piper --llm
offline-voice run --config config/pi.yaml --wav samples/utterance.wav

NVIDIA Jetson

config/jetson.yaml is a starting point for a Jetson in the Orin class on JetPack 6, not a tuned profile.

The file asks faster-whisper for the base model with device: cuda and compute_type: int8_float16. The PyPI faster-whisper and CTranslate2 wheel is a generic build. CUDA transcription needs a CTranslate2 build that matches that Jetson's CUDA. Until you have one, set stt.device back to cpu.

Ollama's ARM64 build is usually the shorter language-model path, which is why the Jetson config uses it. A CUDA build of llama.cpp is board-specific. Switch llm.backend to llama-cpp only after that build exists. Piper stays on CPU. The voice file is small next to the language model. Power mode changes results, so measure with offline-voice latency on the device you care about. If you already use NVIDIA's PyTorch and want Silero, a virtualenv created with --system-site-packages can see that torch. The demo does not require it.

What a latency run is, and is not

This repository does not publish measured latency for Whisper, llama.cpp, Ollama, Piper, a Raspberry Pi, or a Jetson. offline-voice latency prints only timings that process measured, and it labels every row measured by script together with the backend name. A run with config/demo.yaml measures the fake backends. Those durations are real measurements of the mock pipeline. They are not speech-model latency and they are not robot latency.

offline-voice latency --config config/demo.yaml --text "what time is it"
offline-voice latency --config config/default.yaml --wav samples/utterance.wav

Tests

Unit tests mock speech to text, the language model, and text to speech. They do not download weights. GitHub Actions runs Ruff and pytest on Ubuntu with Python 3.11.

ruff check .
pytest

config/ holds the demo, default, Pi, and Jetson YAML. src/offline_voice/ is the pipeline, the backends, the CLI, and the latency log.