NVIDIA releases a 4GB Nemotron model for on-device voice assistants

NVIDIA published NVIDIA-Nemotron-3.5-Bolt-E4B-4GB on Hugging Face, a small language model further post-trained from Google's Gemma-4-E4B-it. The model has 8B total parameters with 4B effective parameters, is quantized to Q4_K_M precision, and supports up to 64K tokens of input and output context.

NVIDIA built it for low-memory, low-latency use cases such as local voice and device assistants, interactive personas, and embedded interfaces, with a focus on multi-turn tool use, instruction following, and grounding inputs. The model was trained using reinforcement learning with GRPO across instruction-following, tool-use, multi-turn conversation, and structured-output environments, using NVIDIA's NeMo RL and NeMo Gym frameworks. A companion drafter model, DFlash, is offered for speculative decoding to speed up inference.

NVIDIA's own benchmark tables, run under its NeMo Gym harness, show the quantized model scoring competitively against quantized Gemma-4-E4B-it and Qwen3.5-4B on tests like MultiChallenge, RoleBench, and ToolSandbox, though it trails on some others such as Tau2-Bench. The model runs through llama.cpp and is available on Hugging Face and Build.NVIDIA.com as of October 23.

Why it matters

A compact, open, locally runnable model aimed at edge devices gives developers a lighter-weight option for voice and embedded assistants without relying on cloud inference.

Source: NVIDIA on Hugging Face