Nvidia says its conversational AI can run in about 4GB of memory

Reachy Mini robot in Nvidia's demonstration, with the robot's camera view inset at upper left

Nvidia has released a conversational AI model designed to run in about 4GB of memory for its target workloads. The October 23 release, Nemotron 3.5 Bolt E4B 4GB, is intended for local assistants and interactive characters that need room to run alongside other software.

Nvidia’s announcement describes a model that can carry a conversation while calling connected tools. Its conversational setting encourages short responses suitable for speech, and developers can turn reasoning on or off to balance response time against the task.

The compact model handles text inside a larger application

The model card lists text inputs and text outputs. A voice assistant built around Bolt still needs components that hear the user and speak the response, while a visual application needs a way to supply information about what the model should see.

Nvidia illustrates that arrangement with a Reachy Mini robot running on Jetson Orin Nano 8GB hardware. Its example uses a separate Cosmos model for visual understanding, with Bolt calling animation scripts and supplying the robot’s responses.

Still from Nvidia’s demonstration, showing the robot and its camera view. Image: Nvidia

That makes the roughly 4GB figure a budget for the language model in specified workloads, rather than a guaranteed memory requirement for an entire voice or robotics application. The surrounding models and application still consume resources.

Choose the response mode for the job

Bolt’s model card provides separate controls for reasoning and conversational style. It also lists a 64,000-token context capacity, allowing an application to supply conversation history, instructions and information returned by tools.

These settings solve different problems. A short answer can sound natural when spoken aloud, while reasoning time helps determine how much computation happens before that answer. Enabling conversational style alone therefore does not establish the application’s response time.

In Nvidia’s BFCL-v3 test of selecting tools and supplying their inputs, the compact model scores 67.3%, against 70.6% for Qwen 3.5 4B at the same Q4_K_M precision. That result favors Qwen on this particular task, even though Nvidia’s broader comparison favors Bolt on combined quality and speed.

Nvidia scatter plot compares benchmark scores, time to first token and memory use across small models
Nvidia’s comparison uses an RTX 5090 and selected benchmark tasks; it is a vendor measurement rather than an SEW test. Image: Nvidia

The downloadable model is available for commercial and noncommercial use under the terms linked in its model card. The documentation includes local deployment instructions, rather than requiring every request to go through a hosted model service.

Source: Nvidia

The post Nvidia says its conversational AI can run in about 4GB of memory appeared first on Search Engine Watch.

Source:: searchenginewatch.com