Brainstorming a design for music boxes in perhaps a living room, an office, a kitchen and 3 bedrooms:
All boxes have:
Superior craftsmanship and fidelity
Quality microphones
A personal voice assistant on an SoC
The box in the living room (“the hub”):
Has a powerful SoC (e.g. Nvidia Jetson Nano)
Is on WiFi and connected to the Internet
Runs a STT server locally (e.g. whisper)
Runs an AI-bot locally for “fallback” questions (best one to start with?)
Can get AI-bot answers from the Internet
Has music library in local file system
Can get music and news from the Internet
The other 5 boxes:
Have less powerful SoCs (e.g. RasPi 5)
Are on WiFi but can only connect to the hub (iptables)
Do STT from the hub (“remote”)
Do TTS locally (e.g. piper)
Get “fallback” answers from the Hub
Mounts the music file system from the hub
Can stream music/news w/hub as a proxy
Privacy/security advantages:
Only the hub faces the Internet
No speech leaves the building
AI-bots are done locally unless asked to use Internet
Music/news can be done locally and remotely
Has anyone got an AI-bot running locally? Any suggestions?
Hey @mike99mac
Looks like a HiveMind scenario to me.
I did a video about running everything locally on Jetson Orin Nano a bit more than a month ago so its definitely possible but not super easy at first mostly because JetPack 6.2 sucks and I added a layer of complexity with Kubernetes.
Hello the community!
In this video, we demonstrate Open Voice OS running a fully offline AI voice assistant stack on the ultra-compact NVIDIA Jetson Orin Nano, orchestrated with K3s and managed by Rancher.
Get a glimpse of a truly modern, edge-native AI assistant powered entirely on-device, featuring:
Local LLM (Large Language Model) for smart, private conversations
High-quality Text-to-Speech (TTS) with natural voice generation
Fast and accurate …
Since then I was able to run a newer version of Ollama which now allows me to run Gemma3, Qwen3, DeepSeek, etc… LLMs.
2 Likes
Nice work! You’re way ahead of me. I am enjoying learning. Yes, I agree with you on JetPack. The humor skill could use a little work - maybe we can find a stand-up comedian willing to do FOSS.
1 Like
It is not a skill but the local AI making a joke
1 Like