I did a video about running everything locally on Jetson Orin Nano a bit more than a month ago so its definitely possible but not super easy at first mostly because JetPack 6.2 sucks and I added a layer of complexity with Kubernetes.
Since then I was able to run a newer version of Ollama which now allows me to run Gemma3, Qwen3, DeepSeek, etc… LLMs.
