I’m not sure how much of this you plan on doing yourself, but Neon AI has a service for training custom models that we recently announced: 'BrainForge' Creates custom LLM Chatbot Experts - Neon AI 2024 2nd Quarter Newsletter .
In any case, I’m interested in seeing what kind of datasets you can find and what model(s) you come up with 
.
The STT/TTS part should be pretty easily implemented using Neon or OVOS which both have skills available for custom LLM integration.