Improving Mycroft through Metrics: The Mycroft Benchmark

Mycroft QA (Quality Assurance) and Testing

What do you guys think about exposing a dozen instances of Mycroft to the community to field ‘any’ query (text only) and log the results?

Something interactive like the percise and deep speech training pages at mycroft.ai.

The community could stress test Mycroft and submit desired results for a query that didn’t match expectations.

Croudsourcing the QA would polish the core skills quickly.

This approach would eliminate STT/TTS and would concentrate on accurate X Y Z results? It would also be separate from Persona.

It could also be used for the Mycroft Voice UI, currently there is no response for…
What can I do?
How do install skills?

Inspired from this thread Disappointing start: a lot of skills are not properly working

Thoughts?

1 Like