Will OVOS or Neon Support Continued Dinkum Development? TL;DR No they won't

Archive - Mycroft Project
1

This is fantastic news! This brings up one big question for the OVOS project: Even if it needs to be a separate funding effort, what about OVOS starting an OVOS fork of Dinkum? The community needs to have much greater input into the direction of Dinkum than the company Mycroft AI is able to support right now. If OVOS were to take it on, it would not only help ensure the future of the Mycroft project (not to mention Mycroft.AI the company), it would allow the community to directly guide its future direction. @JarbasAl @jmillerv @clintonthegeek @postman @Dominik @synesthesiam @ChrisVeilleux @osheroff @rmblr

2

This raises the harsh but perhaps necessary question of how many customers would really miss Dinkum against a stable NeonOS or OVOS-Buildroot.

There is an ecosystem, several years old, of classic Mycroft skills and extensions. These remain broadly compatible with OVOS-based systems, and the same framework is portable, so the same skills work on Linux and your Mk2 and Plasma Bigscreen. Dinkum can’t do that.

So, if we did, would the size of the potential userbase justify the dev and support time?

3 Likes
3

I love that OVOS and NeonOS are so very community driven, committed to privacy, customer service and as you say are relatively stable – if not the code itself then certainly overall as dev platforms with infrastucture and a community of devs. However I have to concede that Mycroft Classic Core itself and therefore OVOS, and NeonOS were simply never designed for the non-technical, non-hobbyist customer, and the way they operate bears that out.

Furthermore if we objectively look at the state of the code itself, I believe we are forced to admit that Classic Core and the related projects are much less reliable and that Dinkum is the more solid, if less elaborate starting point to move into the future from. And this completely makes sense when we look at how and why Dinkum came about – after struggling with unreliability of the Classic Core code base for several years the decision was made to create a more solid robust refactored subset to build off of.

One of the main Dinkum developers, Michael Hansen explained it this way:

I started on Dinkum after half a year of working with the team at Mycroft to get Core 1 (“classic core”) stable enough to ship on the Mark II. This was after they had worked on it for over year by themselves without success.

Every week, we’d fix bugs only to find new ones cropping up elsewhere. This was holding up the shipping of the Mark II, so I took action and created Dinkum. Core 1 serves a lot of people in the Mycroft community well, but it is (in my opinion) a messy codebase that is not suitable for a device that (1) is always on, and (2) must be usable by non-developers.

Michael Lewis characterized Dinkum as a:

“retail” version of the software. This version does not have certain features that we found problematic to get right for the non-technical user."

Gez wrote:

We focused on reliability, which required reworking some of the skill API.

elsewhere he went on to say:

“We built Dinkum because we wanted to show what was possible with heavily refactored code, and provide the best possible direct user experience when people plug their Mark II in for the first time.”

and finally

Dinkum is simply far more stable. So for consumers that aren’t comfortable on the command line it provides the best experience as a voice assistant performing the core functions that we’ve seen people use most.

Although the Dinkum is currently lacking skills, an MSK (Mycroft Skill Kit for devs to more easily develop skills) or perhaps most importantly an MSM (Mycroft skill management kit which provides ability for a non-technical user to install new skills), all of these can be completed and at the base level Dinkum remains a better starting place for a device oriented towards non-technical users than Classic Core is.

Add to this that it is Dinkum that is being integrated with Grokotron (not Classic Core), and self-hosted Selene as well as it’s offline replacement. So when you put it all together it continues to make sense that Mycroft’s future for non-technical / non-hobbyist (ie. the wider market) is still Dinkum-based.

So it comes back to the realization that what we need now is for a free software community group like OVOS or Neon (or some other similar project) to take on Dinkum itself or a Dinkum fork.

4

This is MycroftAI’s recollection of how Dinkum came to be. It is a very rosy way of looking at a complicated, unpleasant story which I wouldn’t care to dig into just here.

2 Likes
5

I’ve just realized a clarification might be in order.

OVOS and NeonOS are not, in any sense, based on or related to the smart speaker image called Classic Core.

Mycroft is a voice assistant. Mycroft-core is the core software underneath that assistant. That’s what we forked, slowly, by accident at first, until the first-party software had been stale for so long we couldn’t support our own projects off of it anymore.

Our smart speaker operating systems are mostly original, entirely independent products, based around our fork of the voice assistant. OVOS-Buildroot predates both Dinkum and Classic Core, though you’re right that it was never presented as consumer software. We weren’t out to compete with MycroftAI. We are now out to become consumer software.

I can see why my answer might have been confusing or frustrating, if all that wasn’t clear.

Whereas Dinkum stripped Mycroft down and rebuilt it to suit the Mark II, our projects modularized Mycroft, way back in 2020-21. Supporting a different hardware target is usually a matter of configuration and testing, not rewriting code, and we have a whole ecosystem of features to draw from while we work toward a totally consumer-ready interface.

2 Likes
6

I don’t really want to talk about dinkum here, but i can share some insight on how “classic core” became a number of independent frameworks under OpenVoiceOS, you can mix and match frameworks for different use cases as needed, our reference images bundle everything

below copy pasted from a github issue

mycroft-core → OpenVoiceOS

one common issue with mycroft-core was that you could not run a piece of the stack without importing everything else, in a server you would be forced to run the audio stack even if you are just powering a chatbot, worse, in docker setups this dragged a lot of bloat

initially we just organized imports internally to clean this up, but eventually all the pieces ended up working fully standalone and were split into their own packages

  • mycroft.api + mycroft.identity + mycroft.metrics → ovos-backend-client
  • mycroft.skills.mycroft_skill → ovos-workshop
  • mycroft.util → ovos-utils
  • mycroft.lock → ovos_utils.process_utils
  • mycroft.util.combo_lock → combo_lock
  • mycroft.util.parse + mycroft.util.format → lingua-franca → ovos-lingua-franca
  • mycroft.client.text → ovos-cli-client
  • mycroft.client.speech → ovos-listener
  • mycroft.client.enclosure → ovos-PHAL + ovos-gui
  • mycroft.audio → ovos-audio-plugin-ocp + ovos-audio
  • mycroft.messagebus.client → mycroft-bus-client
  • mycroft.messagebus.service → ovos-messagebus
  • mycroft.session → ovos-bus-client.session
  • mycroft.configuration → ovos-config
  • mycroft.tts → ovos-tts-plugin-XXX + ovos-plugin-manager
  • mycroft.stt → ovos-stt-plugin-XXX + ovos-plugin-manager
  • mycroft.audio.services → ovos-audio-plugin-XXX + ovos-plugin-manager
  • mycroft.client.speech.hotword_factory → ovos-ww-plugin-XXX + ovos-plugin-manager
  • mycroft-skills-manager → ovos-skills-manager

State of Things

mycroft.deprecated

as ovos-core evolved we deprecated functionality present in mycroft-core, old classes providing either dummy behavior or warning logs has been moved to mycroft.deprecated module, this is dead code that remains in core only to avoid syntax errors in downstream projects, anything that was a public api in mycroft-core remains available in ovos-core

one of the removed things was the integration with msm which forced default skills and locked core to Mycroft marketplace

OpenVoiceOS implements a standalone skill installer utility that supports several marketplaces

mycroft-skills-manager → ovos-skills-manager

ovos-utils

at first ovos_utils provided generic utilities to interact with mycroft-core, over time more and more pieces from mycroft.util moved there as they started being needed across ovos ecosystem, ovos_utils is a shared library across most of our repositories and by itself is enough to write simple scripts to interact with OpenVoiceOS

currently ovos-core mostly contains unused utils under mycroft.util for legacy reasons, those should be fully deprecated or moved to ovos_utils

combo_lock was useful enough to become it’s own package GitHub - forslund/combo-lock

  • mycroft.util → ovos_utils
  • mycroft.util.combo_lock → combo_lock
  • mycroft.lock → ovos_utils.process_utils

ovos-plugin-manager

mycroft-core initially hardcoded STT/TTS/WW options, a plugin mechanism was developed but the hardcoded plugins were never migrated, OpenVoiceOS created plugins for every “native” implementation and moved all base classes and plugin loading logic to ovos-plugin-manager

  • mycroft.tts → ovos-tts-plugin-XXX + ovos-plugin-manager
  • mycroft.stt → ovos-stt-plugin-XXX + ovos-plugin-manager
  • mycroft.audio.services → ovos-audio-plugin-XXX + ovos-plugin-manager
  • mycroft.client.speech.hotword_factory → ovos-ww-plugin-XXX + ovos-plugin-manager

ovos-backend-client

as alternative backends / no backend support was added to ovos-core the mycroft.api module ended up migrated to a more generic ovos-backend-client, nowadays with Selene in risk of going down at any time this packages is the defacto client library for several OVOS backends, it can be used in any project to interact with several backends or directly with web apis

  • mycroft.api + mycroft.identity + mycroft.metrics → ovos-backend-client

ovos-workshop

ovos-workshop is where skill base classes live, it started as a way to bring bug fixes and features to mycroft-core via monkey patching, it evolved to provide skill base classes and a mechanism to develop standalone applications that interact with ovos-core

  • mycroft.skills.mycroft_skill → ovos-workshop

ovos-config

as configuration handling started being needed in more packages and stopped referring to a single file format or location it moved into it’s own shared library, it became the key component for XDG standards adoption and downstream voice assistant compatibility and continues to expand to this day with a cli interface recently added

once we fully move mycroft.conf from ovos-core to ovos-config it will be a truly standalone package

  • mycroft.configuration → ovos-config

mycroft.client

the mycroft.client was a target for removal from core since the early mycroft days, all components there lend themselves to replacement or can be considered fully optional

the debug cli utility built on curses was moved into ovos-cli-client, this component had a totally different release cycle from core and was the only one built on top of curses, some of the code in this module was among the code not touched for longer, this indicates this is a mature optional component

the mycroft.client.enclosure module was among the most controversial, as it directly tied to core to specific platforms recognized by MycroftOrg, it was very difficult to add platform support to core

today this module has been deprecated and the PHAL plugin framework is used across OpenVoiceOS instead

one of the things this module did was launch the ovos-gui service as part of the enclosure implementations, in OpenVoiceOS this was made it’s own service and is the source of truth for the mycroft-gui protocol implementation, it has been extended to support platform specific gui extensions

  • mycroft.client.text → ovos-cli-client
  • mycroft.client.enclosure → ovos-PHAL + mycroft.gui
  • mycroft.gui → ovos-gui

ovos-listener

the speech client was one of the pieces OpenVoiceOS improved the most, it supports multiple wake words, VAD plugins and listening modes. It is a standalone component since core can be used via text only, such as in server setups

  • mycroft.client.speech → mycroft.listener
  • mycroft.listener → ovos-listener

ovos-bus

mycroft started migrating the mycroft bus client implementation to it’s own repo, but never finished migrating mycroft-core to use it

the messagebus service is a simple websocket server, as proof of concept we developed an equivalent service in C++ to test performance, the reference python implementation is also available

  • mycroft.messagebus.service → ovos-messagebus / ovos-bus-server (c++)
  • mycroft.messagebus.client → mycroft-bus-client

ovos-bus-client

a central piece of ovos-core design is that the message.context is passed along the pipeline and can contain additional metadata, this is a key feature for supporting multiple users and things such as a different language per query

ovos-bus-client is an extension (not a replacement) of mycroft-bus-client, it adds utils to interact with bus messages and it’s metadata, the mycroft.session module has been extended and migrated to this package. this is the main design conflict between ovos-core and mycroft-dinkum, where in core the session data comes from bus messages, in dinkum this is controlled by the intent manager

  • mycroft.session → ovos-bus-client.session

ovos-audio

with a screen come video capabilities, the OCP framework was created to handle all kinds of playback, it uses the old mycroft audio service internally and replaces it and the mycroft common play framework

  • mycroft.audio → ovos-audio-plugin-ocp + ovos-audio

ovos-core

ovos-core is now a minimal library providing the brains of OpenVoiceOS, it is the piece that understand utterances and triggers intents, it can load skills or interface with external applications

it only depends on the messagebus which can be replaced with FakeBus for test frameworks or even with something like HiveMind. OpenVoiceOS is now several independent frameworks

5 Likes
7

Thank you both for so much more clarity re: OVOS (and Classic Core). That is extremely helpful. For the hobbyist/tinkerer customer, the OVOS and NeonOS options definitely give a route forward. Great! It’s very exciting to see OVOS take the steps necessary to operate as a non-profit and thereby firm up Mycroft’s OVOS-based classic/original generation.

However, although I’m enjoying the experience of NeonOS (I haven’t tried OVOS yet) and see NeonOS (OVOS?) features things that I’d love to see added to Dinkum, I just don’t see the NeonOS interface and robustness being all that close to being “retail ready.” In contrast it seems to me Dinkum’s reliability and looks and feel are much closer to being retail ready. Obviously Dinkum needs skills, MSK, MSM and infrastructure to be brought up to speed, but in general Dinkum appears to me to be much further along towards working for non-technical customers (ie. by far the vast majority of the world).

So my question remains the same as when I started my inquiry. Will OVOS step up to the task of bringing Mycroft into the retail ready future by adopting Dinkum (or a Dinkum fork)?

8

I was trying to avoid putting this so directly, because several extremely talented programmers whom I respect a great deal were compelled by circumstance to work very hard on Dinkum.

But it seems I’ll have to say it.

Dinkum feels more “retail ready” to you because it is Potemkin software.

2 Likes
9

I appreciate your desire to communicate with respect but still put across your point. Similarly I think its best (especially under the present precarious circumstances with Mycroft.AI) to speak directly while trying to our best to also maintain respectful communications, so I appreciate you saying what you think and not beating around the bush.

Similar to you, I also don’t wish to be ungrateful or unpleasant towards volunteers and community members who have done so much hard work over the years. However my experience with NeonOS (and presumably OVOS which underles it) is that it’s a great amateur effort that (please excuse me for saying so) doesn’t come off as that professional, good looking or robust. Dinkum looks and feels much more together and is less glitchy in my estimation.

I think I understand why you think Dinkum is “potemkin software” (ie. mostly a facade with little development below the surface), but even if I had intimate knowledge of both designs and codebases (which I don’t), I still don’t think I would agree with you. Sometimes it’s actually an advantage to start at a simpler place, and I think this may be one of those situations. Some of the architectural choices and subsequent build out that have yet to be completed with Dinkum may actually prove the wisdom of avoiding the sunk cost fallacy issue which the years of prior investment in OVOS / NeonOS may experience. Backward compatibility is always a heavy load to bear.

I hope that enough OVOS folks can be open-minded enough see this point, not fall into the “not invented here” frame and look past the bitterness of past Mycroft.AI mistakes and slights, that OVOS will adopt (or fork) Dinkum.

1 Like
10

I second the motion.

What is the advantage of having two code bases? If the answer is “Dinkum will be more stable in the long run”, I would say to build that stability into one code base.

I know the disadvantage is that there would be two sets of code that require care and feeding.

Less is more.

Just my two cents…

-Mike Mac
3 Likes
11

HUH? How many questions can it answer?

-Mike Mac
14

Dinkum was created in 3 months or so and started by removing everything the mk2 didn’t need, ovos consists of several different frameworks years in the making, multi platform support has been one of our core guiding principles since the start

Ovos depends on some of the frameworks that were removed or crippled in the making of dinkum, we also have disagreements about the new “sessions” feature of dinkum and don’t think it’s a good idea, in fact it goes directly against some fundamental design choices of core.

I can affirm categorically that OVOS will never fork or maintain dinkum.

I do not intend to criticize dinkum in this post, but want to make clear what our stance and roadmap is, dinkum simply is not in our roadmap or something we care about

2 Likes
15

I also don’t think the nature of the OVOS Assistant is entirely clear to the public. Certainly it’s more modular than Dinkum, so we’ve failed to communicate at least one thing.

Much of this is our fault, because we haven’t made very many cohesive announcements, but we’ve been waiting on MycroftAI to do that. After they follow up on Michael’s last announcement, then we’ll start carrying on about the technical details, was the idea. That was a month ago.

If the perception has been spread in other channels that OVOS’ backward compatibility represents tech debt, or retention of old code, I would like to emphatically and permanently reject that characterization. Our original mandate, as a dependency for several of Mycroft’s downstream partners, was to remain code-compatible with MycroftAI’s Assistant, not with the entire history of its codebase.

We permanently diverged quite a while ago, as illustrated above. We’ve been deprecating and replacing Mycroft’s limbs, one thing at a time, for years. The guts are probably next. The difference will be that our future overhauls will take place in public, with input and help from our community and from projects that depend on our code. Backwards incompatible changes will never be a surprise.

What we’re trying to set up around ourselves is a reincarnation of the open source community that used to exist around this code.

We have two codebases to choose from as a “starting” point. Both are modular. One is feature-rich, easily extended or reworked, and has always been cross-platform. It also supports historical extensions, while adding a whole new world of extensions using an additional framework, and it deprecates Old Stuff slowly and gently. Its modular nature made it an easy migration for other Mycroft-based projects, and now it’s the basis for all the survivors.

The choice is clear.

I apologize if I seemed confrontational above. It’s frustrating to be having this conversation in 2023. Dinkum and the Mk2 OS represent Mycroft’s second self-fork, and it’s been a very difficult road. Jobs and friendships have been lost over decisions which were forcefully argued, and nobody is happy to have been right. Just frustrated.

5 Likes
16

The proof is in (the eating of) the pudding. My two cents: don’t announce with words like “Hey! OVOS and Neon can do this”. Rather, announce with videos showing the code actually doing ‘this’. I’m hoping to do a music-playing skill video (soon, Any Day Now :})

Being on the OVOS Matrix channel https://app.element.io/#/room/#openvoiceos:matrix.org, has inspired me again. There’s a good vibe over there - people are helpful - and it’s like we know we’re on the cusp of something huge.

Hope this helps.

-Mike Mac
3 Likes
17

Neon AI has agreed to prioritize the end-user OS side. It will be more efficient for us (Neon AI) to add the “retail-ready” level of polish onto the Neon personal assistant, which we’re working on. The backend represents far more work than the front, and unfortunately Dinkum is not compatible to simply port over their more polished front.

20

Since this was split in it’s own thread I want to take the chance to elaborate a little on what stuff we salvaged from dinkum, while we have no interest in dinkum core we did go through the code and ported some things over, some of the excellent work by @synesthesiam lives in OVOS

before dinkum was dinkum, while still the mk2/qa branch, we kept track of every commit and ported and improved any good changes, some things we attempted to give feedback and help (which was not very welcomed), but mostly we just marched on and ported things over to ovos-core as soon as they were public. At this point we still fully expected these would make it into classic mycroft-core repo. Some improvements we got in this period were the resource file handling refactor and VAD support

In some other instances such as precise-lite we were actually ahead of mycroft.

dinkum was then created behind closed doors and we found about it at the same time everyone else in the community via the surprise announcement, after going through dinkum code we decided we do not like the architectural changes, what makes dinkum dinkum is the reworking of the intent/skill handling, where most breaking changes were introduced and where the new “session” was introduced. This is the piece of dinkum we dislike the most and where nothing can be salvaged

the voice loop for dinkum however was much more responsive, and since ovos is VERY modular, I introduce to you an updated dinkum listener! with very little of dinkum left but nevertheless a huge improvement over the ovos-listener

no dinkum skills are worth salvaging
the audio handling fundamentally conflicts with OCP framework
the HAL approach only abstracts the mk2 hardware not all hardware like ovos PHAL

at this point I think we took everything we liked out of dinkum already and there is nothing left to salvage

8 Likes