A skill for those users who want to rapidly prototype a new skill using graphical building blocks in the Mycroft web based configuration / settings page, rather than cli development - this would rapidly broaden the skillsbase - perhaps even a moderated “Skills-store” rather than gitbhub ?
Similar to ‘scratch’ to drag n’ drop and build skills online.
What third party services, data sets or platforms will the Skill interact with?
There could be a pre-defined set of services available from drop-downs or text entry.
Are there similar Mycroft Skills already?
There are the github templates, but this can be off-putting for those who are new to coding.
What will the user Speak to trigger the Skill?
Whatever the user defines, to activate the new skill they are creating.
What phrases will Mycroft Speak?
Defined by the user creating the base skill.
What Skill Settings will this Skill need to store?
Defined by the user creating the base skill.
Other comments?
Maybe use the pairing key / uid to lock the dev skill to your Mycroft until released and approved in the skill-store >
As a follow-up to this, I’ve been building a new Robot (actually a fully functioning Wall-E for my daughter - a surprise for her so shhh), and as Mycroft struggles on a Pi-Zero, I tried out Watson.
Based on the TJBot, it needed a bit of hacking for the zero, but it works well - the best bit being the Watson Assistant Website - which is very similar to my suggestion above, and the point of this follow-up.
Basically it’s a point n’ click, type in what you want to be recognised and enter the responses or further dialog, not a scrap of coding required. This would be a godsend for us Mycroft fans - any chance of something similar in the future ?
We’ve had a lot of discussion about a visual Skills builder for Mycroft - the idea has very strong internal support. The challenge is the technical approach and implementation. It would probably need some sort of modular approach, where functions are exposed to the visual Skills builder. How do we expose several Python functions visually? We could perhaps get to a very basic building-block level, but how complex could we get?
I’m not sure on the technical aspects of such a proposal… But a WYSIWYG skills creator would - in my opinion - be essential for the eventual mass adoption of Mycroft.
The Everyday Joe/Everyday Sally doesn’t want to learn to code, they want to get their “smart assistant”, drag-and-drop to create their own skills on their computer/smartphone/phablet/tablet and then go on their merry way… One of the “big players” (Google or Amazon if I remember correctly) recently introduced something similar, and I think Mycroft will miss a big piece of the pie if it does not follow the same path.
Yes it’s going to be limited in what it can do, but at the end of the day, it’s no different to a WYSIWYG website builder - you can do a lot of what you want to achieve without learning coding, but if there’s something you can’t do with a WYSIWYG wizard, then you’re just going to need to suck it up and learn to code…
I agree 100%. When I want to create a prototype of a voice interaction / skill I would use Adobe XD where I can draw a flowchart and add voice interactions to it, that are operated by Amazon Alexa. I would love to use open source for designing and for voice interaction.
I am keen to help design / build such a WYSIWYG builder for mycroft. How can we start?
Seems like double work to me. Right now, a skill is the integration of an API or important function into mycroft.
Visual Programming would require to the same thing again, but this time the “skills” would be building blocks for a Visual Language.
That would double the amount of work required to integrate an API.
I have been working in chatterbox, a mycroft based build it yourself assistant for kids, we have been testing in schools with great feedback! if a kid can use it, anyone from this community can also do it
We use a block based approach to build skills, our core is a fork of mycroft with more features, this means not every skill built with chatterbox will work on mycroft, but all mycroft skills will work on chatterbox, we are exploring how to better handle this
you will be making your own skills, in minutes! while possible to download skills, the main idea is that you build everything yourself, we are working hard to make that super easy! this means skills will support your language by default, you just write them in that language
on the interface side we are also keeping language support in mind, all blocks and interface will support multiple languages
development is being made in english, but as a portuguese speaker i’m making sure it also supports my own language, and therefore any other language
like with mycroft, the big hurdle will be the parsing/formatting functions that are language specific, but it is safe to assume that everything mycroft supports, so do we
It’s a great idea @kalirpi3. I’ve been the biggest proponent of building a visual Skill Design Tool but development time is precious.
Last year, I made a couple of minimal mockups (with no descriptions and help text) https://www.figma.com/file/VVBd7hP42xiFSsyUQpFieGVv/VUI-skill-design-tool?node-id=0%3A1, zoom out and then go for the middle of the artboard. To simplify things, from the development perspective, I used a form style instead of drag and drop. To a user, who is unfamiliar with how Mycroft works, it may be unclear how to use the tool at first but that gets fixed with good explanatory text and examples or a video.
The concept is based on creating intents and assigning dialogs (user, mycroft) to each intent. At stage 1, the tool would just generate simple Q&A skills, later skills with variables, and after that skills that use APIs.
Let me know folks if you were interested in building such tool or a similar one, while keeping it simple so it can be developed fast and shipped to actual users. I do design at Mycroft but have no problem with HTML and CSS.
I personally would like to see this as well.
To me it makes sense to have a form for name, description, and activation words and which words to turn to vars. (Optional settings page, for user defined settings)
List of actions(that could be filtered by model, apis, etc), ability to chain actions and redirect outputs or stop to ask for more input(which is really just another action).
Generates python code, that could be downloaded or uploaded to github or installed to your mycroft via the home.mycroft.ai page.
Requiring much more than to make a skill feels like a hindrance to me ( I am very lazy )
I started one too, for a more basic webpage type. With the idea that it would be intergrated into an existing page (but could be ran from home as well).
I would think using the msk https://mycroft.ai/documentation/skills/msk/ for the back end makes sense. It seems (from the docs) limited to the comment boiler plate and creating utterace testing currently.
Found another tool that maybe useful. It looks good from videos, requires webgl 3.0 (which is a potential drawback for ease of use). It seems more aimed for kids as well (but hopefully that just means simplicity!)
It is also the tool of choice for the q.bo robot that will be shipped with mycroft by default.
Website:
Github:
Another one in the vain of gui based programming is Snap. Which doesn’t look as visually polished(to me), but does provide an easy way to convert to python (among other languages). It also seams to be a little more powerful overall for programming.
Github:
and was told that the repo is using node-red block programming platform like Scratch, with the idea to enable anyone to create skills just by dragging and dropping.
A useful tool for skill building I’ve used in the past is VoiceFlow, its a visual skill builder that targets alexa and google. Perhaps Mycroft will be supported in the future.
