Mycroft-gui on my Mark-II - Simplified app

guhl · May 21, 2020, 9:21pm

Hi,

after getting the mycroft-gui-mark2 running on plasma-nano with the help of @AIIX and working with it for some time I decided that I do not like the architecture. In my opinion running X11, SDDM, plasma (even if it is nano) is a total overkill for my mark-2.
So I turned back to the mycroft-gui-app and worked on making it work in portrait mode. After a lot of struggle with the Kirigami framwork and its Application Window I decided that this is also totally unnecessary for running it on my mark-2. I don’t need a settings page and I also don’t need a input field.
So I forked the mycroft-gui repo to https://github.com/guhl/mycroft-gui and created a simplified version where I added to parameters --rotatePortrait and --webSocket (which defaults to ws://127.0.0.1) and the result is this:
Video

Sorry, there is no sound on the Video because I care a lot about my privacy and I definitely do not want to hear my voice on the internet. But you can trust me it was all done via voice, including the shutdown.

There are still some issues like for example the text output in the wikipedia skill but I’ll work on that.
In my opinion running the simplified mycroft-gui-app directly on eglfs makes much more sense than the plasma-nano approach.
The app is started (for example running on the mark2) using:

QT_QPA_PLATFORM=eglfs mycroft-gui-app --rotatePortrait --webSocket ws://127.0.0.1

Have fun - Guhl

AIIX · May 22, 2020, 10:40am

I don’t think there was a need to fork mycroft-gui as it was a simple change, mycroft-gui already allowed setting rotation of screen with --rotateScreen command line option but it was limited to 180 degrees, anyways the limitation has now been removed on mycroft-gui-application on git master and can be set to any degrees as per screen configuration using --rotateScreen 90 (for 90 degrees as example), additionally mycroft-gui-application also already had the option to hide the text input bar with simply passing --hideTextInput in command line.

j1nx · May 22, 2020, 5:22pm

At the moment I am working on trying to make use of; eglfs-wayland -> Weston -> plasma-nano -> QT mycroft-gui.

This with a patched version of the mycroft-gui-mark-2 look and feel package to not use the whole kworkspace dependency.

If it works, this is with very minimal dependencies as you only need the bare minimal kf4-framework packages, qtwayland and plasma-nano. So no whole Xorg or X11 packages and all of its dependencies nor SDDM.

Ofcourse without kworkspace there are no system calls available for halt/reboot or network, so an enclosure-skill is needed to handle those, but it is my oppinion that those calls should be handled by an enclosure skill anyway.

Will report back if I figured things out (or not).

guhl · May 22, 2020, 8:12pm

Hi @AIIX,

I am aware of the --hideTextInput functionality and I also tried that to just add a 90 degree rotation but that does not work.
I compiled your version and called it with:

QT_QPA_PLATFORM=eglfs mycroft-gui-app --hideTextInput --rotateScreen 90

It look like this:
mycroft-gui-rotate-90-01 mycroft-gui-rotate-90-02

While it rotates the content, the size of the rotated Rectangle is not Portrait but still Landscape.
I really tried to figure out why this does not work with the Kirigami.ApplicationWindow but I could not make this work so I decided to replace the Kirigami.Application window by a regular Window. I am of course by no means a QT or QML specialist but it has something to do with the size of the ContentItem of the ApplicationWindow itself.
I would be glad if you can make this work correctly but I gave up after some frustrating days.

Have fun - Guhl

guhl · May 23, 2020, 9:36am

@j1nx:
Interesting approach.

While I still don’t see the need of a window rendering software stack for this application in general.
If you take away the settings page and the entry field there is simply no need for windows on an embedded voice controlled device.
This might be different if you think about running it on TVs but for something like the mark II it is just bloat. And you don’t want to have unnecessary bloat on devices with limited memory and CPU capabilities.

I worked quite some time on porting Sailfish OS to different mobile phones and for mobile phones I see of course the need for something like Wayland as you have multiple applications and need a window management. Therefor I also understand why KDE is taking that approach for their strive to get a hold on mobile phones.
But the Mark II is not a mobile phone. It is a voice controlled assistant that provides additional visual feedback (which I do find very important and great) and that is a different game.

If you need help with getting your approach working than please show me where you publish your work and I’ll try to contribute.

Have fun - Guhl

j1nx · May 24, 2020, 12:42pm

@guhl I agree with you to some extend. For the current software stack as it is available now, there is no need yet for window rendering. But some where in the near future, especially in my use case of using a normal non rotated screen in landscape mode we would possible want to launch another window overtop of the mycroft-gui.

For me this would be for instance the wpe-webkite accelarated browser for youtube or netflix. Not really urgent at the moment as they are not readily available yet, nor is the option to choose where to output the video (Play on TV, screen, chromecast, etc)

With all the touchscreens available, touching the screen and have an overlay virtual keyboard is also very nice.

Anyhow: Digging in more and more, it appears the whole look and feel package is not easy or even impossible to run without KWIN which is part of the kworkspace package. So I think I will have to let that go for now.

I did however figured out how to launch weston as wayland compisitor and launch mycroft-gui on top of that. With that in the future I can launch multiple windows in the same weston-wayland session. So I could launch other windows and have them layed overtop of each other.

For my project; MycroftOS I would like to use most if not all parts of whatever teh Mycroft Dev team makes / releases, so it really is just an OS that connects all the Mycroft dots so to say.

It is based on buildroot, so a very minimal fast OS. If you would like to have a look or follow the development, it is on github.

To see it in action in the current state;

I already have the --noinput enabled, so it is basically the same as your version.

I let go of the weston stuff for now. It is there, but not started nor configured and just boots straight into mycroft-gui with eglfs.

Going to fix the wifi stuff I broke and then push it as alpha8 release as I promised some people to have a new version with the latest and greatest mycroft stuff included.