Mycroft-gui on my Mark-II - first light

Hardware Mark II
1

Hi,

well maybe this is all easy and obvious for the people here but I had a hard time to setup the software stack on my build of the Mark II prototype and did not find to much info in this forum.

In short:

  • I used picroft as basis
  • made it work with pulseaudio
  • built a slightly modified version of the mycroft-gui based on a custom QT 5.12 and custom build of the KDE components.

I will not copy all the steps to this forum post but you can find my complete build docu/notes here: My Mycroft-Mark-II build wiki

This is the first light on my Mark-II:
mark2_first_light

As said, maybe this is all just obvious for the pros here but for me it was a steep learning curve.
I hope it is useful to someone.

Have fun - Guhl

7 Likes
2

That is amasing. I like it, and your wiki is helpfull. I were thinking on how to add mycrfot-gui to Picroft and you got it. Would be nice to have it in default or as an option so picroft with an screen had a GUI. (i mainly wants that for my kitchen Pi as it has a 7" screen and is built into the wall.

3

Amazing!!! Bookmarked.

Check out these;

4

Hi j1nx,
interesting, I’ll look at that. The question is why did I not find that?
But on the other hand I am happy that I did not and did take the road to get the QuickView/QML stuff running.
Have - fun Guhl

5

Hey andlo,
Thanks for the feedback.
My next steps will be to provide the QT and the KDE stuff as deb packages so that one can install them without compiling them. Let’s see if I can get WebView compiled too.
And probably I will fork the mycroft-gui repository and provide the modified gui and a working dev_setup.sh from there.

I have been using gentoo linux for 15+ years so compiling stuff and fixing things that are not compiling is a routine for me and I have been doing porting of Sailfish OS to various mobile phones for quite a while (which is totally eglfs, wayland, QML based).
The downside is that I do have very little experience with the Debian packaging and binary installations in general but I think I can learn that.
Have fun - Guhl

6

Basically. the skill-mark-2-pi will take care of your volume control and some initial framebuffer drawing before the GUI kicks in.

Search the Mycroft github repositories, becauswe there is also an enclosure repo, showing you how they set up certain things.

For your screen being rotated;
https://jumpnowtek.com/rpi/pitft-displays-and-qt5.html

dtoverlay=pitft35-resistive,rotate=90,speed=32000000,fps=60

Set the rotation parameter to what you want. Values of 90 or 270 orient the display in landscape mode.

7

Congratulations!
As far as I can tell you are among the first users outside the Mycroft Dev team to get Mark-II/GUI software stack running on actual hardware. Keep up the good work.

1 Like
8

Hi j1nx,
Thanks for the rotation tip.
For the waveshare that I am using (as of BOM) it is described here Waveshare 4.3 Orientation Raspeberry Pi
and for me this works with:

# waveshare screen
max_usb_current=1
hdmi_group=2
hdmi_mode=87
hdmi_cvt 800 480 60 6 0 0 0
display_rotate=1 #1:90; 2: 180;  3: 270

in /etc/config.txt

1 Like
9

Hi,
it seems that there is a install able deb for QT5.12 (including Webengine) available here https://github.com/koendv/qt5-opengl-raspberrypi
did not check and install it but this might save a lot of time.
Have fun - Guhl

1 Like
10

In the meantime I built QT5.12 with Webengine myself on the pi using distcc to cross-compile it on regular amd64 linux hosts.
I did update the instructions on My-Wiki on how to setup distcc and compile QT5.12 with webengine.
Have fun - Guhl

2 Likes
11

@guhl is this on github at all if not could you post it on github.

12

Hi,
at the moment this is mainly software building instructions and they are posted on my trac wiki (see the OP) and I constantly update them. As soon as there would be code changes I will probably fork the mycroft-gui project and add the building inbstructions there.
Have fun - Guhl

1 Like
13

This is so cool!

You’re very welcome to contribute back any build documentation to the mycroft-gui project.

Also I might point out the https://github.com/mycroftai/mycroft-devices repo with the debos build recipe for our WIP image. This is currently arm64 architecture and will only work with the Pi4. We’ll likely be switching back to the armhf to get support for some hardware acceleration.

14

Instead of building/using the entire plasma stack you can look at deploying mycroft-gui on top of plasma-nano https://github.com/KDE/plasma-nano also adding specific shell configuration https://github.com/MycroftAI/mycroft-gui-mark-2

Scripts from these should also help if building with above:


https://github.com/MycroftAI/mycroft-devices/tree/master/overlays/mark2/etc/xdg // add to your image at same paths
https://github.com/MycroftAI/mycroft-devices/tree/master/scripts //to run at time of build

The skill that controls volume and gui enclosure: https://github.com/MycroftAI/skill-mark-2

If building on top of picroft it would be easier to upgrade picroft buster to debian testing so you can use system qt packages without having to rebuild qt. http://www.linuxandubuntu.com/home/upgrading-debian-from-stable-to-testing

15

@AIIX Am I correct in understanding that; “mycroft-gui” is what you currently use on bigscreen and therefor for normal landscape screens? While “mycroft-gui-mark-2” is the portrait versioni what is used on the Mark-2? Both doing the same / being the same, the GUI for Mycroft?

I am also looking into implementing the GUI within MycroftOS. What would you recommend for such a “normal” OS that just utilizes a widescreen landscape HDMI screen?

16

Mycroft GUI consist of two parts one an application that houses the mycroft skillview and second the library which provides the skillview. Both are part of the mycroft-gui repository.

The mark-2 prototype runs plasma-nano with the above mycroft-gui-mark-2 shell package that embeds the skillview directly into the shell so it runs as part of the shell and not inside the mycroft-gui application.

To better explain the “mycroft-gui-mark-2” package is a shell package for plasma nano that embeds the mycroft skillview onto the shell itself and also provides configuration to set the orientation, it provides support for a virtual keyboard, and provides some basic GUI interface for settings like wifi. All of this is provided in this package as plasma-nano is a much reduced package with lesser dependencies compared to what plasmashell provides on the desktop, The orientation is basically of the skillview and the shell, it shouldn’t be needed on a normal screen unless someone wants to build some custom enclosure with a rotated screen.

There really isn’t a direct concept of landscape/portrait mode since the skillview simply can utilize the whole screen if embed in a shell or whatever window like the GUI application, the skills should simply adjust to the screen/resolution, (it largely depends on skill devs to create response designs with qml, all of the current skills that implement a GUI are responsive so they should have no issues being displayed in any form factor / orientation )

Plasma Bigscreen is also based on plasma-nano but we use the mycroft-gui application to display the voice apps/skills in a window rather than embedding the skillview into the shell itself as bigscreen has a larger usecase of being able to use multiple applications.

1 Like
17

Thanks for the quick response. Much appreciated.

I think with “running directly into shell” you mean, straight onto the linux framebuffer instead of a full fledged X11/Wayland KDE desktop right?

In that case, that is exactly what I also have in mind for MycroftOS as everything runs on the framebuffer. No window manager or desktop environment what so ever. So if that is the case; mycroft-gui-mark-2 it is then!

This intrigues me. So you basically the whole “wifi setup wizard” stuff is not needed anymore if you use this GUI? If I remember correctly, plasma uses NetworkManager for it’s network setup right? Or does it talk to and / or configure the wpa_supplicant stuff straight away?

Sorry for bugging you, but now you are here… :smiley:

18

By the shell I mean plasmashell or the reduced version of it plasma-nano, it still requires X/Wayland. Window management is optional but for something like virtualkeyboard it could be required.

This as well uses network manager but the WiFi setup GUI has been added to the gui containment on the mycroft-gui-mark-2 package and doesn’t require the systemsettings or plasma network manager applet which would normally be found on desktop.

1 Like
19

Perfect!

Enough information to start cracking (although it might take a while before I can start timewise).

Will report back if I run into problems.

1 Like
20

Hi AIIX,

thanks for the response and all these very important insights.
I will have a thorough look at all this and rebuild my software stack based on that.
I’ll probably keep keep my self build QT/KDE but the plasma-nano is something that I was not aware of.

Have - fun Guhl