How to get Mycroft to speak and understand Dutch?

Localizations and Accessibility
1

Hey! Newby here, I’m looking for a way to get my mycroft to understand some simple Dutch words/sentences and also reply in Dutch. Is this already possible? And how? It does not need to be extensive or really high quality for now, just a couple of specific sentences will be enough. It is quite important for me to get this to work since I’ve bought the Mycroft Mark 1 to use as a Demo for some product, and the date for it is fast approaching.

I tried setting the language in mycroft.conf to “nl-nl” and “nl”, but the picroft then stays stuck at the yellow eyes. Are there maybe skill that really need the nl-nl files or otherwise startup fails? Or should it fall back to using english sentences? I have made nl-nl languages folders in some of the skills (not all yet).

Regarding the understanding of Dutch: I’m guessing the only thing I need to implement for this is a STT engine that can understand Dutch language. But changing the STT engine setting to google seems to stop picroft from booting. Do I need to set additional settings? Setting the TTS engine to google with “lang”: “nl” did work, it produces the typically dutch van gaal-like accents!

I think I don’t need to add a Dutch library file since the wake-word in Dutch and English sounds exactly the same. Is this true, or do I still need to implement a library with ‘hey’ and ‘Luna’ (that’s my picroft’s name) in there?

I know these are a lot of questions, I hope someone can help me out! Many thanks in advance!

/home/mycroft/.mycroft/mycroft.conf:

`{"max_allowed_core_version": 18.8, 
"tts":{"google":{"lang": "nl"}}, // This works!
//"stt":{"module":"google", "google":{"lang": "nl"}}, // uncommenting -> picroft doesn't start
//"lang":"nl-nl" // uncommenting -> picroft doesn't start
}`
2

you are missing the comma-seperator after each parameter, try following:

{
"max_allowed_core_version": 18.8, 
"tts":{"google":{"lang": "nl"}},
"stt":{"module":"google", "google":{"lang": "nl"}},
"lang":"nl-nl"
}
3

You can also see a full list of all the elements that are required to support a new language here;

TTS and STT are starting points; we’d also love a hand translating Skills into Dutch on Mycroft Translate;

https://translate.mycroft.ai/nl/mycroft-skills/

4

Thanks! Unfortunately with comma’s inserted (and comments removed) the picroft still can’t get past the yellow eyes :frowning: Any other ideas to try?

5

Thank you, I found that list! Unfortunately it only states to configure mycroft to use a specific STT engine via the mycroft.conf file, but not how exactly this needs to be done. Do you know how to do this?

I really like the effort mycroft makes to make their software usable in different languages and therefore I already have contributed (and intend to contribute more) to the ‘official’ Dutch translation library! Unfortunately for me I need to have my mark 1 speaking Dutch in about 3 weeks and I recon that the Dutch software from mycroft will not be done by then…

1 Like
6

In order to sort out any configuration issue can you please post your complete mycroft.conf files here - the one located in /home/mycroft/.mycroft and in case you have modified it the one from /etc/mycroft as well

1 Like
7

/etc/mycroft:

{
  "enclosure": {
    "platform": "mycroft_mark_1",
    "platform_build": 19,
    "port": "/dev/ttyAMA0",
    "rate": 9600,
    "timeout": 5,
    "update": true,
    "test": false
  },
  "VolumeSkill": {
    "default_level": 6,
    "min_volume": 0,
    "max_volume": 83
  },
  "ipc_path": "/ramdisk/mycroft/ipc/",
  "hotwords": {
    "hey mycroft": {
      "module": "precise"
    }
  },
  "tts": {
    "mimic": {
      "path": "/usr/local/bin/mimic"
    }
  },
  "SkillInstallerSkill": {
    "path": "/opt/venvs/mycroft-core/bin/msm"
  }
}

/home/mycroft/.mycroft:

{
"max_allowed_core_version": 18.8, 
"tts":{"google":{"lang": "nl"}},
"stt":{"module":"google", "google":{"lang": "nl"}}
# "lang":"nl-nl"
}

With the code above I get a Dutch accent, but I’m getting a “I’m having trouble communicating with the Mycroft servers, please give me a few minutes before trying to speak to me” error. I commented the “lang” line out since with it on it would not go past the yellow eyes after rebooting. I must say I currently don’t have any skills containing the “nl-nl” language directories since Mark decided to go AWOL on me and I had to reburn the mycroft image onto the pi to get it working again… Can that be a problem? Are there specific skills that need to have these files available in order for the picroft to boot correctly?

Thanks again for helping me!

8

As far as I remember my first steps “teaching” my Mark-I to talk in German at minimum you need a dialog-folder in your language “nl-nl” for the skill “fallback-unknown” and inside there a file “unknown.dialog” with at least one phrase like e.g. “ik begrijp het niet”

EDIT: your mycroft.conf looks good so far I can tell. You may want to check home.mycroft.ai under Settings - Advanced if Text-to-Speech Enginge is set to “Google” as well.

9

Text to speech is set to google indeed! I added a dutch file with some lines of text in his own dutch nl-nl folder for the unknown.dialog, but still it won’t fully boot when I enable the language line in the config file… Maybe @KathyReid knows which skills/dialog files minimally need to be translated in order for the picroft to boot in another language?

EDIT: Added nl-nl files for all vocab and dialog files but still isn’t booting. I did notice that the sync message while booting changes from “syncing” to “message syncing” when decommenting the language line, don’t know if that could provide a clue?

10

Yes, this tells us that some dialog-files in the mycroft-core are missing for Dutch.
Compare ˋmycroft/res/text/en-usˋ with ˋmycroft/res/text/nl-nlˋ
Among the missing files is the one that has the dutch translation for „syncing“. This is a good sign as your Mark-I is running Dutch language Now.

As you rebuild your Sd-Card you are now on Mycroft 18.02 which had a bug that caused the Mark-I eyes to stay yellow and the Button Menu did not work. Try talking „Hey Mycroft“ - if you are lucky it will respond despite the yellow eyes…
You may also want to log in via SSH and force an update to 18.08 via ˋsudo apt-get update ˋ followed by ˋsudo apt-get upgrade ˋ

11

Apperently I was already running 18.08; updating and upgrading did not result in the following output:

0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

I complemented the files in /opt/venvs/mycroft-core/lib/python3.4/site-packages/mycroft/res/text/nl-nl so they match the files found in en-us. Unfortunately, still yellow eyes and no reaction from Mark :frowning:

EDIT: I just found out that it will react to my voice occasionaly! Unfortunately it still replies in english instead of Dutch, despite having the “lang”: “nl-nl” in my config file. Also it never uderstands what I’m saying (also when I’m typing things in the client), it always immediatly responds with with “I don’t understand” (or any other sentence of the fallback skill)

12

Can you show us the skills.log (/var/log/mycroft/skills.log) from such a conversation?
Log excerpt should start at the line containing ..... - INFO - Begin Recording... and end where it speaks “I don’t understand”.

13

Oops, I messed something up with the log files which I couldn’t resolve so did another manual rebuild. I did all the things mentioned above, at this point the picroft will not completely finish booting and therefore I cannot talk to it out loud, but if I try to communicate using the client it responds with “Wacht tot ik ben opgestart” (which is the dutch phrase I set for “please wait while I finish booting up” in mycroft-core). Therefore I did not find the exact log entries you were looking for, but I hope these help:

20:57:53.539 - SKILLS - DEBUG - {"data": {"lang": "en-us", "utterances": ["yeah"]}, "type": "recognizer_loop:utterance", "context": null}
20:57:53.559 - SKILLS - DEBUG - {"data": {"utterance": "yeah", "lang": "en-us"}, "context": {}, "type": "intent_failure"}
20:57:53.565 - mycroft.skills.padatious_service:handle_fallback:133 - DEBUG - Padatious fallback attempt: yeah
20:57:53.567 - WolframAlphaSkill - DEBUG - WolframAlpha fallback attempt: yeah
20:57:53.566 - SKILLS - DEBUG - {"data": {"handler": "fallback"}, "context": {}, "type": "mycroft.skill.handler.start"}
20:57:53.568 - WolframAlphaSkill - DEBUG - Non-question, ignoring: yeah
20:57:53.574 - mycroft.skills.core:handler:1383 - ERROR - Exception in fallback.
Traceback (most recent call last):
  File "/opt/venvs/mycroft-core/lib/python3.4/site-packages/mycroft/skills/core.py", line 1374, in handler
if handler(message):
  File "/opt/mycroft/skills/fallback-unknown.mycroftai/__init__.py", line 40, in handle_fallback
for l in self.read_voc_lines(i):
  File "/opt/mycroft/skills/fallback-unknown.mycroftai/__init__.py", line 28, in read_voc_lines
with open(self.find_resource(name + '.voc', 'vocab')) as f:
TypeError: invalid file: None
20:57:53.582 - mycroft.skills.core:handler:1387 - WARNING - No fallback could handle intent.
20:57:53.584 - SKILLS - DEBUG - {"data": {}, "context": {}, "type": "complete_intent_failure"}
20:57:53.591 - SKILLS - DEBUG - {"data": {"exception": "No fallback could handle intent.", "handler": "fallback"}, "context": {}, "type": "mycroft.skill.handler.complete"}
20:57:53.621 - SKILLS - DEBUG - {"context": null, "type": "speak", "data": {"utterance": "Wacht alsjeblieft even tot ik opgestart ben"}}
20:57:56.611 - SKILLS - DEBUG - {"context": null, "data": {}, "type": "recognizer_loop:audio_output_start"}
20:57:56.662 - SKILLS - DEBUG - {"data": {}, "context": null, "type": "Command: mouth.talk"}
14

The Error-message tell us that the fallback-unknown skill is missing files in the vocab-dir. You must create a subfolder “nl-nl” there and copy/translate the *.voc files from the “en-us” folder.
When already there please check file ownership for all files you have touched or created with ls -la - files should have “mycroft:mycroft” as owner/group.

1 Like
15

I think we could benefit of a skill that actualy set thise tings up. That would malke it easy to Chang language and test stuff.

The skill could be as simple as Chang relevant setting and rest art service.
And setting were read from a setti gsfile, so adding more language were easy.

I did look i to this, but did t undrrstand what setting in which files nedded to be changed when Changng language.

16

Sorry for the late reaction! We wanted to use mycroft AI for a demo but the Dutch language was crusial so we decided to go with another robot. Since I needed to program that in about a week and a half I did not have time to spent on the mark 1. Unfortunately I also had to hand it in again since the demo day is over so I won’t be able to test anything for a while. I’m planning on buying my own in the near future and I will definitely get back to trying to get it talking and listening in Dutch!

That being said, I still was able to add the voc files in a new nl-nl folder and check the ownership: the ownership of everything I made is root:root, not mycroft:mycroft.

Again, thanks a lot for helping! I will let something know as soon as my new mark arrives :slight_smile: