Dialog rendering and grammar (NLP)

I want to kick things off here instead if dumping it as an issue.

There are a bunch of Lingua Franca helper functions to parse and format parts of speech, trying to hash out the intricacies of different languages. Yet the dialog renderer is a big blocker to be grammatically precise.

On the skill level you normally compute the data (eventually leverageing common parsing/formatting procedures by LF or through custom code), which is then sent to the mustache renderer picking one line constructing the scentence to be voiced.

This poses a challenge for certain languages, since to be accurate the context (of the dialog line) is needed beforehand. As is the skill dev is pressed to form all lines of dialog in a nominative case. In return you get a pretty stale sounding phrase, sometimes impractical or outright impossible. I’ve seen some helper functions recognizing this problem offering a contextual parameter, yet core falls short to facilitate that with the given pipeline.

The “easy” way would be to prerender a dialog, tokenize it and pass it as context to the helpers, but i would prefer a more robust “formatting pipeline” as a part of the dialog rendering process in core (Note that only a fraction of a fraction is getting the LF treatment). The ultimate goal should be - in my book - to abstract this away from the skill level, since i don’t think this should be a skill devs concern.

The context should be at least POS-tagged, dependency parsed and optimally have some morphological analysis attached. I always come up with the tool spaCy in that regard, which is -unfortunately- dubbed as too chunky. On the other hand has a pretty accessable, flexible and extendable pipeline, which should be beneficial to LF. Its purpose is foremost analytical NLP - the part core/LF is lacking -, but i can see a win-win scenario.

Thoughts on this?

Not directly in the scope of What is wrong with mycroft-core right now? but certainly an addendum


I see this as a natural language generation problem, specifically multi-language. Ideally, you would be able to specify the semantics of the dialog only (e.g., abstract meaning representation), and the syntax would be generated for you.

How do you imagine people would specify dialogs at the semantic level? Or were you thinking of something more syntactic, like “use dialog X if the subject has case A, and dialog Y otherwise”?

1 Like

this subject has been brought up a few times in mattermost and github, here are some relevant discussions from github

1 Like

My first thought is “Wait what?” Your post looks like English, but it was a bit over my head :grimacing:

How would this apply to a specific issue? I posted here:
First prototype for better music playing skills
on better music playing skills. There is hard-coded English such as album/artist/band/by/song/track, etc. How should that code be internationalized based on your suggestions?


-Mike Mac

P.S. Does Am%zon Al%xa play music by requests in languages other than English? I’m guessing so…

Michael seems to be 10 steps ahead. Sentences being free form would be next level. I wasn’t thinking that far.

to borrow a comment/example from another discussion (from NeonMaria):

In Ukrainian and Russian languages numerals change their form not only depending on the case or plural/singular form of the dependent word, but also on the numeral itself. There are several numeral groups (7 in Ukrainian, 4 in Russian) each of them has its own rules of getting flexions. That’s why putting numerals in correct form in Ukrainian, Russian and some other synthetic languages requires dependency sentence parsing and POS tags determination. Below there are some examples of those:

* Group I:

1. **один** (one) - changes flexions by gender, case and plural/singular forms of the following noun: **одне, одна, одні, одного, однієї (1)**

e.g. Ми бачили одного(1) хлопця. (We saw one guy.)
Numeral(1) depends on singular noun in genitive case (хлопця), according to this we put numeral (один -1) in certain form in genitive case (одного).

* Group II:

1. **Actually quantitative numerals** (власне кількісні: два, три, чотири), (2, 3, 4), (two, three, four).
   Change their flexions according to the case of the noun in plural form on which it depends.

e.g. По вулиці йдуть два(2) хлопці. (Two boys are walking on the street.)
Numeral (два - 2) depends on plural noun in nominative case (хлопці). That’s why we put quantitative numeral (2 - два)

2. **Composite numerals** (збірні числівники: двоє, троє, четверо, п’ятеро, …) (2 – 20 і 30) (two, three, four -, twenty, thirty).
   Change their flexions according to the case of the noun in plural form on which it depends. Can not be dependent on nouns in the nominative case.

e.g. По вулиці йшли двоє(2) хлопців. (Two boys were walking on the street.)
Numeral (двоє - 2) depends on plural noun in genitive case(хлопців). That’s why we put composite numeral (2 - двоє)