Mycroft not responding while Spotify is playing

redacted · February 26, 2020, 3:21pm

I am using picroft and everything seems to be working. However when I am playing spotify mycroft will not respond to anything. Once the song stops via stopping or it being over, I will get a response for my previous queries that were stated during the song.

Is this expected behavior?

pcwii · February 26, 2020, 9:24pm

Unfortunately the Rpi does not have built in Echo Cancellation so, yes this is expected behavior.
There is information scattered throughout the forum on this.
Look at some of the work being done with respeaker as it might give you some ideas.

StuartIanNaylor · February 26, 2020, 9:38pm

You can use pulse audio for echo cancelation.

https://wiki.archlinux.org/index.php/PulseAudio/Troubleshooting#Enable_Echo/Noise-Cancellation

sudo nano /etc/pulse/default.pa

### Enable Echo/Noise-Cancellation
load-module module-echo-cancel use_master_format=1 aec_method=webrtc aec_args="analog_gain_control=0\ digital_gain_control=1" source_name=echoCancel_source sink_name=echoCancel_sink
set-default-source echoCancel_source
set-default-sink echoCancel_sink

That should now set the default source to echo cancelled
pulseaudio -k to kill pulseaudio the daemon should restart auto 1-2 secs most

From the Archwiki

Possible ‘aec_args’ for ‘aec_method=webrtc’

Here is a list of possible ‘aec_args’ for ‘aec_method=webrtc’ with their default values [4][5]:

analog_gain_control=1 - Analog AGC - ‘Automatic Gain Control’ done over changing the volume directly - Will most likely lead to distortions.
digital_gain_control=0 - Digital AGC - ‘Automatic Gain Control’ done in post processing (higher CPU load).
experimental_agc=0 - Allow enabling of the webrtc experimental AGC mechanism.
agc_start_volume=85 - Initial volume when using AGC - Possible values 0-255 - A too low initial volume may prevent the AGC algorithm from ever raising the volume high enough [6].
high_pass_filter=1 - ?
noise_suppression=1 - Noise suppression.
voice_detection=1 - VAD - Voice activity detection.
extended_filter=0 - The extended filter is more complex and less sensitive to incorrect delay reporting from the hardware than the regular filter. The extended filter mode is disabled by default, because it seemed produce worse results during double-talk [7].
intelligibility_enhancer=0 - Some bits for webrtc intelligibility enhancer.
drift_compensation=0 - Drift compensation to allow echo cancellation between different devices (such as speakers on your laptop and the microphone on your USB webcam). - only possible with “mobile=0”.
beamforming=0 - This can significantly reduce background noise. See [8][9]
- mic_geometry=x1,y1,z1,x2,y2,z2 - Only with “beamforming=1”.
- target_direction=a,e,r - Only with “beamforming=1”. Note: If the module does not want to load with this argument, set azimuth (a) to the desired value, but set elevation (e) and radius ® to 0.

high_pass_filter=1, voice_detection=1, extended_filter=0 are questionable if the actual provide benefit.
VAD I thought is a Voice Active Detection filter but with nothing to use that signal prob pointless.

intelligibility_enhancer=0 isn’t implemeneted.

drift_compensation=0 Never tried.

I do use the rest and because I have a PS3 eye with a linear 4 mic array approx 20mm apart I have beamforming set. Noticed it seems to like an escape character of \ before the next parameter so my setup.

load-module module-echo-cancel use_master_format=1 aec_method=webrtc aec_args=“analog_gain_control=0\ digital_gain_control=1\ noise_suppression=1\ beamforming=1\ mic_geometry=-0.03,0,0,-0.01,0,0,0.01,0,0,0.03,0,0” source_name=echoCancel_source sink_name=echoCancel_sink

redacted · February 26, 2020, 9:46pm

Thanks for the reply, I am not sure how echo cancellation comes into it. So to be clear there is not supposed to be any reply to any voice commands while spotify is playing ? I am almost positive this used to work when I had this set up before.

redacted · February 26, 2020, 9:46pm

I will give this a shot, thanks!

StuartIanNaylor · February 26, 2020, 9:56pm

You might be better just grabbing and flashing Rasbian desktop to a card and just playing.

sudo apt-get install pavucontrol the simple gtk pulseaudio volume/config app.
You can view conf, current inputs and outputs there.
Pain to install desktop just for test, but to be able to see it all in one utility makes it a lot easier.
If it works there it will work in the raspbian lite mycroft image which is the same but desktopless.
Guess you could install the desktop if that makes it easier for you.
I just played with it once as now know it works

Add those parameters 1 by one and see what you think just doing a pulseaudio -k after each change to sudo /etc/pulse/default.pa

redacted · February 26, 2020, 10:02pm

I may do that, I never run raspbian desktop usually headless, too much overhead on the cpu and ram for graphics.

Thanks for your help! i appreciate it

edit

what does echo cancellation have to do with mycroft responding, does it just not listen when music is playing?

pcwii · February 26, 2020, 10:05pm

More info here…
https://www.bountysource.com/issues/56160402-bounty-implement-noise-cancellation-on-rpi-3-based-hardware-devices-mark-1-and-picroft
And here

StuartIanNaylor · February 26, 2020, 10:22pm

Pulseaudio & ALSA is absolutely amazing if you can work how to get it working and how the distro has set it up.

I just got one of these https://www.ebay.co.uk/itm/Luminous-Lights-Rechargeable-Wireless-Bluetooth-Speaker-Portable-Mini-Super-Bass/193351677179?
As thought £3.24 tiny neat little speaker/mic means the Pi can reside elsewhere.

Turned up today and speaker was just connect bluetooth but mic is not there. Windows it works auto on connection.
So its on a back burner until I find what settings bluez and pulse need for it, as I love linux but sometimes it can just really get on your nerves that a conf is needed somewhere and you have to go trawling google.

redacted · February 26, 2020, 10:30pm

Thanks for the information. That is a huge issue really. I’d say 95%+ of my usage of mycroft is for spotify. I was hoping to replace alexa because, well it’s amazon and I try and support OSS as much as I can.

I hope this gets figured out at some point, but back to alex for now.

Thanks for your time

Fellhahn · February 26, 2020, 11:16pm

I’m new here but I think you’ll want to check this setting in your mycroft.conf:

// Stop listing to the microphone during playback to prevent accidental triggering
// This is enabled by default, but instances with good microphone noise cancellation
// can disable this to listen all the time, allowing 'barge in' functionality.
"mute_during_output" : true,

From:

github.com

MycroftAI/mycroft-core/blob/master/mycroft/configuration/mycroft.conf

{
  // Definition and documentation of all variables used by mycroft-core.
  //
  // Settings seen here are considered DEFAULT.  Settings can also be
  // overridden at the REMOTE level (set by the user via
  // https://home.mycroft.ai), at the SYSTEM level (typically in the file
  // '/etc/mycroft/mycroft.conf'), or at the USER level (typically in the
  // file '~/.mycroft/mycroft.conf').
  //
  // The load order of settings is:
  //   DEFAULT
  //   REMOTE
  //   SYSTEM
  //   USER
  //
  // The Override: comments below indicates where these settings are generally
  // set outside of this file.  The load order is always followed, so an
  // individual systems can still apply changes at the SYSTEM or USER levels.

  // Language used for speech-to-text and text-to-speech.

This file has been truncated. show original

pcwii · February 26, 2020, 11:27pm

The rpi hardware just doesn’t have what it takes for true echo cancellation. There are hardware microphones available that also have speaker loopback channels for handling true echo cancellation. The previous links on respeaker provides information. Keep in mind this is still very much a work in progress project. The mark 2 supports EC as well.

redacted · February 26, 2020, 11:38pm

AMAZING! this is what I needed! thanks I will try it out

redacted · February 26, 2020, 11:41pm

I am looking forward to the mark 2, I hope it all works out as planned. The project is great! and we do need more privacy oriented voice assistant. I appreciate what the team is doing!

StuartIanNaylor · February 26, 2020, 11:50pm

Try the pulse audio settngs first as you will see Auto Echo Cancellation is pulse audio module as its the Google WebRTC that is Auto Echo Cancellation that is why its sets your source and sink to AEC.

Give it a go first before turning anything off.

PS Andlo is giving a radio skill a go and might also be of interest.

Not really sure what ‘The rpi hardware just doesn’t have what it takes for true echo cancellation.’ means to be honest as device aec isn’t hardware based ?!?

pcwii · February 27, 2020, 12:22am

I and others investigated this quite a while ago now with limited success. It wasn’t hardware specifically that was the issue but cpu resources that were the problem. The software EC took quite a few cpu resources to accomplish the task which is why it is not enabled by default and. A bounty exists to solve the problem. Maybe with the extra horsepower of the pi 4 things would be better but i have not tried. If you have success the community would be interested in how you achieve it.

StuartIanNaylor · February 27, 2020, 12:27am

I am using a Pi4 and the Webrtc AEC uses about half the load Precise does from top.

Dunno about pi3 but thought it would… ?

With the desktop test I suggested with pulse audio running I seemed to be getting about .5 load that is also with a 4 array beamforming running.
If that adds an extra… but definately on a Pi4 it copes easily.
I think pulseaudio with aec was adding about 10-14% load but would have to go back and check as I mainly remember it was half what Precise produced.