EC respeaker echo cancellation

Archive - Mycroft Project Hardware
1

I managed to get EC to work as in run but actually wondering if it works.
Anyone else had any success with EC from.

My main problem was when I stopped recording ec would stop.
I got past that by using a loopback device sudo modprobe snd-aloop

So firstly my /etc/asound.conf

 pcm.!default {
    type asym
    playback.pcm "eci"
    capture.pcm "plughw:CARD=Loopback,DEV=1"
}


pcm.eci {
    type plug
    slave {
        format S16_LE
        rate 16000
        channels 1
        pcm {
            type file
            slave.pcm null
            file "/tmp/ec.input"
            format "raw"
        }
    }
}

pcm.eco {
    type plug
    slave.pcm {
        type fifo
        infile "/tmp/ec.output"
        rate 16000
        format S16_LE
        channels 2
    }
}

pcm.cap {
 type plug
 slave {
   pcm "plughw:CARD=CameraB409241,DEV=0"
   channels 4
   }
 route_policy sum
}

start ec with
./ec -i 'cap' -o 'plughw:CARD=ALSA,DEV=0' &
then redirect the mic to the loopback
arecord -D eco -q -r 16000 -f S16_LE -c 2 | aplay -D plughw:CARD=Loopback,DEV=0 &

The big question is does it actually work as yeah in the cli I can see aec switching on & off on play.
But does it make a difference?

2

Perhaps you play play some music over the speakers while you record. Then if you play back your recoding, the music should be removed isn’t it?

3

Yeah that would seem wise :slight_smile:
Must be because the mic volume is low, but does a pretty good job.
Wake word doesn’t seem to like and its been confusing me.

that will be pretty ace if I can get the mic volume louder but it my ps3eye and amixer or alsamixer just fail if I try and access.
Its got quite low load but the ps3eye is driving me mad, I can not even get anything in alsamixer for it and was sure it used to show.
Maybe on top of that the one I have is on its way out, just going to wait and wait for the respeaker.

4

I did some experiments with “ec” some time ago. You need an audio device that has hardware loopback channel otherwise results will be disappointing.

5

@StuartIanNaylor You can try to let alsa ignore the pcm errors by;

Not sure if that works for the PS3, but give it go. Alsamixer doesn’t crash/exit anymore, but don’t know if it brings you anything.

6

That is a kernel compile option @j1nx and prob not a good idea.
ps3eye cam is a bad hack from a ps3 and to be honest now I know more its not even that cheap.
I am not going to hack a kernel just so a relatively buggy piece of equipment can work as there working usb mics such as the respeaker usb 4 mic.
Prob should take a look at what they did with the driver on that.

Its something to do with channel maps is it IEC958? The format that sets out all the surround sound mapping as at times the ps3 has worked and not worked.
I think its been changes to the IEC or maybe Raspbian have been adding and removing your line at points in time, but doubt that.

@Dominik I don’t know what are good results with EC are as never really listened to the output just switched it on and used.
I just used a loopback device sudo modprobe snd-aloop and with my 50watt boom s-pipe about .7M away from the mic is an extremely loud sound source.
Its sort of wierd as the echo source is rendered totally to this really quiet muffled background and my voice is at the normal recording volume of the PS3eye and seems to sound the same.
But wake word detection seems to go to hell?!

Thing is the normal volume is pretty crappy with the ps3eye as always been bemused how well mycroft does when I actually record and playback myself with it.
PS3eye got me going but its destiny is the bin.

@Dominik I was wondering as the instructions are pretty sparse for ec if I was mixing ec & ec_hw as was looking at the input and and output and wondering if that made sense, as sort of doesn’t.
I will wait for the 4 mic Pi respeaker so I have a clean datum to play with as the minimal load it produced is worth giving it further play.

I can not work out how the PS3eye seems to of gone worse was it my move from pi3 to pi4 or does Pycroft contain that kernel option? As the thing does seem to be worse and the volume on recording is bad.
Its just not a good datum for any testing and is said is destined for the bin.

PS3eye was a good recomendation when we where lacking options but there are new 2/4mic versions that are just a couple of $ more and maybe not have the possibility of being an unknown used cam that could be 15 years old.

7

@StuartIanNaylor I know! It is just a “quick fix” to make sure stuff get’s loadeddespite ALSA kernel freaking out.

Did a quick test yesterday and indeed, if you have ALSA ignore the ctl error, alsamixer doesn’t error out / quits selecting the PS3 card. But as I though, it doesn’t bring you anything as then of course alsamixer says there are no controls :man_facepalming:

No clue about setting things with amixer on the CLI, but as you already threw your PS3 out of the window it doesn’t matter anymore.

BTW: On MycroftOS with my PulseAudio setup, I was pleasantly suprised about the PS3. Picked up my normal speaking voice from acroos the room. No clue how it holds up with heavy background noices though. Will have a look at it later on…

1 Like
8

That is the wierd thing @j1nx it does pick up in a quiet room across the room but if I record generally the volume is always low even close up.
Sort of bemused with it and its heading for the box of bits and pieces which is often a precusor to the bin but I am terrible for hoarding electric bits & pieces, but doubt honestly I will use it.

Was wondering if others had played with EC as the config setup and instructions are not simple or great.
The load on a Pi4 was nothing and didn’t try but instantly thought if that worked OK and not sure about settings or the Mic I picked that would also be OK on a Pi3A+ for load.

“quick fix” is great you provide I am very tempted to purchase a 4 mic clone just to say if the work without probs.

9

I have the official ReSpeaker 4-Mic HAT and it works perfectly, but to be honest you are better of buying either the 4-Mic linear;

https://respeaker.io/linear_4_mic_array/

Or the 6-Mic;

https://respeaker.io/6_mic_array/

Because of the hardware loopback channels on them. With those EC works SO MUCH better. Plus you can use the audio out on those devices which are way better that the onboard jack of the RPi.

They are a bit more expensive, but give you more freedom to tinker with.

10

Confused as the loopbacks are the echo channels for on board ec?
Shouldn’t really make much difference to quality with the right algorythm?
Its just the EC processing is done on the card so no load?

11

PS do you guys know if alsabat -Pplughw:CARD=ALSA,DEV=0 -Cplughw:CARD=Device,DEV=0 --roundtriplatency -B1024 actually works?

Seems to come up with the same figures whatever
With EC you can set a delay which is fine but how do you know what the latency is.
Thought there would be tools unless the above is correct and is 53ms but that is very good for a Pi but it is a Pi4 so maybe?

12

Sorry, no clue. Never looked at EC.

Perhaps @Dominik as he played with EC before (and discarded it, when there is no hardware loopback channel)

13

Sorry, I don’ remember any configuration specifics - my experiments date back to August 2018 when working on the bounty source issue.

14

I think that is what you will find the hardware loopbacks are for built in EC.
You dont really need EC if you have hardware EC, but would have to read up with my memory.
The pulseaudio AEC works well but for some is a bit load heavy I was just on a mission to find an ALSA one and see what it does for load.

The other cards starting climbing out of the price bracket I was looking at to be honest hence why was having a look at software EC.
I am not particularlly bothered about beamforming, but ec is pretty important and maybe forced to use the pulseaudio module but webtrc are no longer supporting and apparently it will go from pulse also.

Decideded to read.

ReSpeaker 4-Mic Linear Array Kit for Raspberry Pi support 8 input & 8 output channels in Raspbian system. The first 6 input channel for microphone recording(only first 4 input channels are valid capture data), rest of 2 input channel are echo channel of playback. The first 2 output channel for playing, rest of 6 output channel are dummy.

Not sure but when I read it echo channel is the audio to consider echo I thought they where looped back as that should correlate with system latency… Or anything else played that you have a signal.

Its one I am not going to buy so can’t tinker.

With ./ec -i plughw:1 -c 8 -l 7 -m 0,1,2,3 use loopback channel 7 but why / what is beyond me.

15

Now then after reading the Speexdsp manual which ec is based on its says you can not do echo cancelation on different sound cards as they have seperate clocks that drift and will stop the EC adapting.
I haven’t got the foggiest what that means really apart from it aint going to work well.

https://github.com/xiph/speexdsp/blob/master/doc/manual.pdf has the details @ 6.2.1

At least it gives me something to play with as now the 2 mic respeaker is looking back in the frame.

Spent some time googling about hardware loopbacks and can not find a reason apart from on board EC which is great if you have it.
You can do EC without a hardware loopback but clock drift and seperate cards at least stops the algs I can get hold of.

So I guess you can use any microphone / soundcard combo if you are prepared for a vocal forrest gump otherwise it has to be same card for capture/playback or really no EC.

So have one of these on the way as much prefer the layout of the 4 mic but looking like for EC that might be a prob.
https://www.ebay.co.uk/itm/ReSpeaker-2-Mic-Pi-HAT-V1-0-Expansion-Board-for-Raspberry-Pi-Zero-W-B-2B-3B/312242135809?

Wish when they designed that they though about maybe having connectors on one side and mics, button leds on the other, but hey ho.

16

Keep us posted! That 2-mic with it pricepoint can never be a bad decision to buy.:stuck_out_tongue_winking_eye:

17

Yeah it sort of fits my approx ÂŁ10 per module price range and also means no dac is needed.

£9.57 aint bad I paid £7.99 for a PS3 eye but didn’t shop much.
THe layout of the 4 mic is much better but will have to see how EC compares but prob think that will set my budget picks.

The other drawback about EC is audio quality as in all I have seen its set to 16K sampling and I am pretty sure if clock drift matters so much as does sampling rate that playback and capture must match?

18

PS @j1nx @Dominik
There is also a python script for that EC which with a bit of molesting I guess you could add portaudio streams to the script and negate the malarky for the fifo and EC usuage.

"""Acoustic Echo Cancellation for wav files."""

import wave
import sys
from speexdsp import EchoCanceller


if len(sys.argv) < 4:
    print('Usage: {} near.wav far.wav out.wav'.format(sys.argv[0]))
    sys.exit(1)

frame_size = 256

near = wave.open(sys.argv[1], 'rb')
far = wave.open(sys.argv[2], 'rb')

if near.getnchannels() > 1 or far.getnchannels() > 1:
    print('Only support mono channel')
    sys.exit(2)

out = wave.open(sys.argv[3], 'wb')
out.setnchannels(near.getnchannels())
out.setsampwidth(near.getsampwidth())
out.setframerate(near.getframerate())


print('near - rate: {}, channels: {}, length: {}'.format(
        near.getframerate(),
        near.getnchannels(),
        near.getnframes() / near.getframerate()))
print('far - rate: {}, channels: {}'.format(far.getframerate(), far.getnchannels()))
echo_canceller = EchoCanceller.create(frame_size, 2048, near.getframerate())

in_data_len = frame_size
in_data_bytes = frame_size * 2
out_data_len = frame_size
out_data_bytes = frame_size * 2

while True:
    in_data = near.readframes(in_data_len)
    out_data = far.readframes(out_data_len)
    if len(in_data) != in_data_bytes or len(out_data) != out_data_bytes:
        break

    in_data = echo_canceller.process(in_data, out_data)

    out.writeframes(in_data)

near.close()
far.close()
out.close()

With the https://github.com/xiph/speexdsp/blob/master/doc/manual.pdf and the SpeexDSP source https://git.xiph.org/?p=speexdsp.git;a=summary
There is an absoute plethora of new toys to play with.
VAD, AGC and all sorts that will prob wait until I have a all-in-one soundcard to play with.
The tail length of the EC is confusing as there is an optimal small length for EC but also smaller means more load, so might take some playing with to get it optimal.

EC also has a delay that gives an example approximation of 200ms which when I ran alsabat -Pplughw:CARD=ALSA,DEV=0 -Cplughw:CARD=Device,DEV=0 --roundtriplatency -B1024 on a Pi4 its a long way out as was an approx average of 50ms but don’t know how good that test is.
Audacity and the other main audio app on linux that I can not remember (http://ardour.org/) have some latency tools that could prob verify alsa latency.
If you only have a tail length of 100ms then that is a country mile out.

It will be weeks before the 2 mic turns up pointless me testing on something sub optimal of 2 sound cards.
I did notice int frame_size = config.rate * 10 / 1000; // 10 ms whilst speex suggest 20ms (stereo?). Noticed also the default filter length is 2048 but if 100ms is a ‘good choice’ at 16khz then shouldn’t that be 1600?
Also its not an order (multiple) of the frame size.
Also the buffer which is 128x the filter length ?

Maybe the manual & latest release are a bit out of sync or some mistakes have been made.
Been sat waiting for ages for the 4 mic and might have to wait longer as should of got the 2 mic at least for initial setup.

19

Been there, done that. I even built a LADSPA plugin from speexdsp-lib myself. I never got satisfying results.

Try to understand that and you will see that it is a waste of time to get this working on RPIs for soundcards with hw-loopback. There is a reason why the better Respeaker devices have a dedicated DSP onboard…

20

No you tried with a Pi3 but from windows to pulseaudio there have been excelent software EC.
The EC part unlike beamforming isn’t really rocket science or that heavy and doesn’t nessitate a hw-loopback or dedicated dsp.
It even has working software on mobile phones, the speex implementation might not be great but from what I can see EC not has implemented it great either.
Shame webrtc doesn’t have something with the Aec via python but what you are saying doesn’t make sense as EC has had many software implementations on many platforms the only thing is load on a PI and with the Pi4 its prob likely.

The point is the ‘better’ respeaker products create an opensource product that is out of whack with commercial reality.
Maybe if you have commercial uses for them but your paying the price for a mic that is the price of a Google home.
That was my main gripe with the initial designs as unless cheaper modules can be found its not about expense its cost disparity to what is already available.

So yeah I am going to give it a go. :slight_smile:

If its been working on android for as long as it has I am sure it can work on a Pi4.

If Speex just doesn’t manage it then it will be a matter of pulseaudio but prob is next release its going to be dropped as its been dropped from webtrc.
So again doesn’t matter if pulseaudio or asla but an alternative to webrtc or rehash the existing code as webrtc has supposedly a great implementation of EC.