First attempt at Precise wakeword

Archive - Mycroft Project Archive - Machine Learning
1

I’m trying to use the Precise engine to cut down on false positives with our custom wakeword. I did the pre-requisite installs, managed to capture a few dozen utterances of the wakeword and downloaded the pdsounds for the random sounds as a start to the not-wake-word collection. I’ll eventually add much more data, but I just wanted to do a pass to make sure everything was running.

The precise-collect ran w/o any trouble, but the precise train failed with an exception that means nothing to me. I’ve copied the exception as well as the pip3 output showing the various packages and versions in the virtual environment. Any help is much appreciated.

[jnickel:~/src/mycroft-precise] [.venv] dev* ± precise-train -e 60 emilia.net emilia/
Using TensorFlow backend.
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Data: <TrainData wake_words=36 not_wake_words=593 test_wake_words=0 test_not_wake_words=0>
Loading wake-word...
Loading not-wake-word...
Loading wake-word...
Loading not-wake-word...
Inputs shape: (629, 29, 13)
Outputs shape: (629, 1)
Test inputs shape: (0, 29, 13)
Test outputs shape: (0, 1)
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
net (GRU)                    (None, 20)                2040      
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 21        
=================================================================
Total params: 2,061
Trainable params: 2,061
Non-trainable params: 0
_________________________________________________________________
2019-10-09 16:49:51.251980: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2593915000 Hz
2019-10-09 16:49:51.252969: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4b51ec0 executing computations on platform Host. Devices:
2019-10-09 16:49:51.253008: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
Train on 629 samples, validate on 0 samples
Epoch 1/60
Traceback (most recent call last):
  File "/home/jnickel/src/mycroft-precise/.venv/bin/precise-train", line 11, in <module>
    load_entry_point('mycroft-precise', 'console_scripts', 'precise-train')()
  File "/home/jnickel/src/mycroft-precise/precise/scripts/train.py", line 178, in main
    Trainer().run()
  File "/home/jnickel/src/mycroft-precise/precise/scripts/train.py", line 171, in run
    initial_epoch=self.epoch, callbacks=self.callbacks
  File "/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/keras/engine/training.py", line 1239, in fit
    validation_freq=validation_freq)
  File "/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/keras/engine/training_arrays.py", line 210, in fit_loop
    verbose=0)
  File "/home/jnickel/src/mycroft-precise/.venv/lib/python3.7/site-packages/keras/engine/training_arrays.py", line 469, in test_loop
    outs[0] /= num_samples  # Index 0 == `Loss`
IndexError: list index out of range
θ60° [jnickel:~/src/mycroft-precise] [.venv] dev* 4s 1 ± pip list
Package              Version Location                                
-------------------- ------- ----------------------------------------
absl-py              0.8.1   
astor                0.8.0   
attrs                19.2.0  
bbopt                1.1.6   
cycler               0.10.0  
decorator            4.4.0   
fitipy               0.1.2   
future               0.17.1  
gast                 0.3.2   
grpcio               1.24.1  
h5py                 2.10.0  
hyperopt             0.1.2   
joblib               0.14.0  
Keras                2.3.1   
Keras-Applications   1.0.8   
Keras-Preprocessing  1.1.0   
kiwisolver           1.1.0   
Markdown             3.1.1   
matplotlib           3.1.1   
mock                 3.0.5   
mycroft-precise      0.3.0   /home/jnickel/src/mycroft-precise       
networkx             1.11    
numpy                1.17.2  
pip                  19.2.3  
pocketsphinx         0.1.15  
portalocker          1.5.1   
precise-runner       0.3.1   /home/jnickel/src/mycroft-precise/runner
prettyparse          0.2.0   
protobuf             3.10.0  
pyache               0.1.0   
PyAudio              0.2.11  
pymongo              3.9.0   
pyparsing            2.4.2   
python-dateutil      2.8.0   
PyYAML               5.1.2   
scikit-learn         0.21.3  
scikit-optimize      0.5.2   
scipy                1.3.1   
setuptools           41.4.0  
six                  1.12.0  
sonopy               0.1.2   
speechpy-fast        2.4     
tensorboard          1.13.1  
tensorflow           1.13.2  
tensorflow-estimator 1.13.0  
termcolor            1.1.0   
tqdm                 4.36.1  
typing               3.7.4.1 
wavio                0.0.4   
Werkzeug             0.16.0  
wheel                0.33.6
2

I found my problem after reading the source code at the bottom of the stack. I hadn’t provided any test wake words and test not-wake-words and the number of samples was 0 and that exception was thrown.

I generated test words for each category and the model is now being created.

2 Likes
3

Thanks for the update Jody, I’m sure others will run into the same problem in the future, so it’s very helpful to have a solution available!

4

Stumbled across this. How do you have managed to get Tensorflow 1.13.2 installed? (running Mycroft in the KDE Bigscreen OS) I’m struggling quite a bit to get this one installed. Is python 3.7 the key?

5

pip3 install tensorflow==1.13.2
usually does the trick. May depend on python version as well.

6

Thanks, but if it would be that easy i wouldn’t bother you. Even tried it from file and url. Always the same Error

ERROR: Could not find a version that satisfies the requirement tensorflow<1.14,>=1.13 (from mycroft-precise==0.30) (from versions: none)
no matching distribution found for tensorflow<1.14,>=1.13 (from mycroft-precise==0.30)

I’ve read that tensorflow has jumped a lot through the python versions during its development cycle, so…
And i mentioned KDE (KDE Neon btw). Picroft and i guess Mark2 environments have to use another tensorflow version, Picroft 1.10 for instance. So i figured Tensorflow1.13.2 and Python3.7 were a conscious choice.

7

hello, we are an italian development team and we would like to know some information about the custom wake word.

We did the train with precise 0.3.0 and it works quite well with a small number of people: 110 registrations.

We wanted to know how many recordings you used and if the wake word works well for you and is recognized in precise times and if you have the same result in mycroft or pycroft.

We with our wake word on precise always recognize it while on pycroft (importing precise 0.3.0) we have problems that often do not understand the wake word.

How many registrations should be made to have 1000 different people of the same nationality recognize the wake word?

Do you have any useful suggestions for us?

2 Likes
8

What do you mean by registrations? Wake word WAV files?

Generally, the more data, the better. Also try and have 3+ times the number of not wake words vs. wake words. It will also take some time and usage combined with saved wakewords to really improve the model in real world usage in an updated model.

I have maybe 400 wake words and about 2500 not wake words for my model, combined with the google voice commands and public noises datasets. The majority of my not wake words come from false activations. At this point it’s highly accurate for me. I also still record the uttrances, and use the not-wake-words for re-modeling on occasion.

1 Like