While Google Assistant more often than not gives us what we’re asking for, there is still room for improvement in terms of accuracy. Google previously let us customize how sensitive we want the hotword to be on our smart displays and speakers. But now they’re using federated learning with a new “Help improve Assistant” option to teach the system to avoid misactivations and misses. This lets you save audio recordings on your device to teach the speech technologies used for Google Assistant.
As per 9 to 5 Google, this new option is turned off by default but when you go to your Google Assistant settings or your Google app’s general preference, you’ll see something called “Help improve Assistant” that you can turn on. Basically, when you use Google Assistant, the audio recording will stay on your device and a “privacy-preserving technology” will be able to combine the information from you and other participants so it can teach the Assistant to be better and to help develop “better smart features”.
The main purpose of this is to refine its hotword detection model like when you didn’t say “Hey Google” but it was still triggered or when you did say “Hey Google” but it was not detected. But federated learning is processed locally so your audio recording is not sent to the cloud. Your device will send a “summary of the model changes to a Google server” and this can contribute to the improvement in Google Assistant’s triggering logic.
This option is turned off by default so this is a voluntary thing of course. And all your voice recordings are encrypted and will not leave your phone while the processing happens when your device is idle, charging, and connected to Wi-Fi. All recordings will be automatically deleted after 63 days or when it’s no longer needed. When you don’t want to join anymore and you turn off the settings, your recordings will be deleted.
This isn’t the first time that Google is using federated learning to improve its services. They already use it on Gboard and the Health Studies app. Remember that this is different from the Google Assistant data that is kept in the cloud with your Web & Activity setting, which you can remove separately or set to auto-delete.