Hey Alexa, what’s my PIN? Cambridge researchers show voice assistants can detect what users are typing on their smartphones by the sound of their fingers tapping the screen
- Researchers built their own version of a smart speaker to access the recordings
- In commercial products the recordings are stored securely for safety
- But study shows that if someone can access these they can decipher passwords
- When typing in a five-digit code within 20cm of microphone the researchers correctly guessed a password 76% of the time with just three attempts
Smart speakers like Google Home and Amazon Alexa could be used by criminals to listen to and decipher a password or PIN being typed in on a nearby phone.
Researchers from the University of Cambridge built their own version of a smart speaker to closely resemble those which are commercially available.
Sound recordings from the gadget were inputted into a computer for analysis and experts investigated if the sound and vibrations caused by typing on a smartphone screen could be used to guess a five-digit passcode.
When the phone was placed within 20cm (7.8inches) of the custom-built device, the computer was able to guess the code with 76 per cent accuracy in three attempts.
Scroll down for video
This graphic outlines the general flow of the experiment. A user inputs their passcode and the vibrations and sounds are detected by the speaker and could be analysed. If an attacker gets hold of these, they can tap into the audio and unpick the code
Pictured, the location and layout of microphones on the Amazon Echo (left) and the layout of the custom-built device built by the Cambridge researchers (right)
But the accuracy of the code-breaking diminished the further the phone was from the microphone.
For example, when the phone was positioned 50cm (19.6 inches) from the speaker the accuracy plummeted to just 20 per cent.
Ilia Shumailov, lead author of the study, said: ‘Using two different smartphones and a tablet we demonstrated that the attacker can extract PIN codes and text messages from recordings collected by a voice assistant located up to half a metre away.’
Smart speakers like the Echo and Home are always listening for a wake word which triggers them and activates their functionality.
Once this is detected the audio is recorded and goes through to be interpreted by artificial intelligence.
However, access to these recordings are heavily restricted, with only the Alexa account holder able to review them in the app. They can also be deleted by the user at any time.
Therefore, any criminals wanting to hijack a smart speaker with the goal of stealing device passwords would have to tamper with the device physically or hack into the server to access the recordings.
The level of brazenness in this endeavour may outweigh the benefits to criminals and the study is therefore more useful as an exercise in acoustic application than cybersecurity.
Sound recordings were inputted into a computer for analysis and experts investigated if the sound and vibrations caused by pressing buttons could be used to decipher a passcode (stock)
‘Right now, it’s unlikely that people would use our attack. However, the world changes quickly and sensors only get better,’ Mr Shumailov told New Scientist.
The findings of the new research, called ‘Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant’ are published on pre-print site arXiv.
Last year, the same researchers found a gaming app can steal a banking PIN by using the phone’s own microphone to pick up screen vibrations as a user enters the code.
In a test, their algorithm could correctly guess 31 out of 50 four-digit login pins in just 10 attempts based on recordings made of the participants as they typed.
These potential attacks would likely begin with the accidental download of malicious software — so users should keep themselves safe by only using trusted apps.
Limiting microphone access to only those apps that need it will also help to make your smartphone more secure.