How Spotify Already Knows Your Next Favourite Song
So I was on the train this morning, listening to my “Discover Weekly” playlist on Spotify. I was sitting there being blown away by it’s crazy accuracy, and thinking “damn, how did it know I would like that song?”.
Discover Weekly is a playlist of 30 songs, that you haven’t listened to before. They are unique to you and your taste in music. And Spotify generates a new Discover Weekly each and every Monday.
Many of us are aware that companies like Spotify, Facebook and Twitter use our data to “enhance” our experience on their site. In Spotify’s case, I’m okay with this! But between your data and your new favourite artist, is an algorithm, a machine learning algorithm.
But I’ve always wondered, exactly how Spotify could suggest songs which had the same FEEL.
How on earth does an algorithm learn THAT!?
I couldn’t help myself – I had to find out.
But first, what IS Spotify?
Spotify is a music streaming service with over 180 million users. And just like Amazon suggests books, Facebook suggests friends, and Netflix recommends shows, Spotify recommends songs. With 10’s of millions of songs, it is quite the challenge to accurately suggest music to users!
When Spotify realised that it’s users were browsing other’s playlists on search of new music, they thought – “why not automate this?”.
So their idea was – why not put all of these playlists into a machine learning algorithm and predict songs for users. This resulted in a much-loved feature of Spotify – it’s “Discover Weekly”.
And all of this is thanks to a term that I’m sure many people are familiar with – a machine learning algorithm.
From playlist to prediction
Taking a brief step back, I will start by clearly defining WHAT exactly an algorithm IS.
An algorithm is simply a sequence of steps or calculations, that is performed to answer a specified problem.
(Although the whole point of algorithms is to be precise, it sure is an ambiguous definition!)
This new buzzword, “machine learning”, is just the use of algorithms to predict or classify something. A machine learning algorithm takes in a bunch of data, performs specific steps, and outputs predictions.
A machine learning algorithm “learns” from the input data using statistics, to build up a knowledge base (or a model), and then applies this model to your data of interest.
What Spotify does, is it takes all of the playlists that user’s have created on Spotify, and uses this as the input data. It also deduces what it calls your “taste profile“, based on your song history (Unfortunately, this is not available to the public!).
Using these two pieces of information, Spotify can predict new songs for you, every week.
Your data as input
For those of you interested in a more in-depth answer, Spotify uses 4 main approaches to bring you new songs every Monday.
- Natural language processing (NLP) models, which analyse text i.e. giving songs with similar titles!
- Audio models, which analyse raw audio tracks, allowing Spotify to find songs with that same FEEL.
- Your taste profile, which is something Spotify builds based on your play logs
- Collaborative filtering.
The first three kind of make sense, but it is collaborative filtering that is a bit more interesting, which is best explained with an analogy.
Say that I am a student, and I like to purchase 2-minute noodles, coffee, and pens.
Then say that you are also a student, perhaps in a different field, but could be considered ‘similar’ to me.
If you normally purchased a lot of pens and coffee, collaborative filtering would suggest 2-minute noodles to you.
You may or may not like noodles – but this is how collaborative filtering works! Where my shopping list, is the accumulation of millions and millions of user playlists.
Machine Learning Magic
And specific techniques are used it different applications. For example, a collaborative filtering approach may be more useful for a genetics application than an NLP. Where NLP is better suited to helping scientists summarise millions of scientific papers. But of course, scientists don’t have much of a use for audio models or your taste profile.
Although at first it seems a bit creepy, I feel that this really does a service to smaller, up-and-coming artists. Machine learning can connect musicians with the soon-to-be-fans, which would have never otherwise come across their music.
For the interested learner…
- Some good advice on how to hack your Discover Weekly.
- This article goes into more detail with the algorithms behind Spotify’s machine learning.
- If you’re interested, the open-source software Spotify uses is called Kafka.
- For more information about how much of your personal data Spotify keeps, and how to download it, click here.