The Vionlabs AI Video Story Descriptors

AI + Humans = magic. Content curation and discovery process on the next level

The streaming services are in a war zone, everyone is fighting for eyeballs and attention.

Lately there has been a lot of focus on algorithms and recommendations as user experience is the key to unlocking the loyalty from consumers, but content curation is an equally (or more) important part of the user discovery experience. The work behind content curation is a creative task for someone with an artistic mind that has seen a lot of movies and TV shows, but these types of creative tasks are very hard to scale without high-quality data and good data coverage. This is where AI can help you to scale this creative task, AI + Humans = magic.


That’s why we created Video Story Descriptors - high-quality keywords to enable better & faster content curation without taking away the professional’s creative opinion. The steps are easy:

  1. Combine keywords like "survival, underdog or revenge" with all of the other data from Vionlabs Fingerprint plus, like Moods and Genres to enhance and create more diverse curated lists.

  2. As your library updates and changes, all new titles will be automatically assigned to the correct list, allowing your start page to be dynamic and relevant week in and week out


The technology behind Video Story Descriptors

We have been training our AI for years to understand storytelling, a very complex and multifaceted problem that involves everything from training the AI to understand production techniques to story line to emotions. After years and years of research, we were finally able to train the AI to identify and understand the important components of a story. Through analysis of the audio/video files we have been able to capture and encode all relevant information related to storytelling with high accuracy into a single high-dimensional content embedding for each content asset. Our content embedding is then used as the foundational component for predicting keywords, genre, moods and much more.


The benefit of this approach:

  • Updates your library efficiently, automatically and accurately.

  • Giving you full coverage of quality data for your content library

  • Eliminating the risk that any title would be left behind without data

  • Enabling your full library potential by surfacing all content.



Whats different with Vionlabs' Video Story Descriptors?

We start from the user experience, and create descriptors as keywords to represent the story of the titles. These descriptors provide high accuracy and relevance for curating lists and improve search and discovery for viewers. The video story descriptors are describing the movie from different perspectives, such as the pace, colors, the core subject, the major event, the environment, location and more. Looking at what is important for the ingredients for that specific title, and with a big focus on the story. Another key differentiator is what we call "keywords weights" which helps you determine how important a concept like "love" is to tot he overall story. Every title has its own unique distribution when it comes to descriptors. Take the following two movies as an example, The Note Book and The Zookeeper's Wife:



The Notebook story is built around the love of Noah and Allie and the emotional twist related to Allie's mental condition (spoiler alert). While in the Zookeeper's Wife, the movie has elements of love, but the story is more built around the emotional sacrifices during world war 2. We can clearly see this in how the keywords are weighted in the picture above, which represents the content scorecard for each movie.



How can descriptors help content curation and leverage the user experience?

Streaming services can use these descriptors to automatically create appealing and relevant discovery lists for viewers. To exemplify this, we analyzed a library of +7000 titles and used our Video story Descriptors to automatically curate the full library into great-looking lists that highlights the diversity of the library. Here are two examples, “revenge” and “survival”:

All of the movies mentioned in the list have “survival” and “revenge” as significant descriptors, but they could be very different in other dimensions such as the pace, the event, the environment etc. The viewers will not feel the recommendations are provided from the linear perspective, e.g. getting recommended a war movie after watching one, but rather that the selection of the recommendations are diverse but still fit into a selected theme. In that way we could increase the possibility of viewers to find what they are looking for, and also find what they are least expecting, but will still enjoy. The benefit of using AI to analyze content is not only to keep accuracy and relevance at a consistently high standard, but also to scale quickly; Our AI-engine analyzes approx. 5-8000 titles per day.



Why are keywords important?

It is very important to organize your library in a way that makes it easy for your viewers to navigate and explore your catalogue, as this leads to increased customer satisfaction and ROI on your content investments. All content has an audience, but the content needs to be accessible and easy to find. If any of your titles in your library are lacking quality data or missing data it is a big risk that the title will never be exposed to your viewers and your ROI on your content investment will decrease.


What matters for user experience? The survey we did involving 2500 users from ages 18-65 earlier gave us some guidance:

  1. 30% of people struggle when they try to find something to watch as every search takes more than 21 minutes;

  2. 90% of people have high expectations on the platforms to provide good recommendations;

  3. Almost 70% of the people follow mood when deciding what to watch;

  4. When people have multiple choices, the details of the movie starts to influence their decision (descriptions, genres, cast info and more).


A good user experience means that the platform understands where the connection exists between each title and each viewer. This could be through genres, mood, emotions, topics, themes etc. You need high quality data to make these connnections.



The extensional value of the video story descriptors

The usage of the descriptors is not only just for curating the lists, but also to understand further why some titles are more popular than others by combining with the user data, to create a better understanding of your viewers and what type of content they view. Insights and analytics need good data input to be able to create understandable data points that can help you make conclusions and from there take action to further improve the experience of your viewers to maximize their entertainment value inside of your service.


Viewers will also be able to benefit from the descriptors directly via the search function. If a viewer searches by keyword “love”, then a list of the movies that contain a high keyword score of “love” could be shown, you could furthermore rank the search response after the relevance of the keyword "love" for the returned titles.


Content curation should be artistic, and not replaced by algorithms. Our video story descriptors will make the artistic workflow easier by providing in-depth content insights, while people who create lists could just focus on the artistic part of the work - adding the professional perspectives & great taste on top of the AI data.

AI + Humans = Magic




83 views