This weekend I strangely found myself trying to come up with excuses to go to the bathroom. Not because I’m having medical issues, but because I got stuck on some of the great shows on Quibi, The Stranger, The most dangerous game, Punk’d and Flipped. With the current corona situation, I’m not commuting anywhere, so the only place in my two-room apartment where I consume content on my mobile is when I’m in the bathroom. So after spending an undisclosed amount of time in the Quibi app I wanted to share some insights and thoughts around the newly launched app:
Mobile first — good start but where do you go from here?
Quibi has a great UI, but some improvements could be made with regards to how they are presenting their content. Even if it’s all short-form the viewers usually need a bit more information before deciding to start a stream, so small previews without sound could help. Another great feature is the UI-function in the video player where you flip the phone and it automatically adapts, that is amazing! Even though the feature is not super useful, it’s a feature that I appreciate and it creates trust to the brand around the fact that they care about their content. They have released great shows with great productions, with an incredible high quality, and this got me thinking even further…
Initially, lots of people from the industry, both users and experts, started to complain about Quibi and the features the service was missing such as casting the video to a big screen TV (something they now released is on the roadmap). But after closely reviewing the service I can say that I see a lot of potential. Very few streaming services out there had a perfect launch with the perfect amount of content. Netflix, Apple TV+, Disney+ even Youtube all had to start with few titles and features missing, bugs, sign up issues or server issues. But with time they all matured.
Quibi is filling a really important gap in the world of streaming with its focus on the mobile device and high quality content. This makes Quibi a strong competitor to Youtube, TikTok and similar services. Because it’s gotten to the point where youtube and TikTok now have sooo much content, that bad content can now disguise it self as good content with nice thumbnails and click bait-titles, making us start a clip and then half way through realize how low-quality the content actually is. So Quibi has the opportunity to provide me with a channel-like experience similar to TikTok and Youtube, filled with content that I can trust is of high quality.
The best of two worlds, “UGC” and high-quality production
Quibi’s quality of content gives them a competitive edge, but the size of the library and the rate of which they can release new shows to keep users engaged will be a challenge. At the same time, the power of YouTube and TikTok lies in the scalability of having User-Generated Content(UGC) where they don’t have to produce everything themselves. Quibi could adopt the same model with some caveats; keep content-quality high, create a platform where creators can host their shows/series at Quibi by applying and being approved by the great and experienced content team from Quibi. Creating a kind of “controlled UGC-platform” where I can trust that the content being put in front of me will have a great quality. Because I know that many people are tired of browsing the endless pit of youtube or TikTok content and then suddenly asking yourself, why did I just spend 4 hours watching people jump-scaring cats?!
Quibi + Apple?
With this type of focus on quality for the content and such a clear mobile device-focus there is only one outcome for the service: Apple will buy Quibi for couple of billion dollars sooner or later. Why? Well if you look at the Quibi productions they are perfect for a company like Apple that have such a device-focused streaming service like Apple TV+ where the Quibi content is already perfect for the iPads and iPhones, and with the seamless billing integrations and bundling with Apple devices, its a perfect match. And we all know that Apple will pay big dollars to keep you in the Apple-universe, so the main KPI going forward will not be the amount of Quibi app-downloads, but how much time users are spending inside of the app. One question in this proposed marriage that would need to be answered is how Apple would view the AVOD-package launched by Quibi, we all know Apple isn’t a fan of exposing advertisement to their users as it disrupts the user experience (although there is a lot of room for improvement in harmonizing the emotional profile of advertisement to make the AVOD experience more enjoyable, but more of that in another post)
The Quibi-content fits perfectly in the Apple TV+ service (photoshopped)
But there is one thing that both Apple and Quibi needs to think about if they are to become a successful player in this market. Even if both players produce high-quality content it does not mean that all their content is perfect, some of their content might not be intriguing enough to engage viewers over a longer period of time. The upper hand both of this players have over Youtube is that their content is emotionally loaded, what i mean with that is that the content is designed with colors, camera angels, cuts, sound, music to affect us emotionally. While a youtube clip is not as well designed to do that, hanging with Pewdipie is like hanging with a friend and that is very different from a production with a cinematic feel to it.
The youtube autoplay is on the right-side, it shifts from keeping you in the same subject-bubble and other subjects you showed interest to. So you can go from cooking to gaming esports.
So if Apple and Quibi starts understanding and combining the analysis of the emotional structure of their content library with tracking usage data to see what we choose, when we choose it, when we stopped watching a show, what type of shows that drives us back to the service, they would be able to set a strong foundation for both how to produce better content and how to create a really good discovery experience. Discovery is not only important for a better user experience, it’s also a tool for understanding your audience and viewers better by presenting them with different options and seeing what choices they make, to ultimately understand why they made those choices. The recommendations on Youtube works in a similar way by presenting content through an autoplay-function, then analyze how you change your path to present you with better recommendations in the future. Quibi have an opportunity to not only use these insights to create a really engaging user experience, but also to ultimately understand what type of content they need on the platform today, and in the future. Creating a user experience where the viewers are presented with choices is a great way of understanding your users. Especially if your content is emotional, the users will, by choosing different paths, let you know if they want stressful content in the evening or late at night. Or if they want colorful and joyful content during the morning or the evening. This data is incredibly valuable and crucial for Quibi and Apple if they are to keep the competitive edge, that I argue they have, and can get more of.
Originally published at https://www.linkedin.com.
Sound design greatly contributes to the feelings of the audience while watching a movie. Valuable information about the emotional aspects of a film is contained in the sound design, so understanding how it works and what it tells you is a key aspect to grasping what your content is all about.
Vionlabs has developed an Emotional Fingerprint API that instead of using sub-par metadata to try to provide users with better content, measures the emotional aspects of various key components, such as sound, color, and pacing. This enables OTT providers to better understand their content, which in turn ensures that they can make better content recommendations.
Since we know what benefits a deeper understanding of the sound of a movie can give to OTT services, let’s examine how sound, just like color, can be used to alter how a cinematic moment is perceived emotionally by its audience.
The Truth Behind Great Sound Design
Looking at the clip from the episode “The Long Night” in the final season of Game of Thrones (2011–2019) listed above, your mind is instantly drawn to what you can see. We observe the whitewalkers climbing the wall and the people waiting in suspense trying to defend it. We see fire turning the environment into a murky brown color. But if you think about it, the story is actually mostly told through the sound. This happens to be the darkest Game of Thrones episode, sometimes rendering its visual aspects too dark to make out, thus putting greater emphasis on the sound. The clip features sounds of people calling out to each other to defend their wall, the grunts of the whitewalkers, and the sizzling of fire. The music has a distinct ticking in it, alerting us that time is of the essence.
And then, timestamped at 00:20 in the clip, we also hear something called a Shepard tone. An effect used quite heavily by filmmaker Christopher Nolan, this sound creates an illusion, a scale that seems to be rising forever without ending. This effect is created by superimposing different tones separated by octaves on top each other and tinkering with the volume of the frequencies. The result is what one might call the sound of suspense; and in this particular Game of Thrones moment it is key to the emotional buildup we feel as an audience.
The truth behind great sound design is that it helps you truly immerse yourself into the story, so much so that you might not even consider that it is there. Whether it be the tumultuous thunder of weaponry in Saving Private Ryan (Spielberg, 1998), the sound of a lightsaber in Star Wars, or the theme song accompanying Indiana Jones as he goes on his adventures, the sound in a film helps create realism, adds credibility to the story and above all, sucks you into the emotional state of the moment.
Sound Throughout Film History — The Short Version
Throughout film history, sound has always accompanied our motion pictures. Even during the art form’s first trembling steps, and what would follow as the silent era, films were never fully silent. While some of the leading industry innovators like Edison had an interest in sound, the technology wasn’t advanced enough to entertain a larger audience. So in the silent era something else emerged to keep viewers engaged — music. Often played by a live musician hired by the cinema, it was the music that made all the difference. It was there to set the tempo, it created the mood, and told the audience what they should feel at any given moment.
In Japan, where filmmakers took a different approach from its Western counterpart, they even went so far as to hire benshi — men who would stand alongside the screen and narrate what was happening in the film. The benshi gained a lot of popularity among crowds and would make sound effects, take care of all the dialogue, and be the true performer of the film.
When The Jazz Singer (Crosland, 1927) hit theaters in 1927, it surprised audiences to hear the actor Al Jolson’s chatty adlibs in between his singing. They instantly wanted more; and while most features included a mix of prerecorded audio and the various story-telling techniques of the silent era, just two years later in 1929, 75% percent of movies had turned into talkies.
Since then, storytellers and filmmakers have been doing their utmost to challenge how we hear and feel through sound in movies. In Francis Ford Coppola’s Apocalypse Now (Coppola, 1979) for example, sound designer Walter Murch forced both theaters and Hollywood to adopt a six-speaker surround system by changing the way sound was edited and mixed. Murch created sound that was able to travel around the movie theater, allowing the audience to experience the film through the traumatized war-heavy mind of Captain Willard. It is clear that sound has now become a major component of the filmmaking process.
Sound Design and its Components
The sound design of any movie can be broken down into several different parts. Filmmakers are able to manipulate each sound component to contribute to the emotional feelings created in that moment.
Usually, upon filming, the only sounds recorded would be the dialogue. Other key components, such as music, weather, the sound of objects, or a character’s interaction with them is all added later. The reason why filmmakers do this is because in order for the audience to fully grasp the emotional moment of a scene, the sound is hugely important. Being able to manipulate the sound after filming is therefore major in creating what you would like to convey. Dialogue, sound effects, music, and mixing all come together to tell you the story.
In a film there is diegetic sound — which is sound heard by the characters inside the filmic universe. Examples of this is dialogue, sound effects, or music played on the radio, like in the infamous scene in Reservoir Dogs (Tarantino, 1992) where a cheerful song accompanies Mr. Blonde as he performs an act of violence.
There is also non-diegetic sound — which is sound not heard by the characters — often added to make sure the audience feels a certain way. The biggest example of this is music, like the orchestral track featured in Steven Spielberg’s Jaws (Spielberg, 1975), a tune that quickly ensures that suspense and fear are added to the shark’s appearance on screen.
The Emotional Orchestrator of Film — Music
Ever since the beginning of cinematic history, music has been a major component of creating emotional and suspenseful stories. Music can be considered the emotional orchestrator of film. It suggests to the viewer what they should be feeling. For example the piano tune created by the AI in Her (Jonze, 2013) expresses the bittersweet technological love felt between the characters, whilst the intense violins in the Psycho (Hitchcock, 1960) shower scene convey precisely how horrifying this encounter is to the audience.
By attaching a signature melody to the story, filmmakers are able to use our expectations to quickly alert us as to what we should be feeling. Take a look at Darth Vader for instance. The menacing imperial march sets the tone. After hearing it a few times throughout the story, even someone not looking at the screen would be able to tell when the Imperial Empire is present and strong. Without it, Darth Vader would just be a man breathing heavily in a black suit.
The Deep Emotional Cues of Sound Design
But we also live in a world where nonmusical sounds are connected to deep emotional cues. The sounds of cars honking, birds chirping, or waves crashing against a cliff all induce feelings.
Since most of what you hear on screen is actually added in post-production, it gives the filmmakers the freedom to use sound effects to tell the story emotionally. In Gravity (Cuarón, 2013) the sound designers really explored how being inside a vacuum-like environment would affect the sound. In Eraserhead (Lynch, 1977) dreary, abstract sounds not reflected on screen were used to create a soundscape, a mood, and an atmosphere. In Brokeback Mountain (Lee, 2005), different types of wind were used to signal the isolated characters’ shifting emotional states.
Since sound effects provide a sense of realism to what we see on screen, each sound has to be carefully considered. Footsteps will sound different depending on whether the scene is happy or scary, if the character is stomping or sneaking. Foley artists — professionals who recreate sounds for moving pictures — therefore work hard to introduce different elements to enhance a sound in order to make it believable and just right for the story at hand.
And to go even further, sound designers also frequently use something called “sonic metaphors”, where a different sound is added to the mix to create emotional depth. In Mad Max: Fury Road (Miller, 2015), as the War Rig roars through the desert, whale sounds were added to the mix to reinforce the feeling of this vehicle as a metaphor of a heavy, slow animal. As the truck takes a hit, the whale groans. It is not noticeable unless you know it’s there, but it adds emotional depth to the sound of the vehicle. This is a powerful tool that keeps the audience engaged.
How To Become Cutting-Edge by Using Vionlabs Emotional Fingerprint API To Measure Sound
So in times where more streaming services are launched than ever, how can understanding sound be utilized to help OTT providers stand out, become cutting-edge, and reduce churn?
The problem with today’s content data is that it is not detailed enough. It simply fails to detect the nuances necessary in order to give good content recommendations. While it makes sense that the metadata for I, Tonya (Gillespie, 2017), a black comedy about figure skater Tonya Harding and the event that ended her career, and Blades of Glory (Gordon, 2007), the sports buddy comedy where Will Ferrell and Jon Heder decide to become an all-male figure skating duo, would be similar, the two films might not attract the same audiences. But, when using metadata, they would most likely be linked to each other.
Vionlabs’ Emotional Fingerprint API goes further than this. It measures and examines thousands of factors in a film including colors, pace, audio, object recognition, and much more. These factors are distilled by AI into a fingerprint that encodes the emotional structure of the content. Such fingerprints will allow the industry to make better informed decisions, and enable the consumers to come into contact with content they would actually want to consume.
Originally published at: https://medium.com/vionlabs-tech-blog/how-sound-design-triggers-emotion-bf052d3da2a9
Read our other blog posts: