We humans are complicated mammals and we are no longer satisfied with just texts and words to express our emotions and opinions.
From the primitive time, our ancestors knew how to use symbols and other linguistic elements to communicate and preserve cultural values. Since then, text has been a global code representing for what we want to say and which emotional state we want to express. It has also powerfully connected us with people all around the world through social media.
However, as said, we are no longer content with just text. Thanks to the technological development, we have digital cameras and smartphones at our disposal to make our opinions more interesting and trustworthy. Photos and videos help amplify what we want to communicate to the world because people tend to react to visual content more. Here are some statistics convincing everyone (even you!) to use more visual materials for social media posts:.
- Posts including images can reach 650% higher in engagement rate (Inc.)
- An estimated 84% of communications will be visual by 2018 (Inc.)
- Social video generates 1200% more shares than text and images combined.(Wordstream)
- When people hear information, they’re likely to remember only 10% while if a relevant image is paired with that same information, people retain 65% after 3 days. (Louise Myers)
So we have come up with different ways to express our thoughts via photos and videos, which is a huge challenge for those working with Sentiment Analysis. Actually, you can bet that most Sentiment Analysis done with typical social media analytic tools is totally fu**ed up. Excuse my language. But true.
Sentiment Analysis of Photos
Instagram has now prevailed over Facebook and many users share beautiful photos, snapshots of their daily lives, and sometimes show off something to seek fame. In Instagram, using a photo and a succinct caption can tell the audience a story because everything in a photo visually depicts the context in which the caption is being written.
Sense of humour sometimes discretely lies in sarcasm and people love to use and read it because, well, it’s funny. While one is saying something in a post with ironic tone, a photo helps people to understand the joke perfectly and the exact emotion he/she is trying to express.
Let’s take one Instagram post as an example:
A post shared by MeiZhang (@meizhang6077) on
A typical sentiment analysis tool might consider this as a “positive” post, because it comprehends only the text that is written. “Lovely coat”, “new style”, “great idea”. Those texts points to “positiveness” for sure. But you need to add the image into consideration and really analyze what is being said here. By looking at the image with the hole in the coat, for sure, this is not a new style, and @meizhang6077 is not happy with her “new” coat that came with a hole. Sentiment=negative.
How about this one from Twitter:
The text would clearly indicate that the post is positive. Social media analytics tools in the market will understand this completely wrong! You need to take the image into the context. When a human sees the combination of what is written in the image versus in the text section, the sentiment would be categorized as negative.
And one more example from Instagram:
If this is "well-medium" standard of KJoan Restaurant as the waiter said, I wonder how bloody the rare steak is... the waiter wished our kid a great meal with this and said there was no need to change. Our Jim also had a great time in the toilet then. Thank you so much KJoan! #customerservice #kjoan #kidmeal
A post shared by Susanne Rosenberg (@susanne_rosenberg88) on
In this post, @Susanne_Rosenberg doubted the quality of her child’s steak who ended up in the toilet after eating it at the KJoan restaurant. She blamed it on the stubborn waiter sarcastically. If you let social media analytics interpret this, it will categorize this post as “positive” due to texts such as “wished our kid a great meal", “great" and “thank you so much", which is totally opposite to what a human would understand. If KJoan restaurant is using any social media analytics tool, they would be thinking that there is positive discussions in social media but if this particular post goes viral, they are facing a crisis situation. Sounds tricky, right? This one, sentiment=negative.
While technology has indeed advanced, and even with the big hype with artificial intelligence, computers and algorithms are just not yet capable of analyzing the true sentiment of lots of posts like these. Human intelligence is still needed to truly understand the context in which this post was published.
Sentiment Analysis of Videos
In addition to photo images, videos have also become a popular method to spread one’s opinions. Many people make tips & tricks tutorials of products they use, whether it is a software or make-up.
Videos can be even more difficult for computers to interpret. In addition to “motion”, there is the tone of voice, facial expressions, body language, music, and the whole atmosphere that comes into the context. It is not enough for your Sentiment Analysis tool to analyze the texts that is written underneath the video post, but also understand the sentiment within the video.
Current State: Sentiment Analysis is totally fu**ed up. It really is.
In the business world, it is extremely critical to determine the correct tone in the image and videos of your products when your customer posts and writes something ambiguously. If you rely on these social media analytic tools to do customer posts’ sentiment analysis for you, well, you are fu**ed up! You will get a completely wrong conclusion from the analysis and miss a chance to either satisfy unhappy customers or cap on the opportunity when a seemingly negative post is actually positive. If you miss reacting to a negative post because your analytic tools misinterprets it as positive post, you might be looking at a crisis situation by the negative post going viral. Stop it before crisis happens.
Remember, human intelligence still provides the most accurate analysis of people’s sentiment.