Dark Data is the phenomenon of consumers increasingly opting to share things away from the public platforms that social media networks offer, to more private confines of social messaging apps, and similar services.
Facebook’s frequent appearances in the headlines in the last months will likely contribute towards this trend growing.
Before I get into the specifics it is important to mention that Dark Data is by no means exclusive to marketing (although I will focus more on the marketing apsects in this article). Dark data is can exist in almost any context.
It is often referred to as data that exists somewhere (usually) on the internet, but an individual in need of the data might not know it exists, or the sheer quantity of data can make it difficult to process.
Gaining insights into dark data can be challenging since it is no direct way to access it, unlike consumer information that is neatly stored in a database.
In other words, it refers to usually un-structured data that is not leveraged for business (or other) operations.
It should also be mentioned can also be related to a business’ internal sharing (or lack thereof) of information with its employees.
An example would be if you want to find the number of your favorite hairdresser in a phone book, but instead a nice ordered phone book all the pages are random and un-numbered.
The data is still there, but it would take significantly longer to find.
In marketing, Dark Data, or Dark Social, is the information that is consumers share, but marketer’s cant directly track.
Marketers often pride themselves in knowing their consumers’ specific interest, via the data that can be gathered from social media.
But to find out how a consumer feels (how to get this dark data) is private platforms we have to take a different approach.
The problem for marketers is that information that we share on more private platforms (like social messaging apps) can usually not be used to contribute towards our digital profiles.
Sharing things with the people closest to us has always been different to the way we share things that might interest us.
I’ll give you a peek into my social life on Facebook for instance.
I use Facebook’s Groups function to plan holidays with my friends and we often compare flights, places to stay etc. on those groups.
This data can be collected by Facebook and can help them determine what kind of things I’m interested in (and subsequently I can be targeted by relevant adverts).
However, when I’m talking to some close friends and I want to show them more personal content e.g. inside jokes (or, let’s be real, silly animal videos), things referencing a discussion we might have had, or maybe a present/surprise we are planning for one of our other friends, I’m more likely to simply copy the URL and send the content to them via a social messaging service such as WhatsApp or Facebook Messenger.
Now the example I gave above is just of me personally and might not reflect you, but there is a general trend in consumers to share private content on private platforms, and not on public platforms.
According to HootSuite, dark data actually accounts for around 80% of the content that we share. That’s a lot. This specifically is a slight deviation from dark data and was coined ‘Dark Social’ by Alexis C. Madrigal on The Atlantic Daily.
Visits to websites from links that have been directly shared obviously still get picked up on by analytics tools, however, they will be classified as a direct visit, not as a social referral.
While this is certainly not the end of the world, it can make things difficult for businesses that rely strongly on the performance of their organic reach, as it makes measuring the effectiveness somewhat less accurate.
So how can you measure it?
Probably the most common way is to use URL shorteners (like the one used to source that image above).
The social media whiz kids among you will already know the benefits these provide, apart from making your tweets look that little bit sexier.
I’ll use Hootsuite as an example since it’s a popular tool among many social media marketers (myself included). The ow.ly shortener (this might be different if you live outside the UK) provides you deeper engagement analytics that allows you to track how clicks your URL receives.
There are also a number of other, more complicated tools that can track consumer engagement via some clever code or some more sophisticated URL tweaking.
But assuming, you don’t have a classrooms’ worth of analytic team members at your fingertips, how do you deal with this?
Alternatives for measuring Dark Data & Dark Social
One thing most industries have in common when it comes to dark data collection is the use of machine learning.
Think back to the hairdresser.
If you have an algorithm that can read all those pages in a few minutes your life would certainly be a lot easier.
While there is no algorithm to analyse the content being shared via dark social, there are algorithms that can analyse different things!
In a broader marketing context, this could mean that analysing consumer behavior during UX on a website can be one way to tackle this.
When looking at Dark Data, the Watson Platform can be used to analyse if the consumer is, for instance, struggling with a purchase, if the consumer has an account on the website, being able to determine which products/services the consumer researches and how long and how many times he/she does so.
While data collected this way is largely unspoken by the consumer, it might give insights into what their actual interests are versus what they choose to share on public platforms. Furthermore, this data is anonymised and tailored to each consumer individually.
The digital age brought with it a growing trend in the customer journey becoming increasingly non-linear and often spanning multiple channels and devices. Marketers need to embrace this shift and accommodate their customers.
AI and Chatbots such as the Watson platform are still in an infant stage, however they provide interesting insights into possible alternatives when it comes to analysing consumer behavior.
In conclusion, dark data, or dark social, provide a vast and often untapped sea of information. While marketers do not have direct access to this information there are ways to track some of this information via URL tweaking and smart code. Advances in AI such as the Watson platform also present novel ways for marketers to track consumer engagement entirely on the company website