The Internet has more dimensions than you can see. Even your own website has dimensions that are not immediately visible to you, and in these secret grey spaces hide trackers and intruders that see everything and remember all.
Here, no privacy exists.
These phantom intruders send back what they learn about you to their masters, who have become the richest companies on Earth by selling your data to hungry marketers in a run-amok industry based on behavioral advertisement.
But there is a way to make these dimensions visible, and to cast a light on the hundreds of potential trackers that lie in hiding on your website and invade your users’ privacy. The technology is called a cookie scanner, and the Cookiebot cookie scanner finds all cookies and all online trackers – every single one.
In this article, we’ll take an up-close look at the cookie scanner technology at the heart of our Cookiebot consent and compliance solution. What exactly does a cookie scanner do? How does it operate? Ways to understand the technology, if you’re not a techie, and important happenings in the fight for privacy.
Stay with us.
A cookie scanner is (as implied metaphorically before) a piece of technology that crawls through your website in search of trackers that invade peoples’ privacy when they visit your domain.
The Cookiebot scanner is a unique piece of technology that forms the cornerstone of the Cookiebot consent and compliance solution. Our web cookies scanner finds all cookies and all trackers without exception. It took our founder and CEO many years to develop this proprietary technology that is now becoming the de facto industry standard in protecting privacy.
Try a free scan of your domain today.
Cookies are small text files that a website places on your browser. When people visit your website, cookies collect data about them. Cookies are different, though, and far from all are privacy intrusive.
Some cookies are what we call necessary cookies; they serve your website’s most basic functions, making sure that it runs as intended. Others are preference cookies that remember user choice of language, currency setting or log-in details for optimal user experience.
Then there are the statistics cookies that track users and collect different information about them and their online behavior. This data is used to inform website owners on the analytics of their site, a precious commodity for companies looking to understand their sales and marketing strategies.
But marketing cookies harvest vast amounts of data for third-party companies that assemble comprehensive profiles on people and use these to target them with advertisement.
This is known as behavioral advertisement and is a multi-billion-dollar industry.
Some of this data can be sensitive data, such as health data, and a lot of it is collected by companies like Google and Facebook without public knowledge of what they actually do with all this data!
"Can you scan my website for cookies", you might ask. Yes we can.
Here is how it works.
The Cookiebot scanner simulates human beings and their behavior online in order to lure out and detect all of the cookies and trackers in hiding on a given website.
A website cookie scanner cheats the trackers into thinking that a real person is scrolling on the website they are leaching on, and in that way baits the trackers to come out form their hiding and show themselves.
Technically, what this means is that the cookie scanner performs fully-rendered simulations of multiple users (7-8 on average) visiting a website and their behavior on that website, which includes scrolling up to 10,000 subpages, clicking all links, menu points and buttons, moving cursors around, as well as playing and pausing embedded video and audio content.
Basically, the cookie scanner exhausts all of the technically possible options on a website through simulated user interaction. Sort of like rustling a bush to see all the insects crawl out of their hiding.
During these simulated sessions, the website cookie scanner monitors all network traffic between the website and the “browsers” of the simulated users, as well as any traffic sent to other websites. The cookie scanner uses this data to identify all the trackers that are present.
Once the cookie scanner has scanned a domain and all of its subpages, the cookie scanner then catalogues all of the identified trackers by their –
The latter – the purpose of a tracker – is not something that the cookie scanner alone can determine, which is why the Cookiebot research team is constantly working on classifying trackers according to information provided by the third-parties themselves, either on their websites or in response to our direct inquiries.
If no such information is forthcoming, the tracker will be categorized on the basis of its technical properties and the available knowledge about the business model for the third-party company controlling the tracker. The next time the cookie scanner encounters this tracker, it will categorize it accordingly.
Finally, this vast knowledge that the cookie scanner generates is stored in Cookiebot’s cookie repository of more than 22 million trackers, which have been classified and ascribed more than 3,500 unique purpose descriptions.
The Cookiebot cookie scanner finds all cookies and online trackers technology present on a domain. This is a strength in our product that we are very proud of at Cookiebot, because it entails the potential for real and true GDPR compliance.
Of the most common tracking technologies that our web cookie scanner finds are –
All of these trackers can be found on a website, even if the website owner is not aware of their presence. They usually sneak their way in through analytical tools, social media links, embedded videos, and many other website add-ons.
If you have a website within the EU, or if you have a website outside of the EU that provides services to EU citizens, your domain must be compliant with the General Data Protection Regulation – the EU law that came into effect in May 2018 and mandates website owners and operators to obtain prior consent from their users before any processing of user data.
Using a cookie scanner is a vital tool in GDPR compliance – in fact, it’s almost impossible to be GDPR compliant without it. Why? Because consent cannot really be obtained from your users without informing them of all the tracking technology present on your domain.
Real GDPR compliance entails that –
It is impossible for most website owners and operators to reveal all tracking technology on their domain by their own hand. That’s why a cookie scanner becomes the vital and necessary tool to be compliant with the law.
But the cookie scanner technology is about more than just GDPR fines and penalties. In the recent year, the cookie scanner technology has shown itself to be an important explorative tool in investigative journalism – as a revealer of the hidden things of the Internet that invade our private lives and democratic institutions.
Earlier this year, we at Cookiebot released a report on the ad tech surveillance of public sector websites in major EU nation states, in which we used our cookie scanner technology to analyze tracking that occurred during simulated visits to thousands of pages across official government websites and public health service websites in all 28 EU member states.
Cookiebot found over 100 advertising technology companies systematically and invisibly tracking EU citizens when they visit their governments online. Our cookie scan report made news headlines globally, including TechCrunch, BBC and Financial Times.
More recently, findings from the Cookiebot cookie scanner was used in reporting by the Washington Post on data trackers placed invisibly on children’s websites in the US.
Cookie scanners, like our Cookiebot scanner, reveal the third-party trackers that harvest user data for the purpose of profiling: the method by which cookies and trackers on websites harvest user data in order to collect data points of personal information and assemble them in comprehensive profiles on users, ranging from your political beliefs and geographical location to eye color and health, sexual orientation and financial information, and so on.
These profiles are used in what is known as behavioral advertisement.
Contextual ads work by presenting advertisement relevant to a particular search inquiry. You go on Google and enter something into their search engine, and Google will display ads that are relevant to your search. If you searched for rock-climbing shoes, ads for rock-climbing shoes will appear.
No big deal.
Behavioral ads, on the other hand, are non-search ads, i.e. they don’t require the context of your particular search inquiry to serve you advertisement online. Rather, they are based on collected and accumulated information about you as a person and served to you on your preferred social media platforms, as ads in online papers and magazines, and countless other websites you visit every day.
Big Ad Tech is a multi-billion-dollar industry relying on the collection and selling of data about human beings and their individual and collective behavior – the commodification and monetization of private inner lives for the benefit of a small group of the wealthiest companies on Earth.
However, 45 percent of companies using behavioral ads saw no significant benefit from them, and 23 percent said they actually caused a decline in revenue, as reported in the New York Times recently in an opinion piece by Gabriel Weinberg, the chief executive and founder of the search engine, web browser company and Google-alternative DuckDuckGo.
Perhaps behavioral advertisement isn’t the miracle that Google and Facebook seem to be promising. Perhaps our data, mined and harvested and commodified, is just for the benefit of the powerful companies themselves. In which case, protecting privacy online becomes a no-brainer.
That is what our cookie scanner does. It protects privacy.