Website server logs that count unique cookies to measure unique visitors are likely to be exaggerating the size of the site’s audience by a factor as high as 2.5, or an overstatement of 150 per cent, a study has found.
comScore released the results Tuesday of a study analysing the validity of using cookie-based data to measure the number of unique visitors to individual websites or to gauge the number of unique users that were served an ad by an ad server. The study, based on an analysis of 400,000 home PCs included in comScore’s US sample during December 2006, examined both first-party and third-party cookies.
A cookie is a very small text file inserted on a user’s computer by a Web server and is unique to that computer’s browser. Cookies are often used by web servers to identify users and for authenticating, tracking and maintaining specific information about users. First-party cookies are those left on a computer by a website that has been visited, while third-party cookies are those left by a domain different than the site being visited, such as an advertising server that has just delivered an ad to a computer, or certain third-party tools used to measure site traffic.
For the purposes of this study, comScore analysed the cookies from one prominent Web property and one third party ad server, each representative of the total US Internet audience and each reaching well in excess of 100 million Internet users every month. The study examined the degree to which users cleared cookies from their computers, thereby causing servers to deposit new cookies and potentially leading to overstated estimates of unique users when relying on cookie-based server data.
The study observed that 31 per cent of US Internet users cleared their first-party cookies during the month. Within this user segment, the study found an average of 4.7 different cookies for the site. Among the 7 per cent of computers with at least 4 cookie resets, comScore counted an average of 12.5 distinct first-party cookies per computer, accounting for 35 per cent of all cookies observed in the analysis.
Using the total comScore sample as a basis, an average of 2.5 distinct first-party cookies were observed per computer for the site being examined. This indicates that website server logs that count unique cookies to measure unique visitors are likely to be exaggerating the size of the site’s audience by a factor as high as 2.5, or an overstatement of 150 percent.
“While past studies from other research companies have shown a similar proportion of computers that clear their cookies, the comScore study is the first to highlight the disproportionately high percentage of cookies represented by those computers,” commented Dr Magid Abraham, President and CEO of comScore. “For example, with just 7 per cent of computers accounting for 35 per cent of all cookies, it’s clear that a certain segment of Internet users clears its cookies very frequently. These ‘serial resetters’ have the potential to wildly inflate a site’s internal unique visitor tally, because just one set of ‘eyeballs’ at the site may be counted as 10 or more unique visitors over the course of a month. The result is a highly inflated estimate of unique visitors for sites that rely on cookies to count their audience.”
comScore’s analysis of third-party cookies revealed an average of 2.6 distinct cookies per computer in December, indicating a similar rate of overstatement as the first-party cookies. For those computers where at least one cookie reset occurred, the number of third-party cookies observed was slightly higher than first-party cookies at 5.5.
“Panel-based measurement is needed to accurately measure the size and characteristics of a site’s online audiences,” said Richard Castellini, Vice President of Consumer Marketing at CareerBuilder.com. “The comScore study quantifies the magnitude of the discrepancy one can encounter using log files, and highlights the significance of this issue to our industry.”
“The comScore study clearly demonstrates the limitations of measuring Internet audiences based on cookies,” said Mohanbir Sawhney, McCormick Tribune Professor of Technology at the Kellogg School of Management, Northwestern University. “To measure audiences more accurately, it is important to link visits to unique individuals, not unique cookies. As privacy programs become more entrenched, cookie-based audience counts will get even more unreliable.”
“In 2005 while at JupiterResearch, I reported the results of a survey showing that 39 per cent of Internet claimed to delete their browser cookies on a monthly basis,” said Eric T Peterson, well-known web analytics consultant and author of “Web Analytics Demystified.” “The data from the comScore study, especially the findings regarding first-party cookies, clearly highlight the risk to cookie-based measurement.”
“The integration of Internet advertising into multimedia marketing plans requires accurate reach and frequency data,” remarked Karen Francis, Chairman and CEO of Publicis & Hal Riney. “It’s clear from the comScore study that cookie-based counts from site server logs are not necessarily providing the level of accuracy that is required.”
