Why comScore is losing its reputation for measuring digital in Asia

Asia is gaining a strong reputation worldwide for the rapid development that technology and internet are enjoying in the region. Much information and insight of the region is derived, like any other industry, from analytics and reports that consultancies publish.

These reports are typically reported on launch in traditional media, and both in Asia and western markets they wind up in presentations, become the focal points of blog posts (like mine) and, thanks to the internet and Google, are used as proof points for a great many constructing arguments, business plans, article, etc.

Which is why I have one request to make, comScore… please stay out of Asia until you start measuring your statistics properly!

comScore is one of the world’s most recognisable analyst firms. Specialising in technology, it has an excellent reputation and its data and reports are hugely influential, but the system it has in place for measuring data in Asia is flawed.

Currently, as is clearly marked in its charts and press releases, comScore data internet data in Asia counts private, mainline internet access only, i.e. it does not capture data from anyone using the internet on a mobile phone or at an internet cafe – both of which are hugely used access points for internet users across a number of countries.

As I blogged recently:

Smartphone ownership growth and the increase in mobile internet usage have revolutionised the way the web is used today. Browser habits are hugely different when compared to a small screen of limited size with limited multi-tasking/browsing. Furthermore, social network usages makes up a huge part of mobile internet activity, after all they were one catalyst that helped mobile users find a need and desire to use the internet on their mobile phone.

Take for example Indonesia, a much heralded and developing digital landscape, where according to the below infographic from analyst firm Saling Silang (via Facebook) 83% of Indonesians use internet cafes with 22% access the web through a smartphone.

Yet none of this traffic registers with comScore’s system…

Though many users access the web through multiple channels at different times, comScore’s data is missing out on recording additional activity from a huge amount of people, particularly when smartphones and internet cafes play a key part enabling internet access as internet penetration is as low as 12%.

Missing out on this data makes it questionable as to whether the data is useful or accurate.

Take for example the data below looking at the top 10 Facebook markets based on reach.

You’ll note that the data set is indicated as including “visits from home/work location”, meaning that analyses the percentage of people connected to traditional, PC-based web (i.e. internet penetration rate) with the monthly traffic from home/work locations. So no mention of mobile or internet cafes whatsoever, despite their prominence in many of the developing and developed markets ranked.

Someone please tell me the point of this data? Why bother giving a conclusion based on less than half of the story?

At best it is unsubstantiated or at worst misleading given that blogs and newspaper pick up and run with the results.

And it isn’t just the poor data collection either, see for example comScore announcement ‘Smartphone Adoption Continues to Grow in Japan’.

I couldn’t help but tweet the seemingly ridiclous news that just 6.9% of the total Japanese mobile audience used a smartphone, and I soon found my disbelief echoed by a couple of well respected, Japanese digital watchers Serkan Toto and Gen Kenai.

What now?

comScore has a responsibility to be clear about the limitations of its data set and how it can be misinterpreted or, as I’d prefer, stop using their statistics to measure digital trends in Asia until it covers mobile and public access points.

Confusing, inaccurate messages can be put out and picked as gospel by news organisations and/or blogs as was recently the case with Singapore-based Penn Olson – as I covered in this post – who published an ‘exclusive’ claiming Twitter was Japan’s leading social network based on misleading comScore data.

A reliance on dated references such as internet penetration rate must shoulder some blame too.

In a region where mobile is set to become the dominant platform for internet usage, and a continent where mobile internet usage (compared to fixed-line) exceeds all other global areas, looking at the likely reach of fixed-line internet leaves a lot of activity unaccounted for.

Indeed, as I mentioned recently, the number of Facebook members in a country is fast-becoming a more reliable metric, as are quantitative, non-platform specific stats such as tweets per minute, active users, returning users.

Tracking modern day internet data is not easy, but with Asia developing into a key growth market for many digital and online industries, comScore needs to seriously look at its current methods and deliver analysis of the full picture.

The firm has begun to make measure to mobile data in Japan as this post – After the Quake: Mobile Internet Use Soars in Wake of Japan Crisis – demonstrates, expanding this type of analysis across Asia would be the ideal response.

---

---

---