Are the internet numbers adding up?

There are lies, damned lies and statistics. In the case of the internet, there are more statistics than you can shake your moneymaker at. Measurement has always been at the heart of media. Audience research is central to the planning process, to positioning, to optimising any campaign – and, with the huge measurability of the online product, a veritable Pandora’s Box of measurables has been unleashed. So it’s no surprise that a true global metric for the internet has yet to be established.

But although the medium is still in its infancy, the technology seems to be developing too fast for many players to keep up.

The internet is a truly global medium that facilitates highly specific and direct personalised marketing and advertising opportunities. So where does that leave traditional brand auditing, market research and campaign analysis? Without a standard, how can anybody credibly prove their case. And without agreed definitions and universal analysis tools, will the net ever catch up with its own potential?

What can clicks tell you?

Patrick Monahan is vice president of information technology for online audit specialist BPA International. “The situation on the web is not unlike magazines,” he says. “On the magazine side there are audits, there are readership studies and there is the publisher’s own data. On the web side, there are consensus-based audits, panel-based research and statistics compiled by site owners.”

Jeremy Wright, business development director for e-business marketing solutions specialist Engage, says the dynamics of the medium demand increasing sophistication in metrics.

“The first generation of measurement had people very excited at click-through, which we now know is relatively limited, especially when definitions vary and clicks don’t always say the same thing,” he says. “What do clicks tell you? They’re a poor guide to conversion and banner clicks have been dwindling.”

Wright says research has shown a relatively small overlap between click-through and conversion.

Online planning and buying agency i-Level is one of the few agencies to specialise. Chief executive Charlie Dobres says metrics will become increasingly important as agencies and clients grow in experience. “There are so many different areas of measurement,” Dobres says. “We’re not massively concerned with publisher audits looking at page impressions or reach because we don’t want to buy all the inventory. The last quarter of last year proved that a lot of business was being done by relatively inexperienced buyers looking to buy anything that moved. Quality was the last thing on the minds of many.”

Macromedia, the e-business solutions specialist, also has an eye on e-metrics with its web analysis tool Aria.

“Ideally people want to feel part of an online community,” says Macromedia director Tim Clark. “We work aggressively to create tools to make those audiences return. We have sites ourselves – for example, the entertainment site shockwave.com – so we have the e-metric problem ourselves. Knowing who has been on site, what they have experienced, where they have come from and what they do on site are key drivers for all web businesses.”

The key, as in the offline world, is understanding the audience. But the same old tools can’t simply be used online; there’s too much data, for starters.

Two types of information

“We’re awash with information,” says Tim Jones, head of research and development for The Telegraph Group and Hollinger Telegraph New Media. “There are two types. First, information we have ourselves, compiled every second on our own user logs. That information is non-comparative compared with the other kind – industry-wide research.”

Pierre Naggar, European research director at online sales house Real Media, highlights the ongoing confusion in the industry, both in terms of the best metrics to use and as to how precisely they should be defined. “Currently in the industry there are site-centric analyses – the kinds of audit provided by ABCElectronic and BPA International, for instance,” Naggar says. “And then there is user-centric research, mostly panel-based, provided by the likes of NetValue, MMXI or Nielsen/NetRatings.

“The problem in the panel research is that there is no consensus on what the key metrics are, so there are no universal standards against which everyone can be measured,” he adds.

Naggar says the Internet Advertising Bureau (of which he is a panel member) is one of a number of industry groups working hard to get consensus. “There are not enough sites yet being audited, perhaps because advertisers and agencies are less interested in traffic as they are not going to buy 100% of the inventory,” he says.

Is online auditing necessary?

One publisher of a mainstream online news brand, who speaks off the record, agrees with this assessment, but takes it further. “We don’t need it,” he says. “If we have a third party ad server such as Net Gravity or Net Genesis monitoring our traffic, we can guarantee our audience from those server results because they are respected. And if not everyone else is being audited by ABCElectronic, why should we give our game away?”

Unsurprisingly, ABCE managing director Richard Foan disagrees. He highlights the International Federation of Audit Bureaux of Circulations, covering 36 audits across 32 countries, and the work it is doing to agree a set of global metrics.

“I have heard the arguments against auditing online, but realistically we are still trying to understand the growth of a very young medium,” Foan says. “In print there is still the same sort of discussion 70 years after the ABC was set up.”

Foan says research is under way within the IFABC into different measurement tools across the globe. He believes publishers will become more accepting of online audits as more and more “real money” goes into online business. “There will always be a need for accountability,” he adds.

Dobres at i-Level agrees but says ABC’s approach is too traditional. “What would be useful for us would be a campaign delivery audit,” he says. “At the moment we use a third-party ad server and have those figures, which don’t always tally with what media owners are saying. Where those discrepancies occur there is no one to broker a solution with widely accepted authority. I’ve been asking ABC about this for years; although they agree in principle, we’re still waiting.”

The problem of accountable audit figures is a key one. Alicen Stenner, head of marketing at handbag.com, the joint venture women’s brand from Hollinger New Media and Boots, says a standard audit process would bring a lot of credibility into the medium. “You’ve only got to look at some of the figures from MMXI and Nielsen,” she says. “Are some of the sites really getting those sorts of figures? And without a standard, no advertiser or sponsor can claim to compare like with like, so you end up having to go with what the site says. That can be very frustrating when you know it can’t be true.”

Objectives and structure cloud the issue

The sheer amount of analysable data hides the complexity of the issues. Measuring the duration of a visit might be an obvious dynamic, but it depends on the objectives and structure of the site. It could well be that the user keeps getting lost or can’t work out a route to purchase. The classic example of measurement by page impression, when each move in a game on a gaming site could be a separate page impression or measurement of a click, whether or not the user hangs about for a requested page to download, are only the start.

Rob Whitehead, head of UK operations at 24/7 Media Technology Solutions, highlights the analysis provided by sales houses and third party ad servers. “With players like us, DoubleClick, Engage and Real Media the rigour of the data is key to the service. But there can be huge discrepancies with some of the smaller players whose kit was built on a shoe-string,”he adds.

Most sophisticated software is looking to measure the effectiveness of advertising beyond the simple click-through. It tracks the user to see if banner awareness triggers a site visit and potential conversion at a later date.

Make-up of panels vital

Like MMXI and NetRatings, NetValue looks to provide industry-wide data and bases research on panel data. In many respects the industry data is wedded to audited site data, one giving context to the other.

“The make-up of the panels is vital,” says NetValue European vice-president Alki Manias. “And you have to take account of sophisticated behavioural patterns. For instance, big users don’t tend to click through banners.”

By means of placing software into the PC, tracking is made total and data loss is minimal. But because the market is young, users are still defining their tastes, hitting sites only once and not in enough numbers to register within the panel.

Clark at the Electronic Telegraph highlights another problem with panel-based research. “There is a lack of work-based panels,” he points out. “The panels rely on software in the machine, so many companies take a corporate decision not to allow that to happen for security and privacy reasons. Generally speaking, company policies are getting tighter, not easier.”

Clark believes the emphasis on home panels undersells the reach of news-based brands such as ET, which reach a peak of use during the working day, and he is currently working to find partners to develop a joint standard on research issues. “Initially I imagine those partners will be other publishers,” he says. “But the boundaries aren’t the same online and we could just as easily be working with CNN or Carlton.”

Online sales houses such as Doubleclick, 24/7 Media, Real Media and Adlink have also developed research and analysis tools to measure the responsiveness of the market and of campaigns. Here the emphasis is on planning and buying space to optimise client objectives, but again the flexibility and immediacy of the online proposition means creatives can be tested and switched with speed to get the best from both site and ad.

“Initially Dart, our proprietary software for reselling and placing ads, was kept a secret source,” says Doubleclick TechSolutions managing director Gavin Lennox. “Then we made a strategic decision to create TechSolutions and make it available to publishers and advertisers.”

Lennox believes publishers will increasingly outsource money-making infrastructure, guaranteeing the success of product such as Dart. As most generally agree click-through is decreasing, however, the proprietary software needs to keep evolving. “Surprisingly, 70% of inventory sold is in banner ads,” Lennox says. “But these are decreasing all the time, especially as the infrastructure develops towards rich media and broadband. Advertisers want to assess campaigns across 10 or 100 sites and make sense of the results.”

Education becomes paramount

Naggar says that as the sophistication of analysis grows so does the need to educate. “It’s what we do every day when we are reporting on campaigns and traffic on site,” he says. “We have a nine-strong traffic department because it’s important for clients to understand how traffic breaks down. Click-through rate is unfortunately still the main metric, but that will change.”

Real Media is currently developing a tool to measure return on investment. After all, it’s one thing to know that 70% of the audience click through, but it’s entirely another and more pertinent thing to know what they do after that.

Manias at NetValue highlights the importance of broad-minded analysis. “With email becoming an important customer relationship tool, the analysis can prove telling.”

As proof, he points to lastminute.com. NetValue’s analysis showed the travel site with 668,000 unique visitors. But there were 880,000 on the email update database. “Many would say these were the same people, but analysis showed that only 178,000 overlapped,” he says. “This gave lastminute not 7% reach but 14% – a major difference. When we analysed the email audience, we found they spent five times as long on the site when they got there and spent more money.”

Old-style demographics remain important

NetRatings European director of analytical services David Day says old-style demographics are still vital to get the best from the net. “Our panels are key and work in tandem with our software so that we can see what people are looking at and how they’re interacting,” he says. “The demographics vary in importance depending on the offer, but they are always going to be important if you want to drive traffic from one site to another. For instance, women tend to spend between 25% and 30% less time online; if you are chasing that demographic as your target audience, you have to think very carefully about how you reach them.”

Day is less sold on the necessity of cross-industry standards than others. “If you look at market research in other areas, there are no real standards,” he says. “It comes down to which data meets the needs of the client. The bottom line is that you need robust data that fulfills the needs of clients and is respected by the industry.”

With convergent technologies already forcing their way to the forefront of development, the next stage may be applying the rigour of online analysis to other media. “In the future, all protocols will be measured because we will be looking at people from the point of view of their entire experience,” Alki Manias says. “That will answer the question of where they are when they’re not with you. We’re looking at how we can cross-correlate TV, WAP, interactive TV and so on.”

Clark at Macromedia agrees that change will continue to be rapid. “Undoubtedly there will be a need for standard measurements,” he argues. “We are living through the coming of age on the internet. Perhaps soon we’ll be able to drop the ‘e’ from e-business and it will become just another channel. Whatever happens will, however, be driven by the maturity of the market.”

From the agency point of view, Dobres at i-Level says the youth of the market must not be used as an excuse for a lack of professionalism. “Improvements are guaranteed,” he says. “It takes a lot of pressure from the likes of us to make sure people don’t get away with saying ‘it’s just the internet, it’ll get better’. It can and should be better now.”