The Big Data Fallacy

The latest issue of Foreign Affairs features the cover article “The Rise of Big Data” by Kenneth Cukier and Viktor Mayer-Schoenburger, which mostly details some of the incredible ways companies like UPS, Google and Apple have come to rely on vast arrays of numbers in order to run their businesses better. But data has always provided a problem in that it gives a substantive assurance of certainty that has a propensity to foster overconfidence in those relying on it. The article attempts to address this:

“[K]nowing the causes behind things is desirable. The problem is that causes are often extremely hard to figure out… Behavioural economics has shown that humans are conditioned to see causes even where none exist. So we need to be particularly on guard to prevent our cognitive biases from deluding us; sometimes, we just have to let the data speak.”

The sentiment here is admirable, and the context perceptive. But the final part of the quotation (my emphasis) assumes wrongly that data can speak objectively, that there is a fundamental ‘truth’ in a number. All too often though the wrong things are measured, or not all variables are measured. What data does not record, or worse, cannot record, can often be overlooked. While ostensibly data is there to provide assistance with building models and predicting future trends and movements, it sometimes leads to a very narrow view of one particular future, and fails to account for possibilities, that, though while unlikely, could potentially be devastating. This is what Nicholas Taleb writes about in his by turns unreadable but seminal work, Black Swan. The fictional, paranoid loner Fox Mulder of the hit series The X-Files had it right fifteen years ago when he lamented “in a universe of infinite possibilities, we may find ourselves at the mercy of anyone or anything that cannot be programmed, categorised or easily referenced”. The financial system before 2008 was a victim of such narrow thinking. 

Hendrik Hertzberg, in his Talk of the Town column “Preventive Measures” in this week’s The New Yorker, made the adroit analogy with the 2002 film Minority Report in our quest to categorise and predict acts of crime. Hertzberg points out that in reality this “turns out to be a good deal more difficult than investigating such an act once it occurs”. Indeed, such prediction methods are being implemented, just with somewhat less efficacy than in the Tom Cruise movie. The stop-and-frisk procedure currently employed by the New York Police Department points to a sustained effort to engage in preventative measures to reduce crime, effectively what Cruise and his myrmidons were doing, albeit without the help of psychic imagery as in the film. While the psychic “Pre-Cogs” turned out to occasionally disagree, the success rate with stop-and-frisk is even less attractive. “In the final months of 2012″, writes the New York Times, only 4% of stops resulted in an arrest.

Hertzberg also alludes to the dilemma of mountains of data, produced without concern for oversight or management; producing more just because it’s possible to produce it:

“This fall, the National Security Agency, the largest and most opaque component of the counter-terrorism behemoth, will [open] a billion-dollar facility [analysing] intercepted telecommunications… each of the Utah Data Center’s two hundred (at most) professionals will be responsible for reviewing five hundred billion terabytes of information each year, the equivalent of twenty-three million years’ worth of Blu-ray DVDs… that’s a lot of overtime.”

The other problem this data poses – and increasingly this goes for many industries that are jumping on the Big Data bandwagon – is that intelligence departments and businesses alike are able to put quantifiable targets and figures to what they want to achieve, whether they are actually applicable or not. Police claim the low stop-to-arrest ratio implies that they are preventing crimes by stopping someone before they act. There is nothing to argue otherwise. The New York Times article alludes to the debate over what ratio or percentage the Supreme Court would be comfortable with under the tenet of “reasonable suspicion”. This leads down a dangerous path where we treat data as an answer to a question, rather than as supporting evidence to an answer.

How the Obama 2012 campaign harnessed tech to win votes

Last night, at the Royal Automobile Club on London’s Pall Mall, Zeitgeist was fortunate enough to hear Harper Reed, the Chief Technology Officer of the Obama 2012 US presidential campaign speak candidly about how he helped get out the vote and keep the Democrats in the White House. Harper is ex-Threadless, the famous T-shirt company that lets users contribute their own designs, with the most popular becoming actual products sold the world over. It’s a democratic philosophy, one that understandably caught the attention of the campaign committee. It is also the kind of thinking that cities like New York and Chicago are starting to employ; actively gathering, analysing and distributing data to inform policy implications and help citizens. What follows is a brief summary of his thoughts and points that Zeitgeist found interesting.

Harper began the talk with the fundamentals, discussing how, when he arrived, the campaign seemingly already had much of the data gathering resources needed to achieve what he wanted. The trouble was it as all siloed. Putting all the data together was a major step in the right direction, toward cohesive data analysis. He elaborated, saying they went from having fifteen different numbers for doors that needed to be knocked on, to one. On hiring the right people for the task at hand, Harper was explicit in noting that they had hired tech people and taught them about politics, rather than the other way around. He riffed on the state of journalism, saying it was similarly important when hiring journalists that know about tech.

One of the more interesting insights Harper talked about involved the target demographics. Those most likely to vote are male or female 18-28, and women perhaps in her 50s. The younger group is adept and comfortable with all digital platforms, but still uses paper a fair amount. Paper, by contrast, is an essential medium for that middle-aged female voter. So the insight was about making paper use more efficient, given these groups’ use of it. Understandably this was a hard decision for a group of very tech-minded people to arrive at, but the acknowledgement showed they were willing to park their own pre-conceptions on how things ought to be done.

Like many startups, they were constantly trying to fail in order to create redundancies. This involved hosting hackathons where code was obsessively broken and then reconstructed, “ensuring things would break in ways we understood”, as Harper put it. They had the same approach with the content they published, aggressively testing every piece to make sure it was relevant and engaging for the intended audiences. What they failed to foresee was the Internet activist group Anonymous launching a DDOS attack the day before the election to coincide with Guy Fawkes day, which helped trigger a meltdown over at Amazon’s cloud servers, AWS. Harper made it sound like not too much trouble to switch the servers from the East Coast where they had been affected, to the West Coast, but the experience must have been a stressful one.

Lastly, he offered an opinion increasingly shared by many in the industry, which was a reluctance to talk of mobile device use as “second-screening”. Mobile devices, Harper pointed out quite rightly and obviously, are the first thing you look at when you wake up, the last thing you look at when you go to bed, and the thing you’re actually looking at when you’re supposed to be watching TV. Mobile first should always be the initial mindset.

In questions, Ruth Porter asked whether there were any pearls of wisdom that could be applied to those in UK politics and how they go about with their own strategy of getting out the vote. Harper conceded he had met that day with a party “whose name starts with ‘L’”, and believed that what was key was investment, commitment and belief from the very top in what social and data could do for the campaign. Without that, such efforts would amount to nothing. The lessons of the Obama 2012 campaign – and the pitfalls of Romney’s campaign – offer valuable lessons for political parties, but it seems any efforts at cherrypicking ideas or going in half-hearted would doom any prospect of leveraging what the Obama team were able to do.

Any success in Harper’s tech strategy must be qualified against the sheer unpopularity of Obama’s rival candidate

The state of retail

The love of the bargain is what drives them… Click for CNBC’s coverage

It’s a common fallacy to think of a time before a change in status quo as somehow being magically problem-free. A Panglossian world where all was well and nothing needed to change, and wasn’t it a shame that it had to. Similarly, we cannot blithely consign the retail industry of the past to some glorious era when everything was perfect; far from it. The industry has been under continual evolution, with no absence of controversy on the way. It was therefore a timely reminder, as well as being a fascinating article in its own right, when the New York Times provided readers recently with a potted history and a gaze into the future of Manhattan department store stalwart, Barneys. Not only is their past one in which the original proprietor sought to undercut his own suit suppliers, creating a bootlegging economy by literally ripping out their labels and replacing them with his own, but it was also one where department stores served a very different purpose to what they do today. They had less direct competition, not least unforeseen competition in the form of shops without a physical presence. Moreover, today they are run in an extremely different way, with an arguably much healthier emphasis on revenue (though some might say this comes at the expense of a feeling of luxury, in a lobby now brimming with handbags and little breathing room). The problems and opportunities for Barneys could serve as an analogy for the industry of which it is a part.

Despite brief reprieves such as Black Friday (click on headline image for CNBC’s coverage), as well as the expected post-Christmas shopping frenzy, can one of the main problems affecting retail at the moment simply be that it is undergoing an industry-wide bout of creative destruction? Zeitgeist has written about the nature of creative destruction before, and whether or not that is to blame for retail’s woes, the sector is certainly in the doldrums. In the UK, retailers are expecting a “challenging” year ahead. Recent research from Deloitte shows 194 retailers fell into administration in 2012, compared with 183 in 2011 and 165 in 2010. So, unlike the general economy, which broadly can be said to be enjoying a sclerotic recovery of sorts, the state of retail is one of continuing decline. How did this happen, and what steps can be taken to address this?

Zeitgeist would argue that bricks and mortar stores are suffering in essence due to a greater amount of competition. By which, we do not just mean more retailers, on different platforms. Whether it be from other activities (e.g. gaming, whether MMOs like World of Warcraft or simpler social gaming like Angry Birds), or other avenues of shopping (i.e. e-commerce, which Morgan Stanley recently predicted would be a $1 trillion dollar market by 2016), there is less time to shop and more ways to do it. The idea of going to shop in a mall now – once a staple of American past-time – is a much rarer thing today. It would be naive to ignore global pressures from other suppliers and brands around the world as putting a competitive strain on domestic retailers too. Critically, and mostly due to social media, there are now so many more ways and places to reach a consumer that it is difficult for the actual sell to reach the consumer’s ears. This is in part because companies have had to extend their brand activity to such peripheries that the lifestyle angle (e.g. Nike Plus) supercedes the call-to-action, i.e. the ‘BUY ME’. The above video from McKinsey nicely illustrates all the ways that CMOs have to think about winning consumers over, which now extend far beyond the store.

If we look at the in-store experience for a moment without considering externalities, there is certainly opportunity that exists for the innovative retailer. Near the end of last year, the Financial Times published a very interesting case study on polo supplier La Martina. The company’s origins are in making quality polo equipment, from mallets to helmets and everything in between, for professional players. As they expanded – a couple of years ago becoming the principle sponsor of that melange of chic and chav, the Cartier tournament at Guards Polo Club – there came a point where the company had to decide whether it was going to be a mass-fashion brand, or remain something more select and exclusive. As the article in the FT quite rightly points out, “Moving further towards the fashion mainstream risked diluting the brand and exposing it to volatile consumer tastes.” The decision was made to seek what was known as ‘quality volume’. The company has ensured the number of distributors remains low. Zeitgeist would venture to say this doesn’t stop the clothing design itself straying from its somewhat more refined roots, with large logos and status-seeking colours and insignia. Financially though, sales are “growing more than 20% a year in Europe and Latin America”, which is perhaps what counts most currently.

Louis Vuitton’s ‘L’ecriture est un voyage‘ is a good example of experimental thinking and missed opportunities

In the higher world of luxury retail, Louis Vuitton is often at the forefront (not least because of its sustained and engaging digital work). While we’re focusing purely on retail environments though, it was interesting to note that the company recently set up shop (literally) on the left bank of Paris; a pop-up literary salon, to be precise. Such strokes of inspiration and innovation are not uncommon at Vuitton. They help show the brand in a new light, and, crucially, help leverage its provenance and differentiate it from its competition. Sadly, when Zeitgeist went to visit, there was a distinct feeling of disappointment that much more could have been done with the space, which, while nicely curated (see above), did little to sell the brand, particularly as literally nothing was for sale. The stand-out piece, an illustrated edition of Kerouac’s On the Road, by Ed Ruscha, Zeitgeist had seen around two years ago when it was on show at the Gagosian in London. Not every new idea works, but it is important that Louis Vuitton is always there at the forefront, trying and mostly succeeding.

So what ways are there that retailers should be innovating, perhaps beyond the store? One of the more infuriating things Zeitgeist hears constructed as a polemic is that of retail versus the smartphone. This is a very literal allusion, which NBC news were guilty of toward the end of last year. “Retail execs say they’re winning the battle versus smartphones”, the headline blared. What a more nuanced analysis of the situation would realise is that it is less a case of one versus the other, than one helping the other. The store and the phone are both trying to achieve the same things, namely, help the consumer and drive revenue for the company. Any retail strategy should avoid at all costs seeing these two as warring platforms, if only because it is mobile inevitably that will win. With much more sound thinking, eConsultancy recently published an article on the merits of providing in-store WiFi. At first this seems a risky proposition, especially if we are to follow NBC’s knee-jerk way of thinking, i.e. that mobile poses a distinct threat to a retailer’s revenue. The act of browsing in-store, then purchasing a product on a phone is known as showrooming, and, no doubt aided by the catchy name, its supposed threat has quickly made many a store manager nervous. However, as the eConsultancy article readily concedes, this trend is unavoidable, and it can either be ignored or embraced. Deloitte estimated in November that smartphones and tablets will yield almost $1bn in M-commerce revenues over the Christmas period in the UK, and influence in-store sales with a considerably larger value. That same month in the US, Bain & Co. estimated that “digital will influence more than 50% of all holiday retail sales, or about $400 billion”. Those retailers who are going to succeed are the ones who will embrace mobile, digital and their opportunities. eConsultancy offer,

“For example, they could prompt customers to visit web pages with reviews of the products they are considering in store. This could be a powerful driver of sales… WiFi in store also provides a way to capture customer details and target them with offers. In fact, many customers would be willing to receive some offers in return for the convenience of accessing a decent wi-fi network. Tesco recently introduced this in its larger stores… 74% of respondents would be happy for a retailer to send a text or email with promotions while they’re using in-store WiFi.”

These kind of features all speak more broadly to improving and simplifying the in-store experience. They also illustrate a trend in the blending between the virtual and physical retail spaces. Major retailers, not just in luxury, are leading the way in this. Walmart hopes to generate $9bn in digital sales by the end of its next fiscal year. CEO Mike Duke told Fast Company, “The way our customers shop in an increasingly interconnected world is changing”. This interconnectedness is not new, but it is accelerating, and the mainstream arrival of 4G will only help spur it on further. The company is soon to launch a food subscription service, pairing registrants with gourmet, organic, ethnic foods, spear-headed by @WalmartLabs, which is also launching a Facebook gifting service. At the same time, it must be said the company is hedging its bets, continuing with the questionable strategy of building more ‘Supercenters’, the first of which, at the time a revolutionary concept, they opened in 1988.

One interesting development has been the arrival of stores previously restricted to being online into the high street, something which Zeitgeist noted last year. This trend has continued, with eBay recently opening a pop-up store in London’s Covent Garden. These examples are little more than gimmicks though, serving only to remind consumers of the brands’ online presence. Amazon are considering a much bolder move, that of creating permanent physical retail locations, if, as CEO Jeff Bezos says, they can come up with a “truly differentiated idea”. That idea and plan would be anathema to those at Walmart, Target et al., who see Amazon as enough of a competitor as it is, especially with their recent purchase of diapers.com and zappos.com. It serves to illustrate why Walmart’s digital strategies are being taken so seriously internally and invested in so heavily. Amazon though has its own reasons for concern. Earlier in the article we referenced the influence of global pressures on retailers. Amazon is by no means immune to this. Chinese online retailer Tmall will overtake Amazon in sales to become the world’s largest internet retailer by 2016, when Tmall’s sales are projected to hit $100 billion that year, compared to $94 billion for Amazon. The linked article illustrates a divide in the purpose of retail platforms. While Amazon is easy-to-use, engaging and aesthetically pleasing, a Chinese alternative like Taobao is much more bare-bones. As the person interviewed for the article says, “It’s more about pricing – it’s much cheaper. It’s not about how great the experience is. Amazon has a much better experience I guess – but the prices are better on Taobao.”

So how can we make for a more flexible shopping experience? One which perhaps recognises the need in some users to be demanding a sumptuous retail experience, and in others the need for a quick, frugal bargain? Some permutations are beginning to be analysed, and offered. Some of these permutations are being met with caution by media and shoppers. This month, the Wall Street Journal reported that retailer Staples has developed a complex pricing strategy online. Specifically, the WSJ found, it raises prices more than 86% of the time when it finds the online shopper has a physical Staples store nearby. Similar such permutations in other areas are now eminently possible, thanks in no small part to the rise of so-called Big Data. Though the Staples price fluctuations were treated with controversy at the WSJ, they do point to a more realistic supply-and-demand infrastructure, which could really fall under the umbrella of consumer ‘fairness’, that mythical goal for which retailers strive. Furthemore, being able to access CRM data and attune communications programmes to people in specific geographical areas might enable better and more efficient targeting. Digital also allows for a far more immersive experience on the consumer side. ASOS illustrate this particularly well with their click-to-buy videos.

As the Boston Consulting Group point out in a recent report, with the understated title ‘Digital’s Disruption of Consumer Goods and Retail’,  “the first few waves of the digital revolution have upended the retail industry. The coming changes promise even more turmoil”. This turmoil also presents problems and opportunities for the marketing of retail services, which must be subject to just as much change. If we look at the print industry,  also comparatively shaken by digital disruption, it is interesting to note the way in which the very nature of it has had to change, as well as the way its benefits are communicated. It is essential that retailers not see the havoc being waged on their businesses as an opportunity to ‘stick to what they do best’ and bury their head in the sand. This is the time for them to drive innovation, yes at the risk of an unambitious quarterly statement, and embrace digital and specifically M-commerce. What makes this easy for those companies that have so far resisted the call is that there is ample evidence of retailers big and small, value-oriented to luxury-minded, who have already embraced these new ideas and platforms. Their successes and failures serve as great templates for future executions. And who knows, the state of retail might not be such a bad one to live in after all. Until the next revolution…