I've always been known to be a cynic. There is, in fact, something inside my brain that trips the minute I start to partake in any kind of exuberance over matters related to technology. But even in my most hype-averse moments, I never imagined to think as skeptically as James Glanz, who penned a most sober article in this weekend's New York Times:
Is Big Data An Economic Dud?
After pointing at the reams of data being collected by everyone from Amazon to Google, and all of the hype surrounding its purported value, he gets right to work:
There is just one tiny problem: the economy is, at best, in the doldrums and has stayed there during the latest surge in Web traffic. The rate of productivity growth, whose steady rise from the 1970s well into the 2000s has been credited to earlier phases in the computer and Internet revolutions, has actually fallen. The overall economic trends are complex, but an argument could be made that the slowdown began around 2005 — just when Big Data began to make its appearance.
So, let's calm down a minute and really draw a few distinctions about what has happened recently versus what has been happening for quite some time. It is very easy to conflate the old with the new in this space, and then easily draw the wrong set of conclusions.
There's a field called Operational Research that has been around for quite some time that looks, feels, and smells just like Big Data, but it really isn't as bleeding edge as one might think. Mostly because it seems boring, not a whole lot of people have noticed it. Rest assured, though, it has been making your life better and creating tons of economic value. That the fact airports schedule all those flights, quants at hedge funds squeeze out all those pennies, and UPS manages to deliver more packages every year without clogging every city with brown trucks is evidence towards that. (Stupid fact of the day: If you follow a UPS truck through the city, you will notice that they do not make left turns. Faster that way.) The old economy has been using OR to increase efficiency for decades, and it is probably one of many reasons that productivity continued to climb through the 90's.
There is a "new new" thing that, depending on who you ask, is still largely untapped. This is all of the metadata that gets generated when the world creates transactional data. The sheer size and scope of this makes it "Big" - it challenges the limits of the compute infrastructure that companies have to apply to it. To make matters worse, much of it is unstructured - it does not live neatly in the rows and columns of a database. You can't just throw up some SQL queries on it. Unless you understand it deeply, you don't even know what the relations would look like.
What is this stuff? Think of all of the log data from all of your servers and virtual machines. Consider all of the time stamps and file sizes, or maybe how many of the videos posted to YouTube are of kittens. There's a huge amount of intelligence there that can be leveraged to improve efficiency of your business in the same way OR has helped in the past, but the tools are not fully cooked.
I would argue that Google and Amazon, and the like have done a truly outstanding job of leveraging this metadata to streamline their businesses. This, and the hype, is why their stock prices are as stratospherically high as they are: They are far more efficient at targeting ads, moving goods, etc. than their old world peers. Sadly, they are just a small part of the economy. Leveraging all that metadata in every business requires insight, intelligence, and tools. These aren't there yet. Hence, its a bit premature to sit there tapping your toe waiting to feel the effects of the Big Data revolution.
No comments:
Post a Comment