Can Data-Mining Increase Market Efficiency? August 3, 2007Posted by federalist in Markets.
Let us say that the goal of capitalism is to create as many goods and services that people most want and to get those things into their hands as efficiently as possible. In a free market producers try to innovate and market, but we know that even the perfect product at the right price can fail to find all the consumers whose utility it will optimize.
The recent WSJ article, “We Know What You Ought To Be Watching This Summer,” raised my hopes that technology will yet give us massive improvements in consumer market efficiency: Online marketers should be able to mine the ratings and purchases of large numbers of customers to produce tailored profiles that can recommend new products to individuals that suit their needs and interests — products those individuals may not otherwise encounter.
It is a compelling idea, but maybe not quite so straightforward. Tom Slee offers an illuminating analysis of the Netflix recommendation database and concludes, “I see little evidence … that recommender systems are the magic ingredient that will reveal the wisdom of crowds.”