Predicting Price Changes on Amazon Marketplace

Jeff Bezos created the Marketplace in the year 2000 with the intent of allowing Amazon’s customers to comparison shop for the best prices on Amazon’s website. Bezos’s vision was that if some other business could provide the same product at a lesser price than Amazon, then the customers should have the ability to make that choice. Today, 3P sellers account for 52 percent of sales on Amazon.com.

The Amazon Marketplace has been an enormous boon to the many small businesses who sell their wares on it. However, as the owner of the Marketplace, Amazon has considerable advantages in terms of knowledge of the sales of an item, and also has presence across multiple categories. It can use this knowledge to price its own products, or its favored third-party seller’s products. If its pricing algorithm reduces prices to an extent that Amazon sustains a loss in a specific transaction, it can offset those losses through higher margins on other products. The 3P sellers lack both this information that Amazon has, and the sophisticated algorithms Amazon uses to change prices. For small business 3P sellers, this inability to price competitively can be especially hard. Amazon’s dynamic pricing has received much attention in the business press and has been criticized in some quarters for the inherent unfairness to the 3P sellers.

For Tuck assistant professor Sharmistha Sikdar, this presented an interesting case study to examine the predictors of these price changes as an intervention on behalf of the 3P sellers. In a paper she wrote for her PhD dissertation at Cornell University, Sikdar along with her Cornell co-authors Vrinda Kadiyali and Giles Hooker, have developed a machine learning tool that can help 3P sellers and regulators predict the timing and degree of price changes made by Amazon and other 3P sellers.

“The challenge for 3P sellers is that many of them are not established retailers like Amazon,” Sikdar says. “Many are really small businesses operating out of a garage, and they don’t have the technical capabilities to change prices or even know when they should change prices.”

Sikdar’s paper—“Price Dynamics on Amazon Marketplace: A Multivariate Random Forest Variable Selection Approach”—is one of the first studies of price dynamics in this context. An important methodological challenge that Sikdar and her co-authors faced while studying this topic is the statistical problem of a high dimensional covariate space. In layman’s terms, this means there was a large number of variables that could potentially be used to predict the price changes. They studied a single category of electrical kitchenware, analyzing 14 brands sold by more than 200 3P sellers. Without a way to narrow down the factors influencing price changes, this study would have had to do a regression analysis on hundreds of variables—a dataset too large and untargeted to yield helpful results. To get around this, the researchers created a variable selection algorithm using a machine learning tool called the multivariate random forest (MVRF) that winnows the variables down to those that are most influential—reducing them by 86 percent—which then allows a traditional regression analysis to be more effective. Furthermore, the prediction accuracy of the variables selected by the proposed algorithm when applied to the generalized additive regression model is higher than those of traditional methods, such as LASSO.

“What the algorithm is doing is removing all the unimportant variables and retaining only the best predictors,” Sikdar explains. “From there we can do a regression model that, based on prediction accuracy, can provide a descriptive interpretation of what can predict a price change for which seller type.”

Using their machine learning tool, the researchers found that predictors of price changes include factors such as seller ratings, shipping terms and conditions, customer reviews, and past price changes of newer and smaller 3P sellers. However, Sikdar warns that these factors are subject to change with time, depending on the dynamics of the Marketplace. The environment of the Marketplace is always changing, so the model needs to be continually trained to provide accurate, up-to-date predictions.

This paper has implications for academics and regulators. For academics, the machine learning tool can be deployed in any multivariate response problem where there are too many variables to make sense of. Furthermore, marketing researchers can take the machine learning tool and the descriptive observations of this study one step further and design models to make prescriptive recommendations. Regulators, for their part, could be keenly interested in a tool that uncovers the inner workings of Amazon Marketplace’s pricing dynamics.

Sikdar hopes her paper can level the playing field for the 3P sellers on Amazon just a bit. “This project can help 3P sellers learn how to keep up with price changes,” she says, “so they can decide to either change their price too, or pull out of the market.”