Improving choice, maximizing revenue
Dennis Zhang and Jake Feldman’s research started as a debate—a little like the Reese’s candy commercial pitting peanut butter lovers against chocolate lovers. In the end, they combined the ingredients to come up with something bigger.
video
In the case of their
Olin Award-winning research, however, when
Feldman got “customer choice modeling” in Dennis Zhang’s “machine learning” and
Zhang got “machine learning” in Feldman’s “customer choice modeling,” the result was 28% higher revenue in a week’s time for Chinese online retail giant Alibaba.
The result came about because the pair collaborated to create a new customer choice algorithm designed to better populate six available slots for products in online stores hosted by Alibaba. Those six products appear because the platform instantly crunches millions of variables to display options with the highest likelihood of driving a sale.
Zhang and Feldman, both assistant professors of operations and manufacturing management at Olin, thought the platform could do better—but not before they realized they could combine their disparate approaches into a new mathematical model for presenting product choices to customers. The pair partnered with Alibaba researchers on their working paper, “Taking Assortment Optimization from Theory to Practice: Evidence from Large Field Experiments on Alibaba,” under consideration by a top journal.
“We show that product recommendation systems built on the framework of assortment optimization have the potential to outperform machine-learning-based approaches.”
“I worked at Google as a machine learning scientist,” Zhang said. “I was a true believer of machine learning, which can basically crack most problems. But Jake is a hardcore believer of (customer choice) optimization. In order to convince each other, we started working on this project by combining machine learning with optimization.”
For Alibaba and other online retailers, the problem goes something like this: Among thousands of available products, which ones should the platform recommend to maximize revenue and give customers the most useful choices? Alibaba had relied on machine learning to quickly gauge a visitor’s past purchasing history, age, location, the history of similar customers and a million other variables to come up with a selection of choices to display.
But while machine learning can instantly weigh those variables and display customized product choices, it’s a poor tool for providing wider variety, often displaying products that overlap with or cannibalize other products.
As Feldman describes it, the problem is two-tiered: The first tier is about estimating what customers might like. Machine learning is good at that. The second is optimizing the choices based on what other choices are available and who the customer is—a weak spot for machine learning.
The machine learning-based system might present shoppers with two very similar shirts, or shirts with very different price points—products that don’t offer the right variety or might undermine the retailer’s ability to maximize revenue. It’s particularly troubling when one product is not quite right, but the shopper isn’t given a suitable substitution option.
So, Feldman turned to mathematical models for optimizing customer choices from half a century ago. “They’re not as robust as these machine learning models,” he said. “But what that buys us is we can then solve a more sophisticated optimization problem.”
The researchers tested their new algorithm against the traditional machine learning model Alibaba had used. In a weeklong experiment in March 2018, watching 14 million customers on Alibaba-owned shopping sites, the new combination model showed 28% higher revenues—or nearly $22 million. The results were so conclusive, Alibaba adopted the new product selection model.
The pair agrees that the next step in their research—already underway with Alibaba—is tweaking the model to include data about products the shopper has shown interest in by clicking on them without a purchase. The new model also breaks when more than one product choice is appealing to the customer—another area ripe for further study.
But for now, the pair is very happy with the results of this first collaboration. The results, Zhang said, “seem very inspiring to me because it shows my field and Jake’s field are actually important in the real world of business.”
KEY TAKEAWAYS for Managers
- Online customer recommendations pose a two-tiered problem. The first, estimating what customers might like. The second, optimizing choices based on what other choices are available.
- Machine learning is good at the first tier, not the second.
- Blending legacy “customer choice” algorithms with machine learning lets online retailers “solve a more sophisticated optimization problem.”
“Taking Assortment Optimization from Theory to Practice: Evidence from Large Field Experiments on Alibaba”
Authors:
Jacob Feldman, assistant professor of operations and manufacturing management
Dennis Zhang, assistant professor of operations and manufacturing management
ADDITIONAL RESEARCHERS
Xiaofei Liu and Nannan Zhang, Alibaba Group
Publication:
Working paper, August 2018
