Data Science Asked by Wize Dog on December 10, 2020
We have an optimization problem on hand that is the following.
Let’s say we have 10 different treatments that we might offer, all of them are equally good for us, but people have different propensity to take those treatments.
Before we’ve built the model the rule was to just randomly choose one of the treatments, offer it to a person, and if the person takes it – yaay, if not – so be it.
Based on this historic data we’ve created a binary classifier that predicts a person’s propensity to agree to each treatment type and the idea is to offer a person a treatment that has a max propensity score from the model.
The model built shows great GINI and AUC numbers, but what we struggle with is a more business-related metric that we could use to evaluate the model’s performance.
I’m sure some of you had been in this situation before. What metrics did you use?
Thank you!
On historical data, you can try to determine what would have happened if you add the model before. For each client you proposed some product, calculate the average gain (propensity*cost) of said product and compare with the average gain of said of best product. Say each product cost 100$.
Last year you offered product1 to client1 but he refused. Using your model you know : he only had 20% chances of buying product1 and he had 25% chances of buying product 2. On average you missed out on (25%-20%)*100 = 5 dollars. You can calculate a similar metric on all your clients, than aggregate the total. That would give you an estimation of the amount of dollars you would have made last year if you used your model. There rarely is more business-related metric than money.
Answered by lcrmorin on December 10, 2020
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP