Thursday, November 22, 2012

A Real World Case Study: Business Rule vs Predictive Model

The following is a true story to complement earlier posts Comparison of Business Rules and Predictive Models and Predictive Modeling vs Intuitive Business Rules .

A few years ago, we built a new customer acquisition model for a cell phone service provider based on its historical application and payment data. The model calculated a risk score for each cell phone service applicant using information found in his/her credit reports. The higher the score, the higher the risk that a customer will not pay his/her bill.

A few weeks after the model was running, we received an angry email from the client company manager. In the email, the manager gave a list of applicants who had several bankruptcies. According to the manager, they should be high risk customers. However, our model gave them average risk scores. He questioned the validity of the model.

We mentioned that the model score was based on 20 or so variables, not bankruptcies alone. We also analyzed people with bankruptcies in the data that we used to build the model. We found that they paid bills on time. It might be that people with bankruptcies are more mobile and thus depend more on cell phones for communication. They may not be good candidates for mortgage. But from cell phone service providers' perspective, they are good customers.

This is the bottom line. Data-driven predictive models are more trustworthy than intuition-driven business rules.

3 comments:

anay said...

I really enjoyed reading is g your post. Thank you for sharing so nice and helpful ideas to readers who do not have to face this problem again in their project

Dirk Kettlewell

Unknown said...

you article clearly shows the methods in assessing credit risk is a scorecard. I have seen some post related to this topic in essay writing services.

Jim Kayle said...

I simply want to tell you that I'm a new comer to weblog and definitely liked this blog site. Totally possible I'm likely to bookmark your blog.

PhD Dissertation Help