Harnessing the power of Predictive Modeling using Naïve Bayes

In this series of theory and practicals of data mining in SQL Server 2005, I will be covering the Naive Bayes algorithm.

We will cover the following in the next few posts
- What is Naive Bayes used for?
- What is Classification?
- Probability Theory + Bayes Theorem
- Discussion on Naïve Bayes classifier works?


The Naive Bayes is basically a classification algorithm. Given a series of test cases, it will derive a model based on the conditional probability between the input and predictable columns. Subsequently, it will make use of the model to predict what class a new input case will fall into. One of the key assumption is that the columns are independent (which we will discuss in the next few post).

To illustrate what is classification, let us consider a simple example. Suppose I am a marketing manager from a telco, and have many customers in my database. In the database, I capture the demographic patterns of the customers, as well as whether they churned in the last year. Hence, we have 2 classes: (1) Churn (2) Do not Churn.

By looking at many such examples (i.e. input test cases), I can then compute the probability that a customer will churn based on his demographic profile, which can then be stored as a model. When a new customer arrives, and stay with the telco for several months, and I want to determine all those customers that will churn, I can then feed their demographic profile through the model, and classify
each customer as Churn or Do not Churn.