Analyzing the Results of Your Classification Predictive Model

Once you've trained your classification predictive model, you can analyze its performance to make sure it's as accurate as possible.

Use the dropdown list to access and analyze the reports on influencers and predictive model performance.

What do the values of the two main performance indicators mean? Does the target value appear in sufficient quantity in the different data sources? Which influencers have the highest impact on the target? Which group of categories has the most influence on the target? Is my model producing accurate predictions? Can I evaluate the costs/savings using this model? Can I see any model errors? Is my predictive model producing accurate predictions? What's next?

Click the area for more information.

What do the values of the two main performance indicators mean?

Quickly check if your predictive model is accurate and robust, checking the global performance indicators:

  • Predictive Power is your main measure of predictive model accuracy. It takes a value between 0% and 100%. This value should be as close as possible to 100%, without being equal to 100% (100% would be a hypothetically perfect predictive model; 0% would be a random predictive model with no predictive power). To improve your Predictive Power, you can add more influencers, for example.

    For more information, refer to Predictive Power.

  • Prediction Confidence indicates the capacity of your predictive model to achieve the same degree of accuracy when you apply it to a new data source, which has the same characteristics as the training data source. It takes a value between 0% and 100%. This value should be as close as possible to 100%. To improve your Prediction Confidence, you can add new rows to your data source, for example.

    For more information, refer to Prediction Confidence.

Note

Depending on your business issue, you can look at the other provided performance indicators for the predictive model, and also review the profile of the detected curve. For more information, refer to Assessing Your Predictive Model With the Performance Indicators and The Detected Target Curve.

Does the target value appear in sufficient quantity in the different data sources?

Get an overview of the frequency in each data source of each target class (positive or negative) that belongs to the target variable.

It's usually recommended that you have at least 1000 records of the each class in your data source. Under this threshold, the validity of the prediction confidence is no longer guaranteed.

For more information, refer to Target Statistics.

Which influencers have the highest impact on the target?

Check how the top five influencers impact on the target. Only the top five contributing influencers are displayed as a default.

For more information, refer to Influencer Contributions.

Which group of categories has the most influence on the target?

In the Influencer Contributions report, analyze the influence of different categories of an influencer on the target:
  • If the influence value is positive, we are more likely to get "minority class".
  • If the influence value is negative we are less likely to get "minority class".

The influence of a category can be positive or negative.

For more information, refer to Category Influence, Grouped Category Influence and Grouped Category Statistics.

Is my model producing accurate predictions? Can I evaluate the costs/savings using this model?

Use the Confusion Matrix tab and assess the predictive model performance in detail, using standard metrics such as specificity.

Use the Profit Simulation tab and estimate the expected profit, based on costs and profits associated with the predicted positive and actual positive targets.

For more information, refer to Confusion Matrix, The Profit Simulation.

Can I see any model errors? Is my predictive model producing accurate predictions?

Use a large panel of performance curves in the Performance Curves tab, to compare your predictive model to a random predictive model and a hypothetical perfect predictive model:
  • Determine the percentage of the population to contact to reach a specific percentage of the actual positive target with The Detected Target Curve.
  • Check how much better your predictive model is than the random predictive model with The Lift Curve.
  • Check how well your model discriminates, in terms of the compromise between sensitivity and specificity with The Sensitivity Curve (ROC).
  • Check the values for [1-Sensitivity] or for Specificity against the population with The Lorenz Curves.
  • Understand how positive and negative targets are distributed in your model with The Density Curves.

What's next?

You have two possibilities: