About Preparing Datasets for Predictive Scenarios

Classification and Regression Predictive Model

• Your training dataset contains at least one column for the target variable, and multiple columns for other variables that you think may have an influence on the target variable. These variables are called Influencers.
• The datasets used to train and apply a predictive model must come from the same type of data source (acquired or live). You can't apply a predictive model on a live dataset if it was trained with an acquired dataset, nor can you apply a predictive model on an acquired dataset if it was trained using a live one. However, you can have several predictive models trained and applied with live and acquired datasets in the same predictive scenario.
  Note

  While using live datasets, both live datasets (training and apply datasets) must come from the same SAP HANA system: you cannot train a predictive model with a live dataset with data from SAP HANA system 1 and then apply this predictive model on a live dataset with data coming from SAP HANA system 2.

Times Series Forecasting Predictive Model

• Your training dataset contains a column for the signal variable. This contains the value that you want to forecast. The signal variable has to be continuous, with no missing values.
• Your dataset contains a column for the date variable.
  Note

  The date formats must be:

  • YYYY-MM-DD
  • YYYY/MM/DD
  • YYYY/MM-DD
  • YYYY-MM/DD
  • YYYYMMDD
  • YYYY-MM-DD hh:mm:ss

  where YYYY stands for the year, MM stands for the month, DD stands for the day of the month, hh stands for hour, mm stands for minutes, and ss stands for seconds.

  Example

  January 25, 2018 will take one of the following supported formats:

  • 2018-01-25
  • 2018/01/25
  • 2018/01-25
  • 2018-01/25
  • 20180125

• Forecast accuracy can be improved when the training dataset includes influencers. These are other variables that you think may have an influence on the signal variable. While values would normally be available for these over the observation period, you must ensure that values for influencers are also provided for the period you want to forecast. If values for influencers to cover the forecasted periods are not available, the predictive model won’t be successful, as you need observations to cover all of the requested forecast dates.

  For example, if you want to forecast chocolate sales for the year, you could add the specific dates of festive occasions to your dataset, for example, Easter, Christmas, Mother’s Day, or Valentine's day.

  Note

  The predictive forecasts will be influenced by the quality of the data you provided: both the historical data but also the future values for the influencers can impact your predictive model's accuracy. In some cases, it can be difficult to provide data with high quality for future values. For example: Future values for weather information can only be forecasts.

Example of a training dataset used for a classification predictive scenario

In the example below, you want to predict if a customer will buy a product or not. You have prepared a training dataset containing the historical data on customers that have previously bought a similar product. In this dataset, the values of the target variable ("did the customer buy the product?") are known .

Once you have created your predictive model, you apply it to an application dataset. This dataset contains the same information about the customers you want to target with the new product. The target variable column ("Will_Buy?") is empty or even doesn't exist because this is what you are expecting to predict:

Smart Predict will use the predictive model to calculate the probability that each customer will buy the product. The column "Will_Buy?" is now filled in the generated dataset:

Example of a training dataset used for a regression predictive scenario

In the example below, you want to predict the number of complaints that your customer support will receive this week. You have prepared a training dataset containing historical values for several previous weeks. In this dataset, the values of your target variable ("how many complaints per week") are known:

You apply the regression predictive model to a new dataset which contains the same influencers. The values of the target variable for this week ("number of complaints this week") are unknown.

The predictive model makes a prediction of the number of complaints that can be expected this week.

Example of a dataset used for a time series predictive scenario (Non-segmented):

The training and the application are done at the same time: it consists of generating N forecasts in the future. In this example, we want to generate product sales forecasts for the next 3 months.

The generated dataset will look like the one displayed below:

You can include influencers in your data model to refine the forecasts. In fact, this is highly recommended for improving forecast accuracy. The values for these influencers are filled in for the next 3 months as well:

Example of a training dataset used for a time series predictive scenario with entities:

You can also forecast product sales for multiple products. You will need to add the product-related information to the dataset. In Smart Predict, you will specify that the variable "Product Name" will be used to split your predictive model into distinct entities:

In the generated dataset, the observations are divided into "Product Name" and you will get forecasts per product:

Of course, you can combine entities and influencers. In fact, this is recommended if you want to increase the accuracy of your time series models.