Email
sinha.rk.dr@gmail.com
Are we losing important demand signals through ‘Outliers’ classification?
Supply Chain teams carry out an elaborate procedure to clean the incoming demand data, before using it for demand forecasting and demand sensing. They deploy algorithms to identify outliers in the data before training their AI/ML models. The outliers are either ignored (removed from the training data set) or replaced by ‘expected’ values.
Are we losing valuable information about demand signals in this process?
Outlier detection is a heuristic, using thresholds defined by the user. These are subjective. What may be an outlier for me may just as well be normal for you.
When we ‘correct’ outliers in the data, we may be losing an important demand signal, considering it as ‘noise’. When covid hit in 2020, the entire data for the first few weeks would have looked like an outlier. Was it just noise or did it contain a powerful signal? Of course, it did.
Remember that noise is nothing but a signal we haven’t understood or deciphered yet…