Prescriptive Data Science: Beyond Predictive Data Science
Prescriptive Data Science: Beyond Predictive Data Science
(1) Descriptive, predictive, and prescriptive data science
(2) Required skill-sets to excel in prescriptive data science
(3) key components of prescriptive data science and common misconception
As companies become more mature with data science, predictive data science becomes more of table stakes. Prescriptive data science are gaining more importance in setting the leaders in the data science and artificial intelligence apart from other companies. In this article, I will discuss (1) different types of data science and their relevance with the organizational maturity in data science and analytics, (2) required skill-sets to excel in prescriptive data science, and (3) key components of prescriptive data science and common misconception.
1. Data Science Journey — Descriptive, Predictive, Prescriptive
The business needs for different types of data science change with analytical maturity of the organization. Initially, the focus is providing more “fact base” for day-to-day business decision making. The relevant business question is “What happened?” or “What is going on?” As an example, a retailer ran many different kinds of promotion events in the past, and this company would like to know what is the ROI of each promotion event and what promotion types are driving more incremental profit.
Once an organization become more analytically mature in its data science journey, the business questions evolve into planning for future. The business question here is “What is going to happen for Y if we change X?” As an example, a retailer is planning for Black Friday promotion events, and this company may consider a set of promotion options as candidates. Predictive data science can help to make forecasts under different promotion scenarios.
Having predictive capabilities is great, but most organizations get to realize that there are too many options to consider. At this stage, the business question is “What should we do (among the choices of X’s)?” As an example, a retailer would like to have a promotion recommendation engine, which gives recommendation on what promotion event to run (e.g. Buy 2 Get 1 50% off) at what discount level (50% vs. 30%), at what timing (2nd week of July), at what location (Denver), and what products (Coca Cola Regular 2L Single Bottle).
The graph below summarizes data science journey with organizational maturity with data science. Please note that the relevant data science tool kits also change along this journey, as shown on this graph.
2. Prescriptive Data Science: Required Skill sets
What is the required skills for data scientists when companies focus more and more on “prescriptive data science?” Many readers may be already familiar with 3 circles of skill requirements for data scientists: (1) business domain knowledge, (2) statistics / machine learning (ML) / artificial intelligence (AI), and (3) software engineering. With “prescriptive data science”, an additional skill on “optimization” is required to be more impactful data scientists. “Optimization” is usually in the domain of “Operation Research” or “Industrial Engineering.” Please note that typical data science curriculums do not cover this topic in depth. The visualization below shows 4 major dimensions of skill requirements for the next generation of data scientists, with more details on each dimension.
3. Key components of prescriptive data science and common misconception
What is “optimization?” What are the key components of “(mathematical) optimization” for prescriptive data science? There are many confusions on the definition of “prescriptive data science” or “optimization.” In case the readers are not very familiar with (mathematical) optimization yet, I am describing key components of optimization below. In addition, I also show what is not “prescriptive data science.”
- (1) Objective (function): This is a quantitative measure of the performance that the business uses to evaluate the business outcome. Most of time, total profit (i.e. bottom line) and total revenue (i.e. top revenue) are frequent candidates as an objective in optimization. However, other business metrics such as customer life-time value can be considered depending on the relevant time horizon and the nature of business questions. Please also note that it is possible to have multiple objectives with varying importance (e.g. 80% profit + 20% revenue)
- (2) Decision variable: These are a set of variables that the company can change. As an example, in the “joint price optimization” (i.e. pricing optimization) for retailers, a set of prices for 10 products in a given category (e.g. candy) is an example of decision variables
- (3) Constraints: Often times, businesses have a set of business rules or guard rails that need to be considered to make final recommendation for business actions. It is very common to apply these rules after initial attempt of optimization, which has unexpected consequences of over-riding optimization results. More proper way is incorporating these set of rules as “constraints” in constrained optimization. These rules can enter as “upper / lower bound” constraints, “equality” constraints, or “inequality constraints”. As an example, in the “joint price optimization” problem, a group of products may need to have a same price due to the contractual agreement between a retailer and a manufacturer or due to the needs of simplification for business decision making — so called “line pricing.” This line pricing rule can enter as “equality” constraints for the “constrained pricing optimization” problems. If product 1 price should be same to product 2 price, this can be specified as: price (product 1) = price (product 2). If a retailer is reluctant to make more 10% increase in price from current price level, this can enter as upper bounds for price increases: i.e. price (product 1) ≤ 1.10 x current price (product 1).
The picture below summarizes key components for “prescriptive data science” (i.e. mathematical optimization.) In addition, it also shows “what is is not.”
Hope this article helps you to better understand data science journey with organizational maturity with data science, additional skill set for prescriptive data science, what are the key components of “predictive data science”, and what it is not. In the next article, I will give an overview of different types of optimization methods and what mathematical and analytical techniques are available. If you are a Python data scientist, a good starting place for “prescriptive data science” is SciPy.optimize library.
Comments
Post a Comment