Black Belts + Machine Learning Can Be a Powerful Combination
Manufacturing organizations learned some hard lessons over the last two years. Prior to the pandemic, digital transformation initiatives were conducted mostly as speculative and experimental. In 2018, $1.3 trillion was spent on digital initiatives globally, yet 70% was wasted. A lesson learned from the pandemic era is that the robustness and resiliency of operations need to be prioritized for digital initiatives.
Digital initiatives prior to the pandemic were poorly structured activities, often performed without sufficient operations involvement. These initiatives introduced some machine-learning-based solutions, but they were unfocused, with little payoff. What has become clear is that machine-learning systems cannot be used in isolation from operational teams if we are to solve important labor market and utilization challenges.
We propose an approach that uses data-driven machine learning directly in support of process experts and operational teams to accomplish digital implementations. We co-opt the term “mixed initiative” from artificial intelligence (AI) literature on human-computer interaction to describe our approach.
Process experts, especially (Six Sigma) black belts, possess immense knowledge and are trained to analyze data, but they run into constraints with:
- the amount of data they can process
- the number of confounding variables
- lack of real-time data
On the other hand, machine learning can operate on vast amounts of data, but is limited by:
- lack of reliable data about observations and outcomes.
- lack of contextual information
- relationships that are hard to model
Black belts understand the types of relationships and insights needed for transformational change. Machine-learning models can quickly identify patterns of interest by sifting through large amounts of data. In our mixed-initiative approach, we leverage these strengths jointly. We propose that black belts define the context and objectives, while machine learning engines extract relevant models and patterns from available data. Black belts then validate results and use machine-learning systems to operationalize meaningful process improvements.
We illustrate this approach on the problem of identifying and operationalizing a “golden run.” A golden run is a benchmark performance period where the process achieves the “best” performance. Identifying a golden run requires analyzing process history to find process parameters and recommended settings for reliable replication of this best performance.
A mixed-initiative approach to identifying golden runs can be shown in Figure 1.
Figure 1. A mixed-initiative spproach to identifying Golden Runs
This process has four stages; some are driven by the black belts (blue) and some performed by the machine learning engine (yellow). We describe these stages below.
1. Define golden run context
Black belts frame the problem. They identify the characteristics or conditions that can affect the process and what they believe are the important relationships to uncover. For instance, conditions that introduce change to the process, such as: product changes, seasonality, day/night, new employees, raw material changes, equipment maintenance activities, etc. Process knowledge informs this set of potential conditions and is necessary to fill in gaps that the machine learning engine and data alone cannot. This does a few things:
1. Highlights important considerations for process improvement
2. Reveals sensor and data collection needs.
3. Defines the search space for the machine-learning engine.
These conditions give context for the golden runs, including the influence space of the golden run (e.g. by run, by shift, by product, by season). Additionally, black belts can specify the type of product being manufactured, the process step represented, a specific line on the factory floor, etc. The machine-learning engine uses these to identify golden runs that can be tested. Black belts can be expansive in the number of conditions and in their combinations and relationships that the machine-learning engine needs to explore.
Black belts also provide the objective. The optimization objective defines “best,” i.e., the outcomes desired during operation. For example: Achieve the fastest possible production while ensuring good quality. It captures tradeoffs between overall equipment efficiency (OEE) components such as quality, performance, and availability, and can also include factors like power consumption, cost of production etc.
2. Analyze history to extract golden run
The search for the golden runs is performed by the machine-learning engine given the context:
1. Identify candidate periods from history for the golden run. This involves training models to pre-process and learn patterns of stability from the multivariate process data.
2. Search through possible candidates to find (or synthesize) those that best optimize the objective provided by the black belts. This involves a combination of machine learning and constrained optimization strategies.
3. Extract dominant process control parameters that differentiate the best candidate from others, and develop recommendations for their settings. This involves extracting key contributors through a combination of statistical or information measures, and predictive feature selection methods.
Black belts are then provided with a set of potential golden runs extracted from the data.
3. Validate the golden run recommendation
Black belts then validate the golden runs identified by the machine-learning engine. Validation includes deep dives into the data to understand the results also performing confirmation runs. These are experiments to confirm results under the conditions of interest. Figure 2 below illustrates the validation process.
Figure 2. Validating a Golden Run recommendation
In some cases, results may not be validated because of incomplete context or issues with the data or machine learning engine. In these cases the team “goes back to Step 1” and develops additional potential context for the golden run. This refinement also can improve the data and the strength of the algorithm.
4. Deploy recommendation
As the machine-learning engine identifies golden runs under different contexts, a library of golden runs is developed. An illustration is shown in Figure 3.
Figure 3. Representation of Golden Run library
Each sub-group of short-term parameter distributions represents one golden run. Transitions between sub-groups represent changes in the context, and transitions between golden-run periods. Black belts then fill in more information about how the parameter distributions encompass all of the variation, represented by the large distribution to the right.
The goal of this stage is to operationalize the recommendations. This requires using machine control to set up the runs and shorten transition times between golden-run periods. Two things are required here:
1. The machine-learning engine must be trained to identify transitions as they are happening.
2. The control algorithm must select the appropriate golden-run from the library to set up the process and achieve optimized performance under most conditions.
Until trust in automation is built, black belts support the operations team with the control loop. Of course, there is a continuous learning cycle here—the library has to be updated as the operations team learns about other possible contexts and the process changes.
What about Operations Teams?
Digital is only partially about technology; it is mostly about people. The nature of work will change as technology adoption increases. Perhaps unique to this moment in time is the demographic change also happening in the workforce. As millennials enter middle management and Gen Z’ers enter the workforce, understanding of possibilities and expectations of technology increases. The process of operationalizing autonomous operations will accelerate.
This starts by leveraging a mixed-initiative approach to build a library of trusted golden-run recommendations, and training operations teams on how to interpret recommendations, take action and refresh the context for the machine-learning engine. As millennials and Gen-Z’ers move up in organizations, this trust will pay off as they identify operations and steps that can be turned over to the machine to control. Success in this transfer feeds more opportunities to fine-tune the models, making them stronger, eventually leading to a self-sustaining digital control process.
Deepak Turaga is senior vice president of data science at Oden Technologies, an industrial IoT company focused on using AI to monitor, optimize and control manufacturing processes. He has a background in academic and industry research specializing in using machine learning based tools to extract insights from streaming and real-time data. He is also an adjunct professor at Columbia University, and teaches a course on this topic every spring.
James Wells is principal consultant at Quality in Practice, a consulting and training practice specializing in continuous improvement programs, and specializes in quality fundamentals, including the application of digital solutions to common manufacturing challenges. He has led quality and continuous improvement organizations for over 20 years at various manufacturing companies. Wells is a certified master black belt and certified lean specialist.
