Cereal-Maker Post Consumer Brands Takes the Bite out of HMI Costs with Virtualization

What began in 1895 in a barn in Battle Creek, Michigan, is now America’s third-largest cereal company: Post Consumer Brands (PCB).

Today, the company’s diverse portfolio generated more than $1.2 billion in sales in fiscal year 2015 and accounted for 21 percent of volume market share in the ready-to-eat cereal category.

The Battle Creek barn where Charles Post made his first cereal product still stands but is now dwarfed by a 64-acre production campus – one of eight PCB production locations in the U.S. and Canada. It’s here that PCB makes some of America’s most recognizable cereals, including Grape-Nuts®, Post Raisin Bran®, Fruity Pebbles™ and Cocoa Pebbles™, and Honey Bunches of Oats®.

Recently, PCB began a multiyear migration at its Battle Creek campus to replace all of its human machine interface (HMI) platforms, which had aged to the point that their operating systems were obsolete and they were experiencing regular failures. Rather than simply replacing the HMI platforms, PCB decided to adopt an entirely new approach for its HMI system by leveraging the power of virtualization.

Stale HMI Architecture
The Battle Creek production campus includes four manufacturing facilities, a warehouse and dry-ingredient storage silos.

Production differs by cereal product, but common processes include milling grains into flour, cooking or toasting the grains, shredding or pressing the grains into their familiar shapes, and mixing in ingredients such as raisins, nuts or seeds. Finished cereal products are then packaged into bags, boxes and cases before they are sent via conveyors to the warehouse for storage and shipping.

Operators at the Battle Creek campus use 175 computer-based HMIs to monitor and manage all of these activities – from unloading ingredients off railcars to executing production processes to conducting warehouse activities. However, in recent years, the HMI platforms increasingly presented a number of problems.

First, because the HMI platforms were desktop computers and not built for harsh manufacturing environments, they were beginning to fail at an almost weekly rate. Also, most of the HMI computers were running on Windows® XP, which Microsoft ended its support for in 2014.

“Windows XP obsolescence was a big driver for us,” said Bill Menser, electrical engineering lead for PCB at its Battle Creek campus. “It got to the point where I could not buy new computers that had drivers for Windows XP. So I started collecting old desktop computers. Or I would buy new computers and make trades with the IT group for their old computers – all because I needed spares that could run XP.”

Beyond the operating system, there was the separate matter of needing to be sure spare computers had the right HMI software. This could be a challenge because the legacy HMI computers scattered throughout the campus used a mix of Rockwell Automation software, including FactoryTalk® View Site Edition, and standalone and RSView®32 Active Display System versions.

“An HMI platform failure takes us about six hours to rebuild,” Menser said. “That’s certainly a pain point for us. It’s time where we might not have immediate access to production because operators needed to use another HMI, which might be 50 feet away or on a different floor. But a bigger risk is running out of replacement hardware. Even if we have a computer with the right operating system, we have to make sure it has the right generation of software to match all the different versions running in our buildings.”

PCB’s Battle Creek site also was using about 20 HMI servers across its site, with two pairs of servers in most buildings. Like the HMI computers, the servers were aging and experiencing hard-drive failures within the production site’s harsh manufacturing conditions.

“I’ve seen about eight hard-drive failures in the three years I’ve been here,” Menser said. “None of them took us down because we have redundant servers in place. But again, replacements are challenging for these systems, and the possibility that we would one day run out of spares is a threat that could eventually bring us down.”

Virtual Transformation
Amid obsolescence and hardware-failure challenges, Menser knew it was time to replace the HMI computers and servers throughout the PCB campus. Rather than replace them on a one-for-one basis, he opted for a multiyear migration that would completely overhaul the site’s HMI system with modern virtualization technology.

This includes replacing and consolidating the approximately 20 HMI servers down to fewer than eight servers when the project is complete. The servers will support a total of about 50 virtual machines.

Those virtual machines will run on thin-client HMIs, which are replacing the 175 desktop-computer HMIs. The thin clients require no operating system, and they all will be standardized on the current version of the FactoryTalk View Site Edition software in place of the mix of HMI software currently used on the legacy HMI computers.

The new HMI architecture will use the VMware virtualization software, which enables the thin clients to run as virtual machines. It also will use the ThinManager software from ACP, now a Rockwell Automation company, to centralize management and delivery of the HMI software from the servers to the thin clients.

Immediate Benefits
While the HMI virtualization migration is only about one-third complete at the Battle Creek production campus, the site is already reaping benefits.

Replacing aging and failing HMI computers with virtualized thin clients is helping avoid obsolescence risk with every platform that is changed over. And because the thin clients are designed for harsh industrial environments, unlike the desktop computers, they have a vastly improved life expectancy.

Should any of the thin clients someday fail or be upgraded to a new technology, Menser and his team will be able to replace them without needing to find a spare computer with Windows XP.

“Instead of rebuilding the HMI from scratch, all we need to do is replace the thin client,” Menser said. “That drastically improves replacement time, from about six hours to a matter of minutes. It’s literally just a matter of the time it takes someone to walk to the HMI and swap out the device.”

For operators, the new HMI thin clients are replicating the look and feel of the legacy HMI computers to reduce potential production disruption or the need for training on the new system. The system also is connected to newly installed IP cameras, giving operators unprecedented visibility into production within the massive campus.

“The video connectivity from any HMI can help operators monitor key processes, such as checking on the distribution of cereal during the drying process,” Menser said. “It also can help in large areas where operators aren’t stationed but where issues may still occur. For example, with a virtualized, video-enabled FactoryTalk View SE HMI system, we can instantly check to see if a conveyor line is jammed. In the legacy system, someone had to take several minutes to physically walk there and check the conveyor in-person. Saving those minutes during production directly equates to improved yield and profitability.”