How To Optimize The Value Of An Operator Training Simulator
When assessing the value of an investment in training simulator technology for your plant, it's important to first consider how and in what type of facility the training simulator would be used. For example, would the simulator be used to train plant managers, engineers, and/or maintenance technicians as well as operators? Would the simulator also be used to test control strategies in a safe, offline environment? And, perhaps most importantly, is the target plant a new, grassroots facility, or an existing brownfield facility?
[pullquote]
The answers to these questions will help end users determine the best types of technology to implement for simulating both the process itself and the control system. Depending on what the end user hopes to accomplish, the relative fidelities of both the process and the control system simulations should be aligned to optimize the value of the technology and, ultimately, the return on investment.
Why does it make a difference whether the plant is a new or existing facility? In a new facility or a plant expansion, the cost of the operator training simulator (OTS) can often be budgeted as part of the overall project, for which it would typically represent a very small percentage and thus not be overly difficult to justify. The need to train operators in a new facility is also usually more obvious than in an existing plant, in which the operators have already gained a consider-able amount of operating experience. Furthermore, in established plants that have often been in operation for decades, the, "We've managed fine without one so far," mentality often dominates.
Unless there has been an obvious pattern of operator errors leading to "near-misses," requiring operator qualification and/or re-qualification, it can often be difficult to justify the investment required to purchase, implement, and maintain an OTS in an existing plant. This is particularly true if the plant goes the high-fidelity route for both the process and control system simulator models. On the other hand, an OTS can add obvious value in existing plants when a new or significantly upgraded control system is installed with which the operators do not have experience.
To optimize the value of an OTS, it's important to match the plant requirements to the available technology solutions. This ARC Advisory Group Insight provides some general guidelines for doing so.
Options for Control System Simulation
In the past, before simulators were available, both training and engineering for control systems (DCS and PLCs) were typically performed right on the actual control system. Once the control system was installed, training occurred while controlling the plant processes. This limited the scope and effectiveness of operator training and left the control engineer without any development or test system.
Basic Configuration for a Control System Simulator
As ARC consultants learned during a number of recent visits to process plants, many plants and mills today have only the running control system in place for training and testing new control schemes. This is unfortunate, because a wide variety of options for both simulating the process control system and simulating the actual process are available today. These range from generally low-cost, generic off-the-shelf control and process simulators to highly customized high-fidelity simulators that represent exact models of both the control system and the process and further enhance the learning experience using 3-D virtual immersive plant environments; with many alternatives in between. This ARC Insight will focus on those “in between” alternatives that can often be easier to cost justify in existing plants.
ARC has written extensively about the exciting advances being made today in immersive-type training simulators that simulate actual plant environments in amazing detail (often down to the vibrations, sounds, and smells one would encounter in the actual plant) and employ avatars that place the trainee in this highly realistic immersive environment. While research indicates that this can enhance the learning experience and improve retention significantly, this type of technology is often too costly and sophisticated for the majority of today’s brownfield plants unless they are undergoing a major capital plant expansion or technology upgrade.
Control System Development and Checkout
In years past, control system software for greenfield projects was developed on the actual staged control system. One hundred percent of the control system I/O was tested to make sure it was operational. Test panels were wired to the I/O points. These test panels had switches to drive digital inputs and potentiometers to drive analog inputs. They used lights to read digital outputs and gauges to see analog outputs. At the same time the hardware was being tested, engineers would be able to check HMI graphic displays, alarm settings and certain control functions like controller action, trip settings and alarm functions. This open loop simulation represents the lowest level of fidelity because the simulation engineer, rather than actual process dynamics, sets the process behavior. The I/O can be set to any steady state value of the process and variables manually manipulated one at a time to test the control system software. As long as the system was being staged, it could be used for training operators and testing the control system; although any semblance of “simulation” took place in the minds of the engineers turning knobs and flipping switches.
Today, the control system software is developed by a team of engineers on offline personal computers, often using network servers and, increasingly, cloud computing. To support the control system development effort, most DCS suppliers and many PLC suppliers offer relatively low-cost PC-based control software configurators that create the run-time control software ultimately deployed in the actual control system. The best systems can both configure and execute the control functions in this virtual environment. The ability to execute the control functions in a virtual control system allows options for both operator training and control scheme development and testing.
Turning the Development PC into an OTS
What many end users don’t recognize is that after the control strategies have been developed and deployed to the target controllers, by keeping the configuration up to date, the development software running on an offline PC (in effect, a virtual control system) can also be used as a simulator to help train operators for years to come. The only change normally required would be to toggle the I/O-related function blocks to disconnect from the physical I/O and switch to simulated I/O.
While staged control systems can be connected to physical hardware, which communicates with the process simulators, this configuration is rarely used for operator training. The virtual control system is much simpler with no hardware since it can connect to a dynamic process simulator using standard networking methods.
Although, in principal, it is possible to use the standard DCS or PLC function blocks to simulate the process, it’s usually most efficient to connect the virtual control system to a simulation software solution. Purpose-built simulation software often provides the best option for creating a real process simulation at low cost and at an acceptable level of fidelity to meet the project goals.
Align Fidelity to Specific OTS Requirements
Do you want to train the operators on how to use the control system? In this case, control system fidelity needs to be high (identical), but the fidelity of the process simulation can vary depending on the extent that learning the process behavior is part of the training goal.
Do you want to train the operators on how the process works in the steady state or dynamically? In this case, the process model needs to have high fidelity. Without high fidelity, the operators will learn how to control the simulation but the real process will behave differently. Dealing with process hazards and quality issues may require additional training beyond an OTS.
Even the lowest level of fidelity can be used to accomplish training goals if properly managed. Achieving simulation fidelity that is indistinguishable from the real process is nearly impossible to do for any complex process. Often a simulation that is directionally correct from a steady-state operating point, can be acceptable for most purposes. We often hear simulation experts repeat the mantra: “All models are wrong; some are useful.”
Optimizing Fidelity of Process Simulation for a “Brownfield” Facility
As shown in the graph above, the optimal process simulation from a value minus cost perspective tends to peak at lower model fidelities for brown-field facilities. The reason is that low fidelity process simulations can still be used effectively for training. Each plant may define fidelity differently or value fidelity differently so the optimal fidelity can vary. Higher fidelity of the process simulation can often be justified when you can reduce the cost of building and maintaining the simulation. Simulation vendors have made some progress doing this, but it is inherently difficult because real processes have so many complex parts that change behavior with time. As a result, many high-fidelity simulations developed for new facilities have been abandoned. Many existing facilities have upgraded to a new control systems, so training goals are skewed to learning the new control system with operators and workers that already know the real process.
OTS as Development Platform
Whether using a standard OTS solution or one custom-designed for your plant, if you have both a virtual control system and a process simulation to train operators, you also have a software test and development platform that can be used for that purpose. Configuring complex control schemes on the live control system presents several risks to the process operation. In facilities where the control engineers do not have a development or test platform, they must do all their work on the running control system. It is no wonder they keep control schemes as simple as possible and avoid advanced control schemes. Although taking on the support for an OTS system is some extra work, the control engineers can benefit from the dual use.
Plan Ahead to Maximize Synergies
With a supplier-supported virtual control configuration platform, the OTS system can be used to train operators and other staff with the identical HMI, controls, alarms and other control system software. As training is a part-time activity, control engineers can also use the OTS platform to help develop and test control hardware and software, HMI, alarming and trending. Plant control engineers obviously understand both the process and the process dynamics, and are in a good position to maintain simple process models, possibly with some support from simulation suppliers.
Plants can get more value out of the OTS when synergy with the control solution is planned from the start. ARC has observed that in too many instances, a sophisticated OTS solution is abandoned over time because it is too difficult for the end user to support. There is great opportunity for the synergistic use of an OTS system in existing facilities, especially those that have new control system upgrades. The key points are to use vendor-supported virtualization tools, to make an exact emulation of the control system. There is tremendous value when operators can navigate the identical HMI screens and interact with a simplified process simulation to explore rarely used control functions like equipment trips, overrides, failures modes, and start-ups. Whenever possible, it’s important to keep the simulation fidelity at a level that the plant control engineering group can support.
ARC surveys indicate that OTS users want more fidelity; but they also want lower cost and easy-to-develop models. Creative minds are developing new technologies, and new products are coming. End users should carefully plan their operator training solutions to help ensure that they can achieve short and long-term training goals. Consider the synergistic benefits of managing the OTS within the process control group at the facility. By keeping a multi-use simulation platform up-to-date long term, they can gain great value to improve operator performance to minimize mistakes, while improving the process control performance of the facility.
ARC Advisory Group has prepared several market studies and client reports on OTS, including an OTS market research report, OTS supplier selection guide, and recent OTS strategy reports, including a client report on cost justification. We will continue to follow and report on future developments.
Rick Rys is a Senior Consultant at ARC Advisory Group. He performs research into and consults with clients on technology areas such as process automation, energy management, advanced process control (APC), simulation and optimization.
Over the years, Rick has provided consulting and engineering services to clients in every major industry including pharmaceutical, electric power, chemical, and oil & gas. Prior to joining ARC, Rick was an independent engineering consultant at R2Controls since 1996. He worked for Foxboro (formerly Invensys and now Schneider Electric) for 20 years in the process industries. Rick holds a BS in Chemical Engineering from the University of Massachusetts and is a registered professional chemical engineer in Massachusetts.
