Efficient upfront planning and the flexibility to adapt during execution are hallmarks of successful projects. For a recent project at Sipchem, Saudi Arabia, an integrated project team proved even more critical. What started out as a routine migration project with a ten-month timeline became a ten-week sprint to meet a new target.
Sipchem is a chemical manufacturer with customers in the construction, solvents, automotive, electronics, polymer, coatings and pharmaceutical industries. Like many others in its industry, the company grappled with aging process control equipment and the associated obstacles. Sipchem recognized the risks of production loss due to obsolescence, diminishing vendor support and lack of spare parts inventory. It also wanted to upgrade software to meet its internal standards for cyber security, to protect the safety of the system as well as the business.
So, the company decided to upgrade the automation systems of the carbon monoxide (CO), vinyl acetate monomer (VAM) and acetic acid (AA) plants at its complex in Jubail Industrial City, Saudi Arabia, to the latest version of Emerson Process Management’s DeltaV distributed control systems (DCS). The upgrade at each plant included replacing four servers and six operator stations, the software for approximately ten controllers, and swapping six traditional switches with “smart” switches.
Project planning began in the fall of 2012; capital spending was approved in January 2013.
[javascriptSnippet ]
Obstacles Emerge
It quickly became apparent that, to minimize man-hours and reduce risk to production, the upgrade should occur while the plants were down. We knew it would not be possible to take the plants down more than once. That meant pressing forward to be ready to upgrade at the next planned shutdown — which was in ten weeks.
The shortened timeline presented an immediate challenge. The new workstations to be shipped from the United States had a 60–90-days lead-time for delivery to the Sipchem site. Furthermore, Sipchem required the latest model of the computer, which was scheduled for release 15 days before the target implementation date.
Finding personnel to perform system upgrades simultaneously at three separate facilities put further strain on the project. All three plants share a control room but each has a different hardware rack room. As a result, operations and maintenance teams across these areas are separate. Radios must be used to communicate among teams.
Unfortunately, an assumption critical to the feasibility of completing the upgrades simultaneously proved false. Early in the planning process, the team thought all three upgrades could be completed offline. With further analysis, we realized the AA process could not be shut down fully because of safety risk to personnel working in the plant.
One of the products Sipchem manufactures contains acetic acid and methyl iodide. Shutdown of the scrubber unit for that product stream would cause release of hazardous chemicals. Depending on the size of the release, the chemicals could reach outside the plant to the surrounding area. Because of the plant’s design, the DCS must run continuously even during the production/plant shutdown. So, to prevent disruption to the scrubbing process, the upgrade would have to be completed while the unit remained online.
Careful Preparation
After discussion and analysis, the team developed a two-pronged approach to mitigate the issues and tackle the challenges of the upgrade. We decided to perform the upgrade twice at each facility: once on existing workstations and then again two months later when the new equipment arrived. The controllers would be replaced as part of the first upgrade.
The team anticipated the process of completing the upgrades for the CO and VAM facilities would be relatively easy. Therefore, we decided to upgrade these plants partially online to increase our confidence about the totally online upgrade of the AA plant. This required detailed and time-consuming analysis and preparation to ensure the inputs/outputs and controllers would not be affected. One small error could trip the whole plant.
To prepare for the online upgrades, we performed detailed contingency planning to ensure Sipchem was ready to respond to any situation that might arise. There was a mitigation plan in place, so if anything went wrong we could immediately begin acting to address the issue.
One Team, One Goal
Perhaps the most critical factor for the success of the project was the cohesiveness of the integrated project team. It included a project manager, process engineers and technicians from Sipchem along with subject matter experts from Emerson.
The project manager sought out people with previous experience with online upgrades and expertise on operation and safety. The Sipchem organization worked together to make the right people available to assess impact, identify risks and develop the mitigation plan, which addressed issues such as spare parts inventory and hardware shortages. We also identified other local experts to augment the team as necessary. Everyone was available, on-hand and ready if needed during the upgrade.
The specialists from Emerson leveraged that company’s global network to obtain last-minute materials. They didn’t focus on the time or money needed to get the parts but single-mindedly concentrated on ensuring the success of the project.
The team expended great effort to complete the upgrades simultaneously without affecting the plants. During implementation in April–May 2013, personnel had to work 24 hours a day, 7 days a week.
The team replaced the controllers as part of the first upgrade. We also swapped the existing DeltaV DCS’s switches with smart switches to enable Sipchem to take advantage of their cyber-security features. These switches, completely managed by the DCS, automatically report device alerts to indicate network communication irregularities and also easily allow for auto port lockdown to prevent unauthorized network connections.
During this period, a problem arose: one of the servers failed because of a power supply issue related to dust. A neighboring company with whom Sipchem has a good relationship provided a replacement server so the upgrade could continue.
Other issues occurred during implementation — e.g., the need to replace some old KVM switches and to troubleshoot OPC interfaces after the upgrade. However, each time, people came up with solutions and the work was able to continue.
Two months later, after getting the new workstations and servers, the second upgrade was completed while keeping the controllers intact. It took about 15 days and went smoothly with no disruption to process or production.
Payoff
The cyber-security upgrade brings added peace of mind about network security and new hardware support has improved operations. Obsolescence no longer is a concern. New parts can be obtained as needed. The DCS’s improved graphics and trend displays are easy to use. Sipchem is anticipating great value from the diagnostics. We also intend to develop the alarm help function as a key tool for ensuring a smooth transition as new staff join the company.
Because we worked cohesively as a single team, we were able to meet the accelerated timeline. Successfully upgrading three plants in three and half months was challenging but the result has exceeded expectations.
JAYAPAL A P is a staff process control engineer for Sipchem, Jubail Industrial City, Saudi Arabia. E-mail him at [email protected].
