In a recent blog article – Chicken, Egg, and Chicken Omelette with Salsa – Dale Peterson is squawking like a rooster. Nothing new, but this time his message is scrambled. He once again referred to me as a SCADA Apologist, though this time he also labeled me the “salsa” that accompanies a chicken omelette. While I responded to his opinion in my January 30 blog post, I’d like to revisit this spicy topic.
I am not a SCADA Apologist. If anything, I consider people like myself and Joel Langill to be SCADA Realists. Clearly Joel and I believe security is important. If we didn’t, we wouldn’t be in this business. And our clients don’t pay us to hear: “Do nothing; it’s the other guy’s fault.”
Like Dale, we believe that ICS needs to be secure. Where we differ from him is on how that can be accomplished.
Joel and I promote a strategy that is based on standards (the ISA/IEC 62443 standards to be specific) and that can be deployed in the real world. It is a strategy that considers a mix of the available solutions, including patching, compensating controls, and device replacement, as appropriate. The key here is “as appropriate.”
If a company can replace its entire control system… great! But people need to be realistic about the real costs of Dale’s “rip and replace” strategy.
As an example, we are working closely with a large Oil and Gas company that has several hundred old PLC5 controllers managing their gas turbines on their offshore platforms. These 20 year old controllers are certainly not the latest word in security. And for a mere $5,000 to $10,000 each, they could purchase new ControLogix CPUs. In theory, this means that they could rip and replace the whole works for $2,000,000. That’s chicken feed for a large company, right?
But not so fast…
You can’t just replace the CPU: touch the CPU and the whole turbine control system needs to be replaced and recommissioned. That will cost $250,000 to $350,000 per unit. So for the rip and replace effort that Dale proposes, the total cost would be around $70 million for those 200 units. While the control vendor would love to move ahead with this scenario, it is going to be a hard sell to the company board.
But it turns out that the money isn’t the big issue. You cannot just replace a CPU and start up your turbine the same day. In the case of the customer above, a major replacement process on a platform will require several months of downtime. So in order to replace all those PLCs the company would suffer years of lost production.
And what happens when somebody discovers a vulnerability in those new controllers? Do we rip those out and start all over again?
Dale knows that there’s no single answer to the SCADA security problem, including replacing all that equipment. Good security requires defense in depth. And that means using multiple security solutions. We need to be realistic and explore all attainable solutions.
There are alternatives to the rip and replace strategy. One option is to patch. Another is to install a compensating control for security vulnerabilities. For this there are several possibilities available, but one that I have personally believed in since my days in the BCIT lab is Deep Packet Inspection (DPI) firewalls. These devices, which include the Tofino, are designed to clean up the SCADA/ICS messages going to the PLCs.
The cost to purchase and install the needed Tofino units with the appropriate DPI modules is under $250,000. I’d say that’s a better value than $70,000,000 – and an easier sell to the people who approve your company spending. But most important, a SCADA technology like Tofino is designed to be installed without any downtime. For critical industries like oil and gas, that matters.
Imagine your ICS as a chicken coop. If you think foxes are getting in, you might consider tearing down the whole thing and starting again to build a more secure structure. Of course, there is the question of what you’ll do with all the chickens while you rebuild. So why not just leave the chickens where they are and simply patch your coop? Or you could install a compensating control (like a dog)! There is no single answer – every option needs to be considered carefully.
I am happy to see Dale drawing attention to SCADA and ICS security issues. Just as the rooster crows to get the day started, Dale is crowing to get the conversation started. This is a necessary step in addressing the problem. However, if companies continue to ignore the conversations around security and make excuses for their inaction, the control systems simply won’t get secured and the threats to them will escalate.
While Dale credits me with saying it will be decades before the entire critical infrastructure can be replaced, he fails to mention the second half of my point: that we must do something NOW. These are very different messages.
It’s time to stop just clucking about security and instead get busy cooking up solutions that work.
Do you have questions or opinions that are not addressed in this article? Do you have experience replacing control systems to address security issues? Send me your comments.
• Press Release: Belden Research Shows that Patching for Industrial Cyber Security is a Broken Model
• Press Release: Belden Helps Schneider Electric Secure Critical Industrial Infrastructure
• Blog: "Rip and Replace" Approach to SCADA Security is Unrealistic
• Blog: Securing SCADA Systems: Why Choose Compensating Controls?
• Webpage: Tofino Briefing Kit