U.S. Water Utility Breach and ICS Cyber Security Lessons Learned

Security experts worry that the growing dependence on internet-connected devices is outpacing our ability to secure them. This is particularly true within industrial and critical infrastructure because cyber threats could result in physical disruption, loss of availability and even risk to public safety.

On the other hand, many ICS professionals continue to feel that the actual threat to plant operations and industrial automation is slim given highly purpose-built industrial equipment, specialized communications protocols, air gaps and unique automation systems and processes. Unfortunately, that’s not what the data shows.

This chart offers a snapshot of which sectors experienced cyber security incidents in 2015. Critical manufacturing, energy, transportation and water sectors were most affected. Source: NCCIC/ICS-CERT Year in Review, 2015 (page 19)

Example Scenario: Hack of U.S. Water Utility and Treatment Plant Utility

As some say, “offense informs defense,” so let’s examine a recent industrial incident and then summarize some useful lessons learned.

An unnamed water district, dubbed the Kemuri Water Company (KWC), experienced unexplained patterns of valve and duct movements over at least a period of 60 days as described in Verizon’s 2016 Data Breach Digest. It was discovered that attackers were manipulating the chemicals used to assure safe drinking water, and also altering the water flow rates causing disruptions to water distribution. Many other activities went unnoticed, including theft of more than 2.5 million unique data records, until Verizon’s forensic investigation started.

In this case, physical harm and safety was at risk but luckily didn’t happen due to alert functionality that caught the chemical and flow control issues. Also, it appeared that the type of outside attackers who gained access were likely “hacktivists” – usually not motivated by financial gain.

Verizon discovered that attackers were manipulating the chemicals used to assure safe drinking water.

What’s Wrong with This Picture?

A diagram of KWC’s network, as depicted in the 2016 Verizon Data Breach Digest. 

Verizon’s forensic investigation found that three known threat actor IP addresses had gained access multiple times to the water district’s OT and IT assets, including:

• The SCADA application, valve and flow control applications and the PLC systems
• IT management systems
• Internet webserver application
• Financial and customer account information
  
  

Cyber Security Lessons Learned

KWC had multiple foundational security control weaknesses or exploitable vulnerabilities that Verizon said made them a great candidate for easy hacking:

• Weak Password Hygiene – Water customers used an internet payment application to access their accounts from laptops, desktops or mobile devices. This application only required weak credentials (user name and password – no second authentication factor) to gain access to customers’ personally identifiable information (PII), payment data and water usage.

• Direct Internet Access to ICS (and bad network architecture too) – The internet-facing webserver that hosted the customer payment application was directly connected by cable to the AS400 system, which in turn housed the SCADA management application, giving the administrator (and threat actors) access to interact with the control level. The water district’s valve and flow control application on the AS400 was used by the three known threat actors to manipulate the PLCs and water chemistry.

• Privileged Administrative User – The lone AS400 system administrator had no corporate oversight and for convenience was using the same login credentials for remotely accessing both the AS400 and the payment application webserver from his laptop.

• Login Credentials in Cleartext Available from the Internet – A simpler way to say this? “Hey, here’s how to log onto our AS400!” The AS400 login credentials and IP address were found in clear text within an initialization file (.ini) – an old-school technique known as “Security through Obscurity.” The same credentials worked to log into the payment application webserver. 

• Single Point of Failure – One AS400 served as the water district’s SCADA Application system. The system was old, operating system updates were not installed, nor were patches, and again, one lone administrator working to make things easier but not with security in mind. Need we say more?

• Unnoticed Data Exfiltration (“exfiltration” is cyber security parlance for saying “electronically removed from the premises.”) – Over 2.5 million unique records were stolen. This was good news, because the bad news was that the other activities indicated the hackers had greater interest in disrupting and denying the water district the ability to conduct their business – up to and including the potential for causing public harm.
  
  

Summary

It’s easy to believe “it could never happen to us.” However, noting the weak or absent foundational security controls in the Kemuri analysis gives pause to consider what your environment holds. You may not realize similar risks are probably present to some degree.

Maybe it would be a stretch to catch plant engineers or contractors charging their phone or tablet on your PLC or HMI USB ports or allowing a contractor or family member wireless access from the hidden router in the back room.

However, most security practitioners recommend taking a risk-based approach to address your specific site through a third party cyber security assessment.

Do you think any of these risks (and others) could be present in your environment, increasing cyber security risks more than you know? Belden’s industrial cyber security solutions from Tofino Security, Tripwire, GarrettCom and Hirschmann are integrated and can help your organization detect, prevent and respond.

Contact us if you’d like to talk to one of our industrial cyber security experts.