Security

TACKLING EMERGING THREATS TO MANUFACTURING AND PROCESS CONTROL

Part of a series of white papers produced by Mitsubishi Electric Europe B.V.

Contents
Executive summary The threats The vulnerability Manufacturing systems Other technologies The role of the PC Alternative systems Added benefits Multicore technology Vendors Conclusion Mitsubishi Electric Green Hills Software The e-F@ctory Alliance Glossary 3 4 5 6 7 8 9 10 10 11 11 12 12 12 13-14

Executive summary
Business has grown at an exponential speed since the industrial revolution. Manufacturers are driven by cost cultures centred around production processes and system optimisation. The PC has become the universal gateway to information and the health of the company. However, there is an elephant on the shop floor and no one is talking about it. Its name is security. Awareness of security issues has been high on the agenda of all IT departments for many years. However, the manufacturing shop floor is the domain of engineering departments - and quite rightly so. Flexibility, speed, repeatability and reliability are necessary mantras of an automated business. What is frequently overlooked is that where there is a PC there are hidden threats. Industrial espionage, malicious system attacks, innocent holiday pictures on a memory stick are all routes to infect PCs with viruses or to steal information. Is this the stuff of Hollywood or the best seller book list? The government of the United States of America does not think so and neither does ISA (The International Society of Automation). However, there are solutions which, when intelligently applied, will yield systems with greater availability. High reliability server systems are common in companies running ERP, MES and business systems. Manufacturing automation has developed robust systems such as programmable logic controllers which also offer high reliability control. Why not plug these two secure systems directly together? Then PCbased tools such as SCADA (Supervisory Control And Data Acquisition) will act as windows into the system and not gateways. These can be easily added or replaced without compromising system stability, performance or security. The question all manufacturers should be asking themselves is If that PC gateway went down, how much money would we lose in production; scrap, rework, lost time and penalties?. Should we accept those losses when solutions could be implemented for a fraction of the cost?

The threats
Modern computer-based manufacturing and process control systems have delivered tremendous gains in productivity, quality control, manufacturing flexibility and other benefits. They have put managers more firmly in control of their enterprises than was ever previously possible. Yet the systems rest on the assumption that management is in control. The truth is that whoever controls the system is in control - and there are circumstances when that is not actually the management. There are tremendous risks to companies and the wider community if control is ceded to aggrieved current or former employees, terrorists, criminals or hackers. Possible nightmare - yet improbable - scenarios include: A pharmaceutical company being blackmailed by criminals who have taken control of the manufacturing process to alter the composition of medicines and demanding money to identify the affected batches. Terrorists changing the dosage of treatment chemicals added to the water supply - potentially causing widespread deaths or illness and certainly creating widespread panic and confusion. An aggrieved employee closing down a furnace at a glassworks in protest at a perceived injustice, leaving the company with no option but to manually chip out tonnes of solidified glass before the furnace can be restarted. Hackers accidentally or deliberately changing control parameters at a process plant, leading to an explosion and/or release of environmental pollutants. not These are not just theoretical possibilities. Most companies that are attacked do not publicise the fact (and many companies never know that their systems have been penetrated) but there are some wellknown examples of successful attacks, including: A 19 year old hacker bringing the Port of Houston to a standstill. A computer virus infecting the systems of CSX Transportation, halting passenger and freight train traffic in Washington DC. A computer worm taking the plant safety monitoring system of the Davis-Besse nuclear power plant in Ohio offline for five hours. A hacker in Brisbane, Australia, creating raw sewage overflows on the Sunshine Coast, a key tourist destination. The Aurora experiment1 where the Department of Homeland Security in the USA simulated a SCADA system attack which then destroyed a power generator. In 2009 the security agencies of governments on both sides of the Atlantic warned2 that hackers - allegedly sponsored by Chinese and/or Russian government agencies - had placed software in the control systems of national electricity grids, as well as US water and sewage systems, that could allow them to take control whenever they wished. Early in 2010 there were further government warnings that unidentified hackers (again allegedly linked to Chinese government agencies) had penetrated the systems of leading international companies. The extent of these attacks is unknown. However, security experts warn that future international conflicts may involve cyber warfare in which the combatants seek to cripple national infrastructure not with bombs and missiles but by taking control of or destroying computer systems. In the summer of 2010 the Stuxnet worm was reported to be targeting the SCADA and process control systems of a leading vendor.

The vulnerability
Many security problems arise from the vulnerability of the current generation of SCADA systems. Providing internet or intranet access to SCADA systems gives real benefits through remote monitoring and control of, for example, pumps, switches and valves on plant that is far from head office. Yet it also opens these systems to access by people with malevolent intentions. A study3 in 2006 found that many SCADA systems which companies believed to be secure because they were not connected to the internet were, in fact, connected. The connections had often been established, for good reasons, by support personnel logging in on unsecured networks from personal computers. Whether connected or not, SCADA systems are typically protected by weak password systems where the passwords are known to many people and are never changed. Computers and manuals seized in raids on Al Qaeda training camps in 2003 were full of SCADA information about dams and related structures. Internet and intranet access is often provided through systems based on common software such as Windows and Internet Explorer - yet even the latest versions of these programs have acknowledged security flaws. At the time of writing this white paper the website of US-CERT4 (the United States - Computer Emergency Readiness Team) lists security alerts for dozens of well-known programs including Windows*, Internet Explorer*, Adobe Reader*, Adobe Acrobat*, Flash Player* and Oracle*. Although the list changes almost daily the scope is typically wide ranging. A report5 published by the US Department of Homeland Security in 2009 identified a wide range of vulnerabilities typically found in industrial control systems - ranging from poor password protection and poor patch management (where security upgrades to proprietary software are not implemented) through to poor basic design of networks and firewalls. In summer 2010 there were reports6 of a virus targeting SCADA systems. The Stuxnet worm apparently specifically targets SCADA and process control systems produced by a leading international vendor.

Manufacturing systems
Historically, manufacturing systems were proprietary - they were designed and supplied by one company and would typically not interface with systems supplied by any other vendor. In recent years there has been an increasing trend towards open systems working to common standards that allow hardware and software from multiple vendors to be integrated into a single manufacturing or process plant. This is now the dominant business model across most of industry and it has brought many obvious advantages. It has also brought with it a number of security vulnerabilities. For example, market-leading standards such as OPC* (OLE for Process Control), while widely applauded for increasing system harmonization, have clear security weaknesses inherited from their PC origins. This is a point recognised and partially addressed in the emerging OPC UA (unified architecture) standard. The US-CERT website, as well as listing various vulnerabilities such as the use of unsafe C/C++ code libraries in OPC dynamic link libraries or exploitable stack overflows in OPC servers, also refers to three interesting white papers dedicated to OPC and its security considerations: 1) Understanding OPC and how it is deployed7. 2) OPC exposed8. 3) Hardening guidelines for OPC hosts9. Among the key findings and conclusions of the three white papers are: OPC is not just used for data management purposes on the shop floor but is a critical component of many production systems. More than 25 per cent of users surveyed said that loss of OPC communications would result in a loss of production. Approximately 20 per cent of users reported deploying OPC over site business networks, enterprise networks or corporate intranets and approximately 10 per cent use OPC over the internet. Attacking OPC deployments does not require special skills or esoteric process controls knowledge. All the tools and information needed to carry out attacks can be downloaded from the internet. Excessively open firewalls and overly permissive access rights lie at the heart of many scenarios. The white papers also make the point that implementing improved security can sometimes cause a malfunction of the OPC applications themselves.

Other technologies
FDT (field device tool) technology for configuration and access of field devices has addressed a number of cross platform/network maintenance issues, making it easier to remotely maintain networks and network devices. In doing so it has created new security issues as it partially defeats the intrinsic network security by over-riding parts of the network operating system. Ethernet* and TCP/IP* are other examples of technologies with great benefit to all concerned with business and manufacturing of any description. But are we considering all of the possibilities that they bring? The use of TCP/IP (transmission control protocol/internet protocol) running on standard Ethernet in many manufacturing systems makes it relatively easy for hackers to sniff (ie, detect and decode) the traffic passing on the network because this is the basic coding protocol used for all internet communication. There may be particular vulnerability to sniffing on systems where wireless communication is used between devices and the controlling PC. Are there any counter measures? Some Ethernet-based networks use dedicated ASIC (application specific integrated circuit) chips and/or fibre optics which make it much more difficult for network traffic to be accessed and understood by third parties outside of the control system architecture. Alternatively, secured and authenticated communication packets and fixed IP addressing can also be used. Where open Ethernet standards are used the appropriate security should be applied. However, this brings the risk of added operational overheads and the problems if integrating complex and sometimes incompatible systems. There is also the headache of constantly ensuring the protective shield is up to date. This is not a good recipe for a manufacturer which just wants to make products to sell. So can this difficult problem be ignored as conspiracy theory gone mad? No it cannot. There are real cases where production processes at world class manufacturers have been disrupted for days due to a simple, standard virus attacking the gateway PCs - with the net result of millions of Euros being lost in production and penalties. One hundred percent protection is impossible, and protecting from internal malicious damage is stressful, but some small steps and actions should be taken to provide at least a basic level of protection and system resilience.

The role of the PC

The standard PC that is used in offices and homes worldwide is also a key component in most manufacturing and process control systems. Although in some applications the hardware is hardened and protected to ensure it is able to function reliably in harsh manufacturing environments, the technology - and in particular the software - is the same. Sometimes the fact that the system is based on PC technology or embedded PC technology is not immediately obvious because the human-machine interface may appear to be a purposedesigned piece of hardware and software - but beneath it lies the familiar PC operating system. This makes it easy to connect to the internet and/or intranet, with all the benefits and vulnerabilities discussed above. It also provides opportunities for anyone with a minimum of computer knowledge to install unauthorised software and, deliberately or accidentally, interfere with the authorised software. At a more mundane level, it also often means that operators can use the system to surf the internet or play computer games when they should be working - and sometimes in doing so to open the whole system to possible attack. In many companies the corporate IT systems are the responsibility of an IT department which is rigorous in its application of firewalls and other security devices - but the factory systems are the responsibility of a production department which is often not as alert to computer threats. Ironically, when standard PC security software is run on computers being used for manufacturing or process control it often interferes with the operation of the system - and is therefore disabled. There are solutions to these problems and each must be considered on its strengths and merits. Streamlined system architecture is one solution but the use of secure virtual machines can be a serious alternative.

Corporate systems/ERP

Corporate systems/ERP

Corporate systems/ERP

ERP PC
Factory system

ERP PC
Factory system

PC
Factory system

PC-based system Gateway operation

Alternative system 1 Parallel operation

Alternative system 2 Window view

Figure 1: Data flow in manufacturing or process control systems.

Alternative systems
Figure 1 shows a typical PC-based manufacturing or process control system and the way it integrates and communicates with corporate systems such as ERP (Enterprise Resource Planning) through factory-floor PCs. The gateway solution in the first diagram seems to be a good one, building data up in layers and pre processing before use at the ERP or MES (Manufacturing Execution Software). However, it has some significant vulnerabilities. For example, the PC can act as a bottle neck and can also provide back door access to the corporate IT systems if the corporate firewalls and other security features are not respected. It also offers a tempting and familiar environment for unauthorised use. The key factor is that if it fails, for whatever reason, manufacturing will stop because there is then an unbridgeable gap between shop floor and business systems. The figure also shows some alternative manufacturing system architectures in which data passes to/from the factory floor directly to/from the ERP and is then, if required, distributed to users PCs or sent to them in parallel. This means the PC could be treated as a sacrificial node which can easily be rebuilt or replaced without unduly disrupting the manufacturing process as that can continue, albeit in a blind environment. Access to the internet could be available to the factory system and PC users only through the corporate IT system - which might provide comfort to company directors (typically the IT director) who have personal legal responsibility for data security. Because the alternative factory-floor system is not based on PCs but on PLCs (programmable logic controllers) it does not have the same security vulnerabilities. It would seem logical that the two highreliability environments of shop floor controller and business systems should interact directly. The objective is not to remove PCs and PC technology from the manufacturing environment, they are a necessary part of increasing productivity, but to be aware that there are other methods, solutions and techniques which offer the same or similar benefits with less risk. This issue of risk management is really at the heart of security. Of course security can be thrown at any situation until it becomes 100 per cent protected but at what cost? What is the point of doing this if it overwhelms the original objective (to manufacture products) and makes that objective unachievable?

What makes a PLC more secure?

At the heart of a PLC is a bit processing unit (BPU) which is an ASIC with a great deal of the operating system hard wired into the silicon chip. This makes it hard for a hacker (even if he/she can gain access to the system) to change its operation. Control of the central processing unit (CPU) of a PC, by contrast, is much more software-based and therefore more vulnerable to attack. PC-based systems are also, of course, susceptible to computer viruses and similar threats to which a PLC-based system is basically immune. Users have access to PLC-based systems only through purpose-designed interfaces and therefore cannot gain access to underlying operating systems, etc, as is possible with PCbased systems. Moreover, because PLC-based systems are single purpose, there are simply fewer opportunities for security vulnerabilities to arise. PCs, by contrast, are designed as multi-purpose systems and must therefore be far more complex - and the greater the complexity, the greater the opportunity for vulnerabilities. Of course, in both cases corrupted or manipulated data can cause system malfunction but this can be overcome by implementing some types of code review and data integrity checks.

Multicore technology
Recent developments10 in the design of secure operating systems, combined with the greater use of multicore technology in PC CPUs, has created the opportunity to create several secure virtual machines within a single PC. From a security perspective the issues have revolved around the separation of the different system functions. Designers have sought to ensure that one application cannot interfere with another application in the environment - either accidentally or as the result of deliberate malfeasance. For example, an internet browser running at the same time as a manufacturing control system should not permit external intrusion into the system. This is an exciting step forward in secure PC environments but is still in its infancy and has yet to be widely adopted. By adopting a combination of this type of secure PC and the suggested alternative system architectures, manufacturers can start to really reap the benefits of all that technology has to offer but with the assurance that they are not taking too many undue risks.

10

Added benefits
As well as providing enhanced security PLC-MES (manufacturing execution systems) interfaces offer a number of other advantages. These are explored in detail in other white papers produced by Mitsubishi Electric Europe B.V. Lower cost of ownership. PLCs without moving parts such as hard discs - are more reliable and less costly to maintain than PCs. Reduced exposure to risk as the result of software changes - eg, the introduction of a new PC operating system, installed on new PCs, that proves incompatible with the manufacturing software that the system is supposed to run. Reduced risk of data loss. If there should be a system failure or a communication link in a typical PCbased system is broken, data will be lost. In several PLC-based systems, data can be stored and automatically forwarded when the system or link is repaired. Integral protection against data corruption - unusual or meaningless data is immediately detected and appropriate action can then be instigated. Data communication and integration which can give substantial gains in productivity and manufacturing efficiency.

Vendors
In order to help guard against current and future security threats, manufacturing companies need to work with best in class vendors which are constantly developing the technologies that provide the necessary degree of protection. No one vendor, however, can be a world leader in all aspects of the necessary technologies and companies should therefore seek to align themselves with groups of vendors with a range of skills and products. Seeking to achieve this with a disparate group of vendors brought together by the manufacturing company itself is likely to lead to unnecessary financial and organisational cost. A better alternative is to work with an established grouping of suppliers that already has formal arrangements for joint working, sharing technologies and project management.

Conclusion
There are real and present threats to computer-based manufacturing and process control systems. Senior managers in companies that are operating such systems should be taking action to mitigate the possibility of successful attacks on their assets - attacks which could have serious consequences not just for their businesses and themselves but for the communities in which they operate. All programmable systems are susceptible to determined hackers or developers of malicious code. However, PC-based systems are a more tempting target than others because they are more easily accessible both in terms of technology and also in terms of spreading malicious code. So the answer to the question "Should manufacturers be worried?" is Yes, but not unduly. The appropriate level of security should be applied to all programmable systems, whether PC based or embedded, but manufacturers should remember that their objective is to manufacture goods successfully and profitably. It is still a fact that the most malicious actions taken against manufacturing and IT systems do not come from outside the organisation but from within. The disgruntled employee is a factor that many companies overlook.

11

Mitsubishi Electric
With more than 80 years of experience in providing reliable, high-quality products to corporate clients and general consumers all over the world, Mitsubishi Electric Corporation (TSE: 6503) is a recognised world leader in the manufacture, marketing and sales of electrical and electronic equipment used in information processing and communications, space development and satellite communications, consumer electronics, industrial technology, energy, transportation and building equipment. The company operates in more than 120 countries, has annual sales of around Euro 25 billion and employs around 100,000 people worldwide. Mitsubishi Electric Europe B.V. is a wholly owned subsidiary of Mitsubishi Electric Corporation, Japan, and has a more than 30 year history of sales, service and support of automation products within the European market place. It has seven national branches, 1,500 employees and an annual turnover of around Euro 1.6 billion. Its factory automation division has more than 50 partners across the continent.

Green Hills Software

Founded in 1982, Green Hills Software, Inc. is the largest independent vendor of embedded development solutions. In 2008, the Green Hills INTEGRITY-178B RTOS was the first operating system to be certified by the US National Security Agency to EAL6+ High Robustness, the highest level of security ever achieved for any software product. The companys open architecture integrated development solutions address deeply embedded, absolute security and high-reliability applications for the military, avionics, medical, industrial and other markets that demand industry-certified solutions. Green Hills Software is headquartered in Santa Barbara, USA, with European headquarters in the United Kingdom.

The e-F@ctory Alliance

Led by Mitsubishi Electric, the e-F@ctory* Alliance11 brings together a range of companies with skills in factory automation, process and data security and software development. Recent additions to the Alliance mean that it has extended its core skills into data use beyond factory automation such as mobile communications and network databases. The aim is to open the doors to Mitsubishis advanced automation technology to a wider range of developers and systems specialists, allowing them to develop new solutions for customers with far higher levels of interconnectivity and functionality than ever before. As more IT systems are employed by manufacturing industry, in order to remain competitive, an increasing amount of shop floor information has to be integrated into wider company systems - from advanced plant control and monitoring through to integrated production, accounts and purchasing systems. Without a unified solution, which provides a transparent view of the business and manufacturing systems, the sheer volume and diversity of data required can overwhelm system users. The answer is eF@ctory*, Mitsubishi Electrics vision for manufacturing that unifies its leading technology control hardware and networks with enterprise IT systems offered by strategic partner companies. The e-F@ctory concept was born out of the expertise Mitsubishi developed inhouse through facing its own challenges as a global manufacturing enterprise. The technology provides the tools to immediately improve almost any enterprises bottom line through increased productivity, shorter lead-times and reduced waste.

12

Glossary
ASIC - Application Specific Integrated Circuit A semiconductor integrated circuit designed for a specific application such as a PLC. BPU - Bit Processing Unit The dedicated ASIC at the heart of a PLC. CPU - Central Processing Unit The part of a PC that controls its overall activity by executing instructions as directed by the software being run on the computer. CC - Link* A standard specification for the communication of real-time plant data between control devices from different manufacturers. It is one of the fastest growing network technologies and is widely used in Asia, with over 6 million nodes installed in the past 8 years, and is now becoming more common in the rest of the world. It encompasses high data bandwidths up to 1GBs and covers various application areas including safety, field devices and data networks. DTM - Device Type Manager See FDT. Embedded system A system in which a microprocessor and associated components (eg, memory) are incorporated into a product to help it accomplish difficult and complex tasks. ERP - Enterprise Resource Planning An integrated information system that serves all departments within an enterprise. It can include software for a wide range of applications including manufacturing, order entry, accounts receivable and payable, general ledger, purchasing, warehousing, transportation and human resources. Ethernet* The standard local area network (LAN) access method, used to connect computers in a company network as well as to connect a single computer to a modem for Internet access. FDT - Field Device Tool An open standard technology for configuration and access of field devices. Device manufacturers add to their individual field devices a piece of software called Device Type Manager or DTM. FDT compliant software tools so-called frame applications - can communicate via DTMs across fieldbuses with each device. IPC - Inter Process Communication The automatic exchange of data between one program and another either within the same computer or over a network. ISA - International Society for Automation* An industry leading group which creates and proposes best practice and standards such as ISA88 (or, in the case of automation based security, ISA99). MES - Manufacturing Execution Systems (or Software) Software that provides real-time access to plant activities that include equipment, labour, orders and inventory. An MES integrates the data with enterprise resource planning (ERP) systems so that management has complete control of the operation from the factory floor to the supply chain. OLE - Object Linking and Embedding* A compound document technology developed by Microsoft*. It allows an object such as a graphic, video clip or spreadsheet to be embedded into a document. Although OLE was originally Object Linking and Embedding, from version 2.0 onwards OLE's infrastructure was built on a new architecture known as COM (Component Object Model) that goes beyond compound documents to include capabilities such as OLE Automation and Network OLE. OPC - OLE for Process Control* A standard specification for the communication of real-time plant data between control devices from different manufacturers. It was designed to bridge between Windows-based software applications and process control hardware. The OPC Foundation manages and maintains the standard. Originally OPC stood for OLE for Process Control but with the widespread adoption of the standard in industrial sectors removed from its original process industry roots OPC is now a stand-alone acronym.

13

Glossary (2)
OPC UA - OPC Unified Architecture* The OPC UA standard can be implemented with software from a variety of vendors, removing the need to use a Microsoft Windows* based platform. PLC - Programmable Logic Controller A programmable microprocessor-based device used to control production and process machinery on the factory floor or in process plant as well as many other types of mechanical, electrical and electronic equipment. PROFINET/PROFIBUS* First developed in 1989, PROFIBUS is a fieldbus with nearly 30,000,000 installed nodes worldwide. PROFINET, a more powerful and broader offering than PROFIBUS, is based on an enhanced version of Ethernet. It embraces PROFIBUS - and other fieldbuses - as well as integrating with IT systems. SCADA - Supervisory Control And Data Acquisition A process control application that collects data from sensors and machines on the shop floor or in remote locations and sends it to a central computer for management and control. Stuxnet A malicious piece of code (worm) which started to circulate during the summer of 2010. It appears to target the automation systems (SCADA and process control) of a leading automation vendor. TCP/IP - Transmission Control Protocol/Internet Protocol* Internet Protocol is the method used to break data into small pieces, called packets, for transmission via the Internet. IP is used to send data (in packets) across the internet to a destination. At this point, a second protocol, called Transmission Control Protocol (TCP), reassembles the packets and completes the data transmission. IP and TCP work so closely together that they are often referred to as a single protocol (TCP/IP) although they are technically distinct. US-CERT - United States - Computer Emergency Readiness Team The operational arm of the US governments National Cyber Security Division (NCSD) at the Department of Homeland Security (DHS). US-CERT is charged with providing response support and defence against cyber attacks for the government. *Trade marks Mitsubishi Electric Europe B.V. acknowledges that product and company names mentioned in this white paper may be registered marks of other companies and organisations and respects all rights inherent in such registration. Some product names mentioned in this white paper are registered trade marks of Mitsubishi Electric or associated companies.

14

References
1. The Aurora experiment http://edition.cnn.com/2007/US/09/27/power.at.risk/index.html 2. Security warnings http://www-cgi.cnn.com/2009/TECH/04/08/grid.threat/index.html 3. SCADA vulnerability study http://stuweb.ee.mtu.edu/~ssmoily/NESEC.pdf 4. US-CERT website security warnings http://www.us-cert.gov/control_systems/ 5. US Department of Homeland Security report http://www.us-cert.gov/control_systems/pdf/DHS_Common_Vulnerabilities_R1_08-14750_Final_7-1-09.pdf 6. Cyber-attack threatens manufacturing systems worldwide - report on www.manufacturingautomation.com http://www.managingautomation.com/maonline/news/read/Cyber_attack_Threatens_Manufacturing_Software_ Systems_Worldwide_33612 Report on BBC news http://www.bbc.co.uk/news/technology-11388018 Report from Krebs on Security http://krebsonsecurity.com/2010/09/stuxnet-worm-far-more-sophisticated-than-previously-thought/ Definition on Wikipedia http://en.wikipedia.org/wiki/Stuxnet 7. OPC white paper - Understanding OPC and how it is deployed http://csrp.inl.gov/documents/OPC%20Security%20WP1.pdf 8. OPC white paper - OPC Exposed http://csrp.inl.gov/documents/OPC%20Security%20WP2.pdf 9. OPC white papers - Hardening guidelines for OPC hosts http://csrp.inl.gov/documents/OPC%20Security%20WP3.pdf 10. Recent developments in the design of secure operating systems http://www.ghs.com/articles/index.php?wp=secure_separation 11. The e-F@ctory Alliance - a description http://www.mitsubishi-automation.com/products/efactory

15

Mitsubishi Electric Europe B.V. Gothaer Str. 8, 40880 Ratingen, Germany Tel: 49 (0)2102 4 86-0 www.mitsubishi-automation.com