This post was authored by Paul Brooks, business development manager, and Valerie Wilkins Tur, technical lead for networking and security services, at Rockwell Automation.
Initiatives like smart manufacturing require the free flow of information across a industrial network architecture—from the point where data is first collected, to where that data is analyzed and contextualized into information, and finally to where that information is presented to workers.
Ensuring this free flow of information, however, is no easy task. Every configured device added to a industrial network acts as a barrier to getting information to where it needs to be. And information increasingly needs to be sent across not only one network but multiple networks.
Historically, two types of devices have been used to manage the flow of industrial network traffic: switches and routers. Switches operate at layer 2 of the open systems interconnection (OSI) model. They primarily interact with information packets using MAC addresses. Routers operate at layer 3. They use IP addresses and subnets to move information from one network to another. But as the boundaries of layer-2 switches and layer-3 routers began to blur, a new solution emerged: the layer-3 switch. With the combined functions of a switch and a router in one device, the layer-3 switch allows end users to logically segment their traffic into virtual local area networks (VLANs). The layer-3 switch not only can operate one or multiple VLANs on layer 2, but it also can route data between those VLANs across layer 3.
This is increasingly important for the process industry, where distributed control systems continue to become more virtualized, with centralized servers, distributed clients, and business logic abstracted from presentation. The routing capabilities that a layer-3 switch can deliver are essential to ensuring security, isolation, and resiliency in plantwide networking.
Types of routing
Inter-VLAN routing can be configured in three different ways: connected, static, and dynamic.
Connected routing
Connected routing involves two VLANs automatically routing traffic between each other using local routes. It can only occur when both VLANs are connected to a layer-3 switch that is configured as the gateway address for both of them. The switch’s configuration looks something like this: L3switch(config)#ip routing.
Think of connected routing as two adjacent hotel rooms with a connecting door between them. The two rooms are separate, and the connecting door can be locked, but ultimately, someone can move from one room to the other via the connecting doorway.
Connected routing is particularly useful if a machine’s I/O adapters always have the same IP addresses across many machines, but the controller has an IP address that is allocated to the production line. In this scenario, the end user can use connected routing to route traffic from the line-level network to any I/O modules on the machine-level network that are connected to the same switch.
Static routing
The static routing approach is commonly preferred in small industrial networks that have only a very limited number of layer-3 switches with IP routing enabled. It involves manually configuring the exact path that packets will follow as they travel through the network.
A sample command-line configuration is:
L3switch(config)#ip route 10.10.240.0 255.255. 255.0 10.10.100.1. In this instance, packets that arrive at the layer-3 switch with the destination address of the 10.10.240.0 255.255.255.0 network would be sent to an adjacent layer-3 switch with the address of 10.10.100.1.
Static routing is similar to planning a commute. There could be several different route options, but drivers will most likely pick the one that avoids impediments like heavy traffic, construction, or frequent stops, so they can reach their destination as quickly as possible.
Static routes are simple to implement, but they are not scalable. Also, the routes must be updated any time the industrial network changes or if more network devices are added.
Dynamic routing
In large networks, manually configuring not only every immediate route, but also all the possible and allowable routes is simply too much work. More than that, it is enormously difficult to manage and maintain in the long term. This is where dynamic routing is used. It automates the process of selecting the paths that data will follow through the networks.
There are two recommended dynamic routing protocols. The first is Open Shortest Path First (OSPF), which operates on the basis that all routers and switches within the same area have an identical map of the industrial network topology. The second is the Enhanced Interior Gateway Routing Protocol (EIGRP), which only shares routing information with immediate neighbors, making it less memory intensive.
When choosing between these two dynamic routing protocols, there is almost no difference in their implementation and outcome. However, information technology (IT) departments often prefer EIGRP when integrating multiple plants into an enterprise-level network, because it offers more efficient route storage.
An example of an EIGRP configuration is:
L3switch (config)#router eigrp 100
L3switch(config-router)#network 10.10.100.0 0.0.0.3
L3switch(config-router)#network 10.10.210.0 0.0.0.255
L3switch(config-router)#network 10.10.220.0 0.0.0.255
L3switch(config-router)#network 10.10.230.0 0.0.0.255
Here, EIGRP is enabled by “router eigrp 100.” The arbitrary “100” is known as the autonomous-system number. It must be consistent across the layer-3 switches and routers that are considered to be in the same autonomous system. The “network” statements are the industrial networks that are advertised to adjacent layer-3 switches or routers, within the same autonomous system. Following the 10.10.230.0, for example, is the “wild card” mask of 0.0.0.255, which is the inverse of the subnet mask of 255.255.255.0.
Other considerations
When evaluating routing options, it is important to remember that routing does not need to be an either-or decision. Any two or even all three routing approaches can be used in a single, well-designed system. Most commonly, connected routing is used within the cell/area zone. At the site level, static routing is used between automation devices, and dynamic routing is used through the software infrastructure to support the servers, clients, and manufacturing execution software. Another thing to keep in mind is how routing can help create more efficient networks. One of the most common mistakes organizations make, for example, is trying to implement an entire control system in a single, flat layer-2 network. This can lead to several hundreds or even thousands of devices existing on a single network, creating network sprawl.
A control system that contains more than 200 Ethernet devices should be segmented into multiple VLANs. Each VLAN should be limited to a maximum of 253 IP addresses and 200 Ethernet devices. Routing should not only be considered from control systems to software systems, but also from control systems to control systems.
Additional resources and training
More information on routing approaches and considerations is available in the free design and implementation guide, Migrating Legacy IACS Networks to a Converged Plantwide Ethernet Architecture. The document, jointly developed by Rockwell Automation and Cisco, covers requirements and solutions for migrating a traditional industrial network architecture to standard Ethernet and IP network technologies. IT and operations personnel also can utilize industry training to learn more about routing in industrial networks.
About the Author
Paul Brooks, business development manager at Rockwell Automation, is a customer-focused business development professional and expert in automation and control technology and markets. He has a proven track record in business management, business partnering, technology management and strategic planning.
