Chapter 12: Introducing NSX

12.1 Overview

Perhaps one of the most inportant pieces in VMware’s strategy to build a software-defined data center (SDDC) is its network virtualization tool NSX. It all started in 2013, when VMware acquired a company called Nicira and since then quite a lot of work has been undertaken to make NSX one of the most interesting and important products in VMware’s product portfolia.

In short, the story behind the product is somehow similar to the one of vRealize Automation. The Nicira team tried to tackle the problem they had seen in large organizations – manually provisioning networks had become a bottleneck. So, similar to vRealize Automation, the initial use case was automation. However, the product evolved in the last years and two other main uses were added: security and application continuity.

There is support for integrating NSX with vRealize Automation from version 6 onwards, and for many companies, integrating NSX into vRealize Automation is a crucial part of setting up their cloud management platform. This is reason enough to discuss NSX. In the remainder of this chapter, we will first introduce NSX. Later on, we will show how to integrate NSX into vRealize Automation and last but not least, we will show how to use NSX components in blueprints.

12.2 Why use NSX?

As discussed, NSX provides a set of features, which can be summarized as follows:

  • The first use case for developing NSX is network automation. There are increasing business needs to not only deploy simple virtual machines, but to deploy services including network and security services in lockstep with virtualized applications. vRealize Automation, in conjunction with NSX, allows the provisioning of such services while guaranteeing connectivity, security and performance requirements.
  • The most well-known use case is – of course – security and the ability to provide micro-segmentation between components in the data center. This helps administrators not only to control the north-south traffic in and out of the datacenter, but also to focus on the traffic between the applications without having to rely on the perimeter security.
  • Recently, VMware has also begun to push another use case: application continuity, which can be understood as the ability to have an application consuming resources across multiple data centers. VMware tries to push this feature based on the NSX overlay technology for networks. In practice this could mean to run NSX in Amazon Web Services (AWS) and to create a link to hybrid clouds in the near future.

While the implementation of the first two use cases has recently become quite popular with more and more enterprises, application continuity is something for the future. Hence, we will now focus on network automation and security and provide some additional details of that topic.

12.2.1 Network automation

Using NSX network automation along with vRealize Automation means being able to deliver secure and scalable applications on-demand. Instead of just provisioning simple virtual machines and assigning IP addresses, DNS entries and VLANs to virtual machines, companies now seek to automatically deploy whole networks and applications on demand.

As an example, let’s imagine a company that repeatedly wants to provision the same multi-tiered applications to its customers (see Fig 1). The deployment consists of three separated subnets that are connected via a logical switch. In order to provide a high-level of security, firewall rules are enforced on a subnet level and on a virtual machine level. There is a load balancer to be provisioned that distributes web traffic across the web servers in the web subnet. In order to obtain internet connectivity, there is a NSX edge deployed as part of the application stack. Customers can connect to their application environment via VPN.

With NSX and vRealize Automation, all these processes can be automated. During the application lifecycle, vRealize Automation will provision, update and decommission the network and security services in lockstep with the virtualized applications.


Fig 1: Dynamically creating multi-tiered applications

Having both vRealize Automation and NSX available allows to create a standardized repeatable process that helps to accelerate delivery and to reduce the time needed to provision a deployment. In addition, automation greatly improves the consistency and the reliability of the final configuration by eliminating manual errors. Fig 2 shows the combination of both tools which can make the provisioning of complex applications a matter of just a few minutes.


Fig 2: Reducing delivery time with NSX and vRealize Automation

12.2.2 NSX and security

The most well-known use case for NSX is – without doubt – security and micro-segmentation. Micro-segmentation is the ability to filter network traffic at a virtual machine level. This allows organizations to logically divide the data center into distinct security segments down to the individual workload level.

Hence, micro-segmentation can greatly limit the lateral movement within the data center. In traditional data centers, once an attacker has successfully infiltrated the network, it is quite easy for him to move laterally within the data center from workload to workload with only few controls to block him. With NSX however, in case one of many web servers is compromised, network security can prevent an intruder from accessing web servers in the same network.

Recent studies have shown that traffic inside the data center accounts for around 80% of all network traffic. Hence, this is another reason to not only focus on the north-south traffic in and out of the data center, but also on the east-west traffic inside the data center. However, monitoring such a traffic is unequally more difficult as this traffic usually does not pass the security devices at the data center perimeter. However, when monitoring such traffic by using techniques such as hair pinning, network performance issues can arise due to the high amount of traffic. To tackle that problem, NSX monitors traffic at the hypervisor level – near the virtual machine. Consequently, with network virtualization, security is built in the network itself.

In order to limit the scope of network traffic, isolation and segmentation are accepted approaches as well. Isolation as an important principle in network security is traditionally achieved by routing, VLANs, access control lists or firewalls. When using NSX, we abstract from these underlying techniques and focus on higher-level constructs like security groups or logical firewalls. NSX enables organizations to segment data center networks based on logical instead of physical boundaries (applications and compliance scopes instead of physical network location) and to automate the alignment of controls and policies to those logical boundaries. All of this dramatically reduces the cost and complexity of compliance.

Using NSX has also operational impacts. Policy definition becomes an upfront activity that is performed by security architects. Once this has been accomplished, security operators can leverage micro-segmentation for controlling network traffic.

12.2.3 Application continuity

Since recently, VMware tries to provide NSX not only for the internal data center but also to extend it to public cloud providers. At the VMworld 2015, VMware already presented how to extend NSX to Amazon AWS. VMware’s stance is that companies will face a cloud lock-in if they rely on the cloud native network APIs. By extending NSX to cloud providers, customers can program to NSX’s API no matter where the actual workload is running, because the underlying interfaces for consuming load balancing, security and other network services becomes consistent across multiple clouds.

12.3 NSX background

VMware has a long-standing history in providing network functionalities. In its vSphere product, a standard vSwitch offers layer-2 network capabilities including support for VLAN segmentation (802.1Q tagging), NIC teaming, security policies or traffic shaping for outgoing network traffic. Having a distributed switch available offers even more far-reaching capabilities like the centralized management of vSwitches, IPFIX/NetFlow support, RSPAN and ERSPAN protocols for remote network analysis, quality-of-service tagging, network I/O control and an API for programming.

However, vSphere only provides layer-2 switch functionality, higher services such as routing are not part of the product.

In 2010, VMware began to offer layer 3 to 7 functionality as well and published a product named vShield. Later on, in 2013, this product was rebranded in “vCloud Networking and Security” (vCNS). Unfortunately, this very promising product did not gain broad acceptance in the market due to the fact that VMware was not recognized as a strong player in the data center networking market.

In order to change this, VMware acquired software-defined networking startup company Nicira and created NSX by combining the features of vCNS and Nicira.

12.3.1 Architecture of NSX

The functionality of traditional network components such as a layer 3 switch can roughly be divided into three different layers:

  • Administrators are using a cable, SSH or the API to connect to the management plane and configure the device itself.
  • The control plane makes decisions about where the traffic is sent. Control plane packets are destined to or locally originated by the router itself. Topology information is exchanged with other routers and routing tables are constructed based on a routing protocol such as Spanning Tree Protocol (STP), Border Gateway Protocol (BGP) or Open Shortest Path First (OSPF).
  • The data plane, also known as the forwarding plane, is responsible for forwarding the traffic to the next hop along the path to the selected destination network.

Basically NSX also comes with these layers, but with the big difference that there is a central management for these layers.

For the management layer, a dedicated NSX manager (that’s a virtual appliance connected with the vCenter server) is provided. There is a public REST API which is consumed by the vCenter Server, with tools such as vRealize Automation (with the help of an Orchestrator plug-in), VMware Integrated Open-Stack or vCloud Director. VMware used this API to integrate NSX into the VMware web client, so no dedicated tool is used to manage an NSX environment. Fig 3 shows the configuration of NSX edges in the VMware web client.


Fig 3: Using the VMware web client for managing NSX

The main component within the control plane is the NSX controller. The NSX controller is a software cluster consisting of three virtual machines responsible of centrally managing the logical network based on the overlay network VXLAN. In addition, the NSX controller coordinates the logical router. If an administrator deploys a logical router, NSX provides a control virtual machine as well that takes care of exchanging routes based on routing protocols like OSPF and BGP. Once there is a new route to be announced, the information is first passed to the NSX controller and then to a user agent running on the ESXi nodes.

The data plane is comprised of the ESXi nodes itself. The nodes run a distributed switch with additional kernel modules for VXLAN, routing and a kernel-based firewall. This implementation assures an optimal data path for the network traffic within the data center. For outbound communication, on the other side, no VXLAN or kernel-based routers are used – instead, VMware’s approach is to deploy an edge that connects the virtual with the physical world.

Last but not least, NSX must run on top of a physical network. In this regard, NSX is quite flexible and can basically run on top of an arbitrary network environment. There is only one requirement: The maximum transfer unit (MTU) must be increased by 50 bytes due to the overhead of the VXLAN protocol. However, in practice many customers tend to use a stable and powerful network infrastructure in order to use all the potential of NSX. Fig 4 shows the NSX architecture with its different layers (physical network, data plane, control plane and management plane).


Fig 4: NSX architecture

12.3.2 NSX switching functionality

One of the key features of NSX is the ability to provision logical networks. Those networks are called logical because no further configuration within the underlying physical network is needed. Logical networks can be created, used and decommissioned without configuring any VLANs. This is due to the fact that all logical networks are getting encapsulated within a physical network using the VXLAN technology. Hence, logic networks help to provide multi-tenancy within the cloud.

From a technical point of view, once an administrator creates a logical network, NSX will create a port group on all the distributed switches participating within the VXLAN transport zone. If there is any network traffic coming from virtual machines on the same ESXi server, the traffic never leaves the ESXi host itself, instead, communication is taking place within the memory of the ESXi host. In the case of virtual machines residing on different ESXi hosts, a VXLAN tunnel end point (vmkernel port) is used. However, at the end it doesn’t matter in which physical layer-2 network the source or the destination host is located, as VXLAN traffic is encapsulated in UDP and IP packets.

Using logical switches, administrators can create broadcast domains in order to create a context for VMs. Hence, control is possible if virtual machines can talk to each other. As an example, consider a scenario of a typical web application consisting of three different tiers (web tier, application tier and database tier) as depicted in Fig 5. The virtual machines in each tier can talk to each other, but communication between the tiers is not permitted.


Fig 5: A web application consisting of several logical networks

12.3.3 NSX routing functionality

Setting up routing functionality is slightly more difficult. Administrators can choose between two different components:

  • Distributed Logical Routers (DLR) are used if routing is only needed between logical VXLAN networks.
  • If any VLAN traffic is to be routed, Edge VMs are more suitable. In contrast to DLRs, edges are deployed as virtual machines (hence, they have a limit of ten virtual machines).


Fig 6: Distributed Logical Router

As a summary, Fig 6 shows how different logical networks can be connected via a distributed logical router.

12.3.4 Security and Firewall functionality

Providing sophisticated security and firewall functionality is one of the strengths of NSX, so let’s discuss the most important components:

  • Once the kernel modules are installed on an ESXi host, the host will run a distributed firewall within the kernel. Distributed firewalls act between the virtual machine and the virtual switch and enforce firewall rules that are centrally configured by means of the NSX manager. Firewall rules can be applied on different levels: MAC addresses, IP addresses, TCP and UDP ports, virtual machines, logical switches or even clusters.
  • Security groups allow you to group a collection of objects in a vSphere inventory that are based on a policy driven tagging mechanism (so it is a kind of labelling mechanism). vRealize Automation can use security groups on blueprints to assign a membership of deployed machines.
  • Security policies represent re-usable rule sets that can be applied to security groups. Rule sets can be categorized into endpoint services (guest-based services such as antivirus solutions), firewall rules or network introspection services (network services such as IDS, IDP, and encryption). As an example, take our scenario depicted in Fig 6: You can create a security group named “DB Access” and configure that the security policy only allows communication on port 3306 for MySQL.
  • SNAT and DNAT Edges: SNAT or Source NAT allows us to change the source IP address of a network packet to another IP address. Using SNAT, we can improve security by masking the IP of the requestor. DNAT or Destination NAT on the other hand is used to translate the IP address of a network packet to another IP address. Using DNAT, we can hide the internal IP address of the application and substitute it with another IP address.
  • VPN: NSX provides site-to-site VPN based on IPsec, SSL VPN and L2-VPN.
  • Load balancer. Although not being a security product it should nevertheless be mentioned here – there are load balancers available, too (technically they are based on the HAproxy open-source project).

12.4 Use cases for NSX integration

After having described the basics of NSX, we want to discuss the use cases for a possible integration of NSX into vRealize Automation.

  • The first use case is certainly the dynamic provisioning of networks. In the previous chapters, we have already introduced network profiles. When combined with NSX, network profiles are much more powerful and provide an easy means to create ad-hoc networks. Under the hood, vRealize Automation creates and manages logical switches and attaches them to Distributed Logical Routers.
  • In addition, we can use security groups to define security tiers. Technically, once a NSX security group has been created and configured with firewall rules, we can reference the corresponding security group from the blueprint designer. Administrators use this approach for providing micro-segmentation. For example, it would be possible to create security groups for each tier (e.g. web, app, DB) within a stack.
  • If you want your virtual machines within a deployment talk to each other, but simultaneously block all external communication other than the designated incoming traffic type, you can use the App Isolation feature on a blueprint.
  • You can also use fine-grained security groups not only on tiers, but also on virtual machines. Imagine you want to deploy a bastion host within your network and define rules that this machine is only accessible on port 22 via SSH, then it is a good idea to apply security groups on an individual virtual machine level.
  • The aforementioned use cases are out-of-the-box features of the vRealize Automation NSX integration. If these features are not sufficient for you or you are missing some (for example, VPN configurations, advanced load balancer settings, L2 bridging, etc.), you are still totally free to use Orchestrator and do some programming with the NSX Orchestrator plug-in.

12.5 Summary

This chapter introduced NSX and showed the basic use cases for a NSX integration into vRealize Automation. After having explained all the basics, we will show how to setup NSX and vRealize Automation and how to implement these use cases in the next chapter.