Developments Around Cloud TAP Capability
Whether it’s for troubleshooting or for security monitoring purposes, being able to capture network packets from inside a network at strategic points is invaluable. Think, for instance, of users reporting that a website is intermittently inaccessible. Captured network packets can be analyzed and an underlying issue can be found by looking at the interaction between the client and the webserver or a router in between.
Another example is the use of an Intrusion Detection System (IDS) that “listens” to a stream of network traffic and alerts when it identifies suspicious or malicious traffic based on known signatures or traffic anomalies. That’s where a TAP comes in.
What Is a TAP?
In order to obtain those packets, they need to be intercepted. A network TAP is either a virtual or a physical device that listens to the network traffic on its interface(s) and sends a copy of these packets to another system or stores them directly to disk.
A physical TAP can be simply a little box with mirrors (duplicating the light carried by an incoming fiber lead) or it can be a powered device, sometimes with built-in logic and software. Many professional switches have the option to assign an interface as a TAP port as well (called a SPAN port).
A virtual TAP is located within a hypervisor such as VMWare ESX or VirtualBox. It works in a similar manner by connecting to a virtual traffic flow or virtual switch. A benefit of the vTAP, though, is that it can monitor traffic between two virtual machines within the same hypervisor without the need for the traffic to leave the hardware. With the virtualization of network devices such as firewalls, switches and proxy servers, this has been a popular option in the recent years.
TAPs in the Cloud
Some cloud service providers (CSPs) have come up with solutions that enable customers to tap into their network traffic. This is important, because whether a system is located in a company’s own local datacenter or hosted, for example within an AWS instance, visibility for troubleshooting and security monitoring remains important.
However, they face some challenges. Firstly, the platform environment is multi-tenanted, and this obviously has privacy and security concerns. The CSP cannot provide a customer access to the lower layer of the network infrastructure.
Another complication is the location independent nature of the public cloud. A customer’s infrastructure, including their virtual servers, can be moved around between data centers and physical systems at any given time. As long as the CSP ensures the availability and adheres to the limitations requested by the customer, such as keeping data within selected geographic areas, this is no issue. However, it does make it very complex to select a static, reliable TAP point.
Finally, cloud network traffic often uses different CSP-specific headers while the packets are in transit. The CSP removes those headers upon delivery of the traffic, but if the traffic was actually intercepted in transit, it would be hard to use in typical security devices and applications.
Where TAP configuration has been challenging for customers, creative users and researchers have come up with alternative solutions and workarounds, such as a NAT setup for AWS. Companies relying on TAP ability for their products to function have also developed new products and services; for instance, Tap-As-A-Service for Openstack.
Microsoft has offered the capture of network data via the “Azure Network Watcher.” This works with the configuration of a network watcher at a point in the network. It will capture traffic and record it to a file at a specified system or location where it can be analyzed.
This works well when troubleshooting specific network-related issues, but it does not allow for the round-the-clock real-time monitoring of traffic in order to detect suspicious activity. That requires a life feed of data, either by placing the security device such as an IDS inline within the traffic flow or by creating a 24/7 mirrored stream of the traffic and sending that stream to the security device (like a TAP).
Now, Microsoft has come back with the just-announced Virtual Network TAP. Even though this is still in the Developer Preview stage (enrollment in the preview is required) and on systems in specific regions only, this is quite a promising development. From a security perspective, having this added visibility opens a plethora of new abilities in the Azure cloud. Vendors can now develop and offer additional security products and users can build open source security controls such as SNORT IDS and a range of anomaly detection systems.
Although it might not seem that way at first glance, this is a very big step forward. Other cloud service providers will likely follow suit now.
The amount of network traffic that will be captured with this solution can be significant. Quite often, long-term storage of network traffic is unfeasible for an organization.
It is important to understand, though, that once the traffic has passed the security controls and once alerts are generated, it is a choice to store this data. There is no need other than to look up historical traffic for security incident investigations. And long-term packet capture data storage can be very expensive: A medium-sized business could easily generate hundreds of gigabytes of data each day, and hosting that within an Azure instance will come at a cost, no matter what storage type is selected.
There are alternatives as well. For instance, it is possible to tap the network traffic, direct it to an IDS, raise alerts on suspicious traffic and then to discard the traffic itself. In parallel, IPFIX and NetFlow-based logs can be stored for an extended period of time. These logs can be used for correlation between the IDS alert and the details of the traffic flow. Even though this means there will not be an option to extract the relevant packet capture data after the IDS alert, this would still provide an analyst with a lot of tools for an investigation.
For any company currently using the Azure platform, the Azure Virtual Network TAP is a development to closely follow. If there is already a clear need to capture network traffic in real-time, enrollment in the preview program might provide some helpful insights before the final official capability is released. Hopefully the rest of the public cloud market will offer similar products soon.
- Bringing More Security to OpenStack Clouds with Tap as a Service, Gigamon
- Packet Capture on AWS, Slideshare
- How We Built an Intrusion Detection System on AWS using Open Source Tools, Medium
- Manage packet captures with Azure Network Watcher using the portal, Microsoft Azure
- Virtual network TAP, Microsoft Azure