In many ways, the landscape for digital businesses has seen a revolution or two. What started as simple CGI scripts written in Perl has now bloomed into clustered deployments running fully automated on Kubernetes and other orchestration frameworks (sorry for the heavy jargon — I’m not making it up at all; it’s just the way things are these days!). But I can’t help but smile at the thought that the fundamentals are still the same as they were in the 1970s. All we have is abstractions upon abstractions supported by hard, physical cables that form the network (okay, there are virtual networks are well, but you get the idea). If we want to get fancy, we can divide the network into layers as per the OSI model, but all said and done, we’re always, always dealing with TCP/IP protocols (warning, heavy reading ahead!), pings, routers, all of which have one goal in common — transmission of data packets.
So, what is a network packet?
No matter what we’re doing — chatting, video streaming, gaming, surfing, buying stuff — it’s essentially an exchange of data packets between two computers (networks). A “packet” is the smallest unit of information flowing in a network (or between networks), and there’s a well-defined method of constructing and verifying network packets (beyond the scope of this article, but if you’re feeling adventurous, here’s more). In simpler terms, every packet represents a link in the chain, and is properly transmitted at the source and validated at the destination. Even if a single packet arrives out or order, the process is suspended until all the packets in the correct order have been received, and only then are they put together to form the data they originally represented (an image, for example). Now that we understand what a network is, it becomes to understand what a network analyzer does. It’s a tool that allows you to peek into individual packets on your network. But why would you want to go to that trouble? Let’s discuss that next.
Why do we need to analyze packets?
It looks like packets are pretty much the basic building blocks in a network data flow, much like atoms are the basis of all matter (yes, I know, these aren’t true fundamental particles, but it’s a good enough analogy for our purposes). And when it comes to analyzing materials or gases, we never bother about what an individual atom is doing; so, why worry about a single network packet on an individual level? What can we know other than we already know? It’s hard to sell the importance of packet-level analysis when you haven’t been bitten in the back before, but I’ll attempt. Packet analysis means getting your hands dirty and reaching down into the very plumbing to figure something out. Generally, you need to analyze network packets when all else has failed. Typically, this includes seemingly hopeless scenarios as follows:
Inexplicable loss of secret data despite no evident breach Diagnosing slow applications when there seems to be no evidence whatsoever Making sure your computer/network has not been compromised Proving or disproving that an attacker isn’t piggybacking off of your WiFi Figuring out why your server is the bottleneck despite low traffic
All in all, packet analysis falls under certain, hard types of evidence. If you know how to conduct packet analysis and have a snapshot, you can save yourself from wrongly being accused of a hack or simply being blamed as an incompetent developer or network administrator. Regarding a real story, I think this comment on the blog post found here is exceptional (reproduced here just in case): In case you didn’t feel like reading through that slab of text, or if it didn’t make much sense, this gentleman was facing performance issues that were being blamed on his application, and the management, as expected, was lending zero support. It was only a thorough packet analysis that proved that the problem was not in the application, but in how the operating system was handling the networking protocol! It took a while battling with account managers and developer support folks at the OS vendor who didn’t understand the problem, my explanation, or that the issue couldn’t be in the application because the application is blissfully ignorant of TCP machinations. It was like talking to a wall. I started at square one with every conference call. Eventually I got on the phone with a guy with whom I could have a good discussion. It turns out that he put the RFC1323 extensions into the stack! The next day I had a patch to the OS in my hands and the product worked perfectly from that point forward. The developer explained that there was a bug which caused incoming ACKs with payloads to be miscategorized as DUPACKs when the stack was in congestion control. This would never happen with half-duplex applications like HTTP, but the application I was supporting sent data bidirectionally on the socket at all times. I didn’t have a ton of support from management at the time (my manager even yelled at me for “always wanting to use a sniffer” to fix problems), and nobody but me was looking at the OS vendor’s TCP implementation as the source of the problem. Wrestling the fix from the OS vendor by myself made this victory particularly sweet, earned me a ton of capital to do my own thing, and led to the most interesting problems showing up on my desk. The fix was not a tune-up in the application, but a patch by the operating system developers! 😮 Boy, oh, boy . . . Without packet-level analysis, where do you think this person would be? Probably out of his job. If this doesn’t convince you of the importance of packet analysis (also called packet sniffing), I don’t know what will. 🙂 Now that you know packet analysis is a superpower, I have good news: it’s not all difficult to do so! Thanks to powerful yet simple-to-use packet sniffing tools, gleaning information from the packet-level analysis can be as easy as reading a sales dashboard. That said, you will need a little more than a surface-level knowledge of what goes on inside a network. But then, again, there’s no rocket science here, no twisted logic to master — just plain common sense. If you start reading through the documentation of one of these tools as you use them on your network, pretty soon, you’ll be an expert. 😀
Wireshark
Wireshark is an old project (it started way back in 1998) that is pretty much the industry standard when it comes to diving deep into networks. It’s impressive when you consider that it’s purely a volunteer-run organization, backed by some generous sponsors. Wireshark remains open-source (not on GitHub, but the code can be found here) and even has a tech conference to its name! Among the many capabilities of Wireshark are:
Support for hundreds of network protocols. Interoperable with many file formats (tcpdump (libpcap), Pcap NG, Catapult DCT2000, Cisco Secure IDS iplog, Microsoft Network Monitor, Network General Sniffer® (compressed and uncompressed), Sniffer® Pro, and NetXray®, and so on). Run-on virtually all platforms out there (Linux, Windows, macOS, Solaris, FreeBSD, and more). Live data reading from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, among others. On-the-fly gzip decompression. Loads of decryption protocols supported (WPA/WPA2, SNMPv3, etc.) Extensive VoIP analysis Coloring rules for faster visual scanning
Check out this fantastic online course to teach you mastering Wireshark. Looking for Wireshark in your terminal? Sure, try Termshark
tcpdump
If you’re old school (read hardcore command-line junkie), tcpdump is for you. It’s another of those iconic Linux utilities (like curl) that remains as relevant as ever, so much so that almost all other “fancier” tools build on it. Like I said before, there isn’t a graphical environment, but the tool more than makes up for it. But installing it can be a pain; while tcpdump comes bundled with most modern Linux distributions, if yours doesn’t, then you’ll end up having to build from the source. tcpdump commands are short and simple, aimed at solving a particular problem such as:
Displaying all available interfaces Capturing only one of the interfaces Saving captured packets to file Capturing only failed packets
. . . and so on. If your needs are simple and you need to run a quick scan, tcpdump can be a great option to consider (especially if you type tcpdump and find that it’s installed already!). Check out this post for some real-time tcpdump command examples.
Tproxy
Tproxy is a simple, open-source, command-line tool to proxy TCP connections over the network. Tproxy is written in the Go programming language and is a Go command-line tool, and packaged as a Go binary so it is available on all modern Linux and macOS platforms. Below image shows the command line to monitor Mysql connections: Its primary use case is to proxy HTTP connections and inspect the contents of the HTTP request, and the responses. It can also be used to proxy other protocols, like SOCKS or TCP. It can be used to proxy TCP connections between two hosts, or to analyze the traffic on a live connection. It is useful for debugging and testing applications that use TCP as a transport protocol, and for building TCP proxy services. It was written with the goal of providing an easy-to-use interface to the tcpdump command line utility, allowing users to proxy TCP connections and perform other advanced network analyses without needing to understand the underlying TCP protocol. Below command line displays gRPC connections: It acts as a TCP proxy, allowing you to connect to remote services and applications as if you were on the local machine. It allows for building custom TCP services on remote hosts, similar to a reverse TCP proxy which allows for a variety of security, troubleshooting, and performance improvements when dealing with applications that use TCP. Below are a few more usages of Tproxy
It is useful for debugging and optimizing TCP connections. It is also useful for understanding how other proxy programs work, such as Burp Suite and ZAP. It can be used to proxy TCP connections between two hosts, or between a host and a remote TCP service, like a database or web service. Monitor MySQL connections and gRPC connections and also checks the connection reliability in terms of Retrans rate and RTT
Most users will find that Tproxy simplifies their workflow and allows them to perform complex network analysis and analyze web applications without needing to alter the application or the network itself. The tool can be downloaded from GitHub.
NetworkMiner
Promoting itself as a Forensic Network Analysis Tool (FNAT), NetworkMiner is one of the best packet-level analyzers you’ll come across. It’s an open-source tool that can analyze a network passively and comes with an impressive GUI interface for analysis that can show individual images and other files transferred. But that’s not all. NetworkMiner comes with excellent other features such as:
IPv6 support Parsing of PCAP files Extract X.509 certificates from SSL encrypted traffic Pcap-over-IP Works with several types of traffic, such as FTP, TFTP, HTTP, SMB, SMB2, SMTP, POP3, etc. OS fingerprinting Geo IP localization Command-line scripting support
Do note that some of these features are available in the commercial version.
Fiddler
Unlike other passive network sniffers, Fiddler is something that sits between your device and the outside world and hence requires some setup (is that why they named it “Fiddler”? 😉 ). It’s a customizable (using FiddlerScript) free tool that has a long and distinguished history, so if your goal is to sniff HTTP/HTTPS traffic like a boss, Fiddler is the way to go. There’s a lot you can do with Fiddler, especially if you’re in the mood to don the hacker hoodie:
Session manipulation: Rip open HTTP headers and session data, modifying them in any way you want. Security testing: Lets you simulate man-in-the-middle attacks and decrypt all HTTPS traffic for you. Performance testing: Analyze page load (or API response) times and see which part of the response is the bottleneck.
In case you feel lost, the documentation is very good and is highly recommended.
WinDump
If you miss the simplicity of tcpdump and want to bring it to your Windows systems, says hello to WinDump. Once installed, it works from the command line by typing “tcpdump” the same way the utility works on Linux systems. Note that there’s nothing to install per se; WinDump is a binary that can be run straight away, provided you have a Pcap library implementation installed (npcap is recommended since winpcap is no longer in development).
BruteShark
BruteShark is an effective Network Forensic Analysis Tool (NFAT) you can use to process and inspect the network traffic such as PCAP files and capture directly from network interfaces. It includes reconstructing TCP sessions, building network maps, extracting hashes of strongly encrypted passwords, converting hashes into a Hashcat format to perform a Brute Force attack offline. The main intention behind this project is to help network administrators and security researchers responsible for analyzing network traffic and identifying weaknesses. There are two versions of BruteShark available: a Graphical User Interface (GUI) based application for Windows and a CLI tool for Linux and Windows. Some of the projects included in this solution can even be used independently to analyze network traffic on Windows and Linux machines. BruteShark can also perform various operations like extracting DNS queries, file carving, extracting VoIP calls (SIP, RTP), Building network diagram for users and network nodes, extracting authentication hashes using Hashcat (NTLM, HTTP-Digest, CRAM-MD5, Kerberos, etc.), and retrieving and encoding passwords and usernames. All the projects are implemented through .Net Standard and .Net Core for cross-platform and modern support. This solution offers a three-layered architecture and includes one or multiple projects, including PL, BLL, and DAL at each layer. Through DAL (Data Access Layer), the project helps in reading raw files of PCAP using drivers like WinPcap, libpcap, and wrapper libraries like SharpPcap. Similarly, the BLL (Business Logic Layer) is responsible for network information analysis (TCP, packet, etc.), and PL utilizes a cross-cutting project and refers to both BLL and DAL layers.
OmniPeek
For larger networks that have tons of MBs of data flowing through them every second, the tools everyone else uses may run out of steam. If you’re facing the same, OmniPeek might be worth a look. It’s a performance, analytics, and forensics tool for analyzing networks, especially when you need both low-level capabilities as well as comprehensive dashboards.
Capsa
If all you’re concerned about is the Windows platform, Capsa is also a serious contender. It comes in three versions: free, standard, and enterprise, each with different capabilities. That said, even the free version supports over 300 protocols and has interesting features like alerts (triggered when certain conditions are met). The standard offering is a notch above, supporting 1000+ protocols and allowing you to analyze conversations and reconstruct packet streams. All in all, a solid option for Windows users.
EtherApe
If powerful visualizations and open source are what you’re after, EtherApe is a great option. While prebuilt binaries are available for only a handful of Linux distros, the source is available (on both SourceForge and GitHub), so building it on your own is an option. Here’s what makes EtherApe great in my opinion:
Multi-node, color-coded monitoring. Support for a ton of packet formats such as ETH_II, 802.2, 803.3, IP, IPv6, ARP, X25L3, REVARP, ATALK, AARP, IPX, VINES, TRAIN, LOOP, VLAN, etc. (actually, many, many, more). Read data live off of the “wire” or from a tcpdump file. Supports standard name resolution As of latest versions, the GUI has been moved to GTK3, resulting in a more pleasant experience.
CommView
If you’re a Windows-exclusive shop and value the convenience of priority support, CommView is recommended. It’s a powerful network traffic analyzer with advanced features like VoIP analysis, remote tracking, etc., built-in. What impressed me most is its ability to export data into formats used by several open and proprietary formats, such as Sniffer®, EtherPeek™, AiroPeek™, Observer®, NetMon, Wireshark/Tcpdump, and Wireshark/pcapng, and even plain hex dumps.
Wifi Explorer
Last on the list is Wifi Explorer, which has a free version for Windows and a Standard version for Windows and macOS. If WiFi network analysis is all that you need (which is pretty much the standard these days), then Wifi Explorer will make life easy. It’s a beautifully designed and feature-rich tool for cutting straight to the heart of the network. Honorable mention: It would be a disservice to close this post without mentioning a macOS-exclusive network analyzer I stumbled upon — Little Snitch. It has a firewall built-in, so it comes with the added benefit of immediately letting you control all the traffic perfectly (which can appear to be a pain but is a massive gain in the long run).
What’s next?
You should also explore these network monitoring software for better infrastructure visibility. If you are looking to build a career in network and security, then check out some of the best online courses here.
