Skip to content
Randy Lindberg 10 Mar 2021 7 min read

How to Test Your Firewall Security & Rules

According to recent security statistics, cybercrime is more diverse and prevalent today than ever before. A digitized business world means more opportunities for organizations to thrive by serving and reaching the masses, but it also results in more vulnerabilities and a need for comprehensive policies and processes to ensure digital safety and security against the unscrupulous.

Firewalls are one of the first lines of defense against cyber attacks. Firewalls are also a solvent approach because threat and penetration testers can easily run simulated threats to test their network’s security. In this post, we’ll cover how to test firewall security including all the tools, methods, and steps you'll take to complete a comprehensive firewall penetration test.


Firewalls 101

Firewalls have a gigantic security job in protecting sensitive and valuable data. Traffic on the external network/internet gets inspected by firewall software as it comes in and out. Software features preset rules, policies, and an access control list that it uses to filter and restrict any connection that’s not in accordance with its standards. Basically, it separates the trusted networks from the questionable ones.


In most cases, the main firewall is placed in the demilitarized zone, or DMZ. Some select additional firewalls closer to the business’s intranet and/or their industrial supervisory control and data acquisition (SCADA) may also exist.


Next Gen Firewalls Explained

The main flaw within traditional firewall models is that they can’t take part in stateful packet inspection. Instead, they go about merely analyzing the network’s current traffic via IP addresses and the packet’s port numbers, which means zero consideration to the earlier traffic that passed through it.


Next Gen Firewalls, or NGFW, represents progress. With them, all active connections are monitored alongside the state of the connections, and dynamic packet filtering takes place. The result is more comprehensive access determination.


Firewall Guidelines

All firewalls require the owning entity to configure a set of policies and rules to be used to secure the network perimeter, which basically regulates network traffic (i.e. what can flow through and what is blocked). Rules and policies can be further applied to various supplemental firewalls throughout the network as well. User roles and permissions can be interpreted with an active directory and role-specific access controls.


Tools For Firewall Penetration Testing


Before we explain how to test firewall security/how to test firewall rules, let’s consider the tools you’ll need for testing.


The main tool you’ll need for testing firewall security will be scanners. These enable you to gather firewall responses by sending it customized packets. The responses can be analyzed to determine critical points, port states, versions and services being run, and system vulnerabilities.


The scanners most commonly used are Nmap, Hping and Hping2, Netcat, and Firewalk.


If needed, you can also use Fpipeand Datapipe tools for tasks like port redirect, and you may use HTTPort tool for HTTP tunneling.


How to Test Your Firewall Security in 12 Steps

1. Firewall Location

The initial step is to locate the firewall you want to test. You’ll use your choice of packet crafting software to create specific IP packets with TCP, UDP, or ICMP payloads.


Most people use Hping or Nmap as their pen-testing tool. Do keep in mind that both tools function about the same aside from Nmap being able to scan a range of IP addresses and Hping only being able to scan a single IP address at a time. So, if you want a more aggressive scan, Hping may be the better option. Using Hping better guards against abnormal activity being detected.


You’ll repeat the scanning process to map the allowed services list of the firewall.


2. Traceroute

You’ve located the firewall. Now, you’ll run a tracert command against the firewall to identify network range and gather system-to-system routing info on packets. This will tell you the devices and routers involved in establishing a connection, info concerning traffic filters, and what protocols being used.


3. Port Scanning

Due to the breadth of customization scans possible, Nmap is the preferred tool for port scanning. Your scan can be designed to a specific type, time, and so forth, and it will deliver results in different formats. The goal is to use these various Nmap scans to determine open ports and the corresponding services those ports run.


4. Banner Grabbing

This step tells you what version of firewall is being used, which you’ll use later to locate potential compromising exploits within the firewall. Most people use Netcat to create the connection request, and custom-made packets used to scan the firewall will elicit different responses that can be used to determine what specific type of firewall you’re attempting to bypass.


Use Nmap or Hping to attempt a plethora of variations of the scan, including different flags, protocols, and connection attributes, so that you can gather as much info as possible from the firewall’s responses.


5. Access Control Enumeration

A firewall’s access control list exists to determine allowable and denied traffic to the internal network. Your only indicator at this point is the state of ports on the firewall as you enumerate the access control list, which is command Nmap -sA x.x.x.x.


The first 1024 ports with the ACK flag raised get Nmap packets, which will return port status results as follows:


  • Open ports are in listening mode


  • Blocked ports are in a filtered state


  • Closed, but passable, ports are in an unfiltered state


6. Firewall Architecture

Identified firewall ports are sent crafted packets, which will offer you a port status listing. You’ll use Hping, Nmap, or Hping2 to gather responses on targeted ports to identify the firewall’s reaction and further map open ports.


After the scan, the firewall sends back action packets on what connections were rejected, blocked, or dropped as follows:


  • A firewall SYN/ACK packet return tells you the port is in an open state


  • A firewall RAT/ACK packet return tells you the firewall rejected the crafted packet


  • A firewall return of ICMP type 3 code 13 packet tells you the connection was blocked


  • No response tells you the crafted packet was dropped from a filtered port


7. Firewall Policy Testing

You have two options here - identify possible gaps by comparing hard copies taken from the firewall policy configuration and the expected configuration, or take direct action on the firewall to confirm the expected configuration.


8. Firewalking

In this step, you’ll map the network devices sitting behind the firewall using a firewalk network auditing tool. This utilizes the traceroute technique to analyze firewall-returned packets. The devices behind the wall are critical to determining A) open ports and B) what traffic gets in said ports. This is basically advanced network mapping that enables you to visualize the network’s topography.


Using your crafted packets with specific TTL values, you’ll then analyze your return packages as follows:


  • Exceeded TTL messages mean open ports.


  • No responses mean the packet was filtered and connection blocked.


9. Port Redirection

Use this testing process to see if inaccessible ports can be indirectly accessed after denial of access. Port redirection tools allow you to bypass the firewall in compromised systems. You simply listen to, or sniff, certain port numbers. Traffic is then redirected to the compromised machine.


10. External And Internal Testing

Externally, you’ll research and attempt to exploit actions that an outsider without the appropriate access and permissions could use to gain access to your system.


Internally, the penetration is much akin to a vulnerability assessment in that it’s an identification tool, but it adds the more tangible caveat of actually exploiting the identified vulnerabilities and determining what info can actually be exposed.


This step is preferred, but it’s not always required. It’s used to gather a more realistic picture of how a cybercriminal may initiate an attack and what’s at stake if they succeed.


If you take this step, you’ll need to analyze the packets received inside the network after sending packets from outside the network.


11. Covert Channel Testing

This type of channel is used by hackers for activities that ultimately enable them to communicate with a system and extract sensitive info from a company unbeknownst to anyone. Often created using a backdoor on a compromised machine, a hacker can then use a reverse shell to establish a connection with an external machine. 


12. HTTP Tunneling

Using an HTTPort tool, you’ll deliver POST requests with a hostname, port number, and path to the server. HTTP proxies can be bypassed, which leaves proxy’s enabled connect methods as the only obstacle to bypass. If the “CONNECT” method is disabled, it’s doable, albeit very difficult, via a remote host. However, an HTTP tunnel is easy to create if the “CONNECT HTTP” method is enabled.


Identifying, Documenting and Reporting

Your final steps are to identify all vulnerabilities, and document what you’ve found. In your documentation, you should include what, where, and how each testing method worked. After all, it’s not enough to know how to test firewall security - you have to then report your findings and determine the best methods of enhancing your firewalls to keep cybercriminals out.


Get in touch with Rivial Security for Penetration Testing Services today. 


Randy Lindberg

Randy Lindberg is the founder and CEO of Rivial Data Security. He has more than twenty years of experience in information security and started Rivial to fix the issues he saw as an Information Systems Security Officer in the U.S. Air Force and Information Security Manager at a $4 billion dollar financial institution.