External Penetration Testing Methodology: A Comprehensive Guide | Virtus Cybersecurity
External Penetration Testing Methodology: A Comprehensive Guide
Chris Kolezynski
External penetration testing is a powerful tool that can help businesses of all sizes identify and address security vulnerabilities in their systems and networks. By simulating a real-world attack, this technical security assessment allows organizations to view their defenses from an attacker's perspective. Doing so helps to identify vulnerabilities that may have otherwise gone unnoticed.
In this blog post, we'll take a deep dive into the external penetration testing process and explain how these assessments can help protect your business and provide some peace of mind in today's digital world.
Table of Contents
What is External Penetration Testing?
Why is it Important?
External Penetration Testing Methodology
Planning Phase
Reconnaissance Phase
Vulnerability Assessment Phase
Exploitation Phase
Post-Exploitation Phase
Reporting Phase
Pen Testing is Cyclical by Nature
External Pen Testing Timeframe Expectations
Selecting the Right Security Provider
Conclusion
What is External Penetration Testing?
First, let's define what this type of test is. An external penetration test, also known as a network penetration test, is a simulated cyber-attack on an organization's computer systems, networks, and web applications. It occurs from the internet, outside the organization's internal network or systems, with the goal of identifying vulnerabilities that a malicious actor could exploit.
During the engagement, a team of cybersecurity experts, often called "ethical hackers," will attempt to gain unauthorized access to the organization's systems and data using various techniques. These techniques may include exploiting known vulnerabilities, guessing or cracking passwords, and social engineering tactics.
The results will be used to identify any security weaknesses within the organization's perimeter devices. In addition, they are also used to develop a prioritized plan for remediation to mitigate the client's overall cybersecurity risk.
It is important to note that there is more than one type of penetration test. Other types include:
Internal
Web Application
Firewall
Wireless
Physical
Social Engineering
Why is it Important?
One of the primary benefits of external penetration testing is that it allows organizations to identify vulnerabilities that may have gone unnoticed. Many organizations rely on automated tools and security controls to protect their systems and networks.
These automated tools and controls, however, can only identify known vulnerabilities. Furthermore, that is only if the tool successfully identifies the technology and version in use on the externally facing device(s). External penetration testing, on the other hand, simulates a real-world attack by a cybersecurity expert. This helps to identify unknown vulnerabilities that automated tools and security controls may have missed.
Another benefit of external penetration testing is that it helps organizations understand the impact of the identified vulnerabilities. By simulating an attack, the tester can determine the risk posed to the organization, such as the potential loss of sensitive data. Doing so allows the penetration tester to provide recommendations for addressing these vulnerabilities in the appropriate order.
This information is critical for organizations to make informed decisions about prioritizing their remediation efforts. For example, imagine the pen tester uncovers a Trace.axd vulnerability, which is identified as a Medium level of risk by the automated tools.
When attempts are made to exploit it, however, the tester can initiate session hijacking and administrative account creation, making it significantly more dangerous than previously assumed. The organization would want to prioritize this vulnerability despite it only being marked as a Medium risk item by an automated tool.
External Penetration Testing Methodology
Performing an external penetration test includes several key phases: planning, reconnaissance, scanning, exploitation, post-exploitation, and reporting.
Planning: The first step is to plan the test by determining the scope, objectives, and expectations of the client. The plan may include identifying the specific systems and networks that will be tested and determining the types of vulnerabilities that will be sought out. Clients usually provide a list of IP addresses but may also wish to provide additional information relative to their unique circumstances.
Reconnaissance: The next step is to gather information about the target systems. This includes both passive and active reconnaissance. It may also result in more detailed information, such as organizational structure, people, and technologies.
Vulnerability Assessment: After reconnaissance, the next step is to scan the target systems and networks to identify potential vulnerabilities. This phase typically involves using industry-standard vulnerability scanners, manual review of identified technologies and attack surfaces, and research into vulnerabilities with public exploits.
Exploitation: Once vulnerabilities are identified, the next step is to attempt to exploit them to gain access to the network(s). Techniques may include password guessing, social engineering, and exploiting known vulnerabilities with publicly available exploits.
Post-Exploitation: The penetration tester will analyze the test results. From there, attempts will be made to further impact or escalate privileges. Once completed, where ever possible, the pen tester will clean up created accounts and modified records or settings to return the production environment to its original state.
Reporting: After the test is complete, the final step is to provide a comprehensive report to the client. The report outlines what was discovered, the methods used to exploit the vulnerabilities and recommendations for addressing them.
Planning Phase
The planning phase includes identifying the scope of the test, determining the objectives, and tailoring it to the client's specific needs. This phase is crucial because it sets the foundation for the rest of the engagement. Some critical elements of the planning phase include:
Defining the Scope of the Test
Defining the scope includes determining which systems and networks will be tested and any specific devices that will be excluded. By completing this step, the tester can understand the network's security posture comprehensively while minimizing the risk of causing any disruption to the client's operations.
Identifying the Objectives
Identifying objectives involves determining the specific types of vulnerabilities that will be sought out, such as network, application, or social engineering vulnerabilities. The objectives should align with the client's security needs and priorities.
Establishing Testing Methodologies
When establishing testing methodologies, the cybersecurity team will determine the tools, techniques, and methods that will be used during the pen test. The team should also establish a schedule and plan of action, outlining the specific steps that will be taken during each phase of the engagement.
Coordinating with the Client
The team should work closely with the client throughout the planning stage. Doing so ensures that the test is tailored to the client's specific needs and confirms that they are aware of the risks associated with the test.
Complying with Legal and Regulatory Requirements
The team should ensure that the pen test complies with all relevant legal and regulatory requirements. Requirements may include obtaining the necessary consent from the client and adhering to any specific guidelines or regulations that apply to their industry.
Reconnaissance Phase
The reconnaissance phase is the next step in the external pen testing process. The cybersecurity team will gather information about the target systems during this phase. This will include publicly available information such as the organization’s:
Infrastructure;
IP addresses;
Open ports;
Operating systems; and
Applications running on the systems.
This phase is essential because it helps the tester to understand the target environment, identify potential vulnerabilities, and ensure the penetration test is effective. Some key elements of the reconnaissance phase include:
Footprinting
Footprinting involves identifying the target systems and their attack surface using passive and active reconnaissance.
Passive reconnaissance includes reviewing publicly available information such as:
Public password dumps;
Email addresses;
GitHub repositories;
Pastebin leaks;
IP address information;
Archived versions of websites; and more.
Information from this stage is commonly used to build username lists for brute-force attacks on externally facing logins. Active reconnaissance, on the other hand, involves interacting with the systems in scope. This includes identifying open ports, running services and versions, web application technologies, and more.
OS Fingerprinting
This process includes identifying the operating systems running on the devices in scope and is done during active reconnaissance. The cybersecurity team can use this information to identify potential vulnerabilities for exploitation later in the engagement.
Enumerating Defenses
In the early stages of testing, precautions are taken to identify web application firewalls and other defenses that are in place because they can affect later testing.
Network Mapping
The cybersecurity team will map the network-level attack surface. The mapping may include identifying open ports and the various protocols and services used.
Application Mapping
The cybersecurity team will crawl externally facing web applications and enumerate technologies in use by the application. For example, this may include identifying plug-ins and themes used by WordPress, an application for building and managing websites.
Social Engineering: Reconnaissance
When social engineering is included in scoping discussions, the team will gather information about the organization by interacting with its employees by phone, email, or in person. This information can include sensitive information such as usernames, passwords, and other access credentials.
Vulnerability Assessment Phase
In the vulnerability assessment phase, the pen tester uses the information gathered during reconnaissance to perform a vulnerability scan of the target systems and networks. The cybersecurity team will use automated tools to identify open ports, services, and applications.
This phase is essential because it helps the team to identify potential vulnerabilities and determine which systems or networks are most at risk. Some key elements of this phase include:
Network Vulnerability Scanning
This process uses automated tools to scan the target systems and networks for known vulnerabilities using network vulnerability scanners.
Web Application Scanning
The cybersecurity team will scan externally facing web applications for vulnerabilities using common web application vulnerability scanning tools. This process also involves testing web applications for vulnerabilities such as SQL injection, Cross-site scripting(XSS), and File inclusion vulnerabilities, among others.
Vulnerability Research
The team will research potential known vulnerabilities in the target systems and software. Numerous sources are used to research potential vulnerabilities and public exploits. This information can be used to identify potential attack vectors that can be exploited during the test.
Scanning Considerations
This phase should be conducted with care, as it can generate a large amount of network traffic. If performed incorrectly, the test may cause disruptions to the target systems or networks. The tester should use the right tools, set the proper configurations, and schedule the scans during non-business hours to minimize any potential negative impact.
It is important to note that vulnerabilities identified during this phase may result in false positives. That is why it is essential to include the human element of the pen testing engagement through the exploitation and post-exploitation phases. Doing so allows false positives to be removed from the final report so that the client can focus their remediation efforts more effectively.
Exploitation Phase
During the exploitation phase of an external penetration test, the cybersecurity team attempts to gain unauthorized access to the client’s systems and networks by exploiting identified vulnerabilities. Doing so allows them to provide the client with an accurate assessment of the overall risk, as well as more specific recommendations for addressing them.
Some critical elements of the exploitation phase include:
Exploiting Known Vulnerabilities
This process exploits vulnerabilities identified in operating systems, applications, and software on the client’s network. Pen testers use a variety of tools and publicly available exploits discovered during the vulnerability assessment stage to accomplish this objective.
Password Guessing
The cybersecurity team will attempt to guess passwords used on the systems and networks. Password guessing is performed using both default passwords that are discovered through review of user manuals, as well as simple password lists. This can include using automated tools to perform dictionary and brute-force attacks or using social engineering techniques to obtain employee passwords.
Password guessing tools are used in conjunction with potential usernames gathered during the passive reconnaissance stage of testing. Finally, account lockout checks are performed prior to password guessing.
Social Engineering: Exploitation
If included in the initial scoping discussions, social engineering can include attempts to trick employees into giving up sensitive information or providing access to the target systems and networks. This is accomplished through phishing, vishing, and pretexting attacks.
Exploitation Privilege Escalation
Once access to the network is obtained, this process involves attempting to gain elevated privileges on the client's network in order to move laterally to other devices. If successful, the team can create enough privileges to give themselves domain administrator access. Doing so helps prevent the testers account from being removed by the client’s IT team.
Network Exploitation
This involves exploiting vulnerabilities in the target network infrastructure, such as routers, switches, and wireless access points.
Bypassing Defenses
During exploitation, client defenses are often a roadblock for the cybersecurity team. The pen tester may make use of rotating proxies, the ToR network, and strategies like Origin IP. These techniques can help bypass defenses such as Cloudflare and other common IP ban strategies in order to gain a foothold in the client’s network.
Post-Exploitation Phase
During the post-exploitation phase, the pen tester will analyze the test results and evaluate the impact of the identified vulnerabilities. This can include identifying the potential loss of sensitive data, determining the potential risk to the organization, and assessing the ease or difficulty of exploiting the vulnerabilities. This information is critical for organizations to make informed decisions about prioritizing and addressing vulnerabilities.
Some key elements of the post-exploitation phase include:
Post-Exploitation Privilege Escalation
The cybersecurity team will seek to further impact or escalate the identified privileges. This can include moving laterally through the network to access other areas and data, as well as trying to gain administrative or root access to the systems.
The pen tester will use a variety of techniques, including exploiting known privilege escalation vulnerabilities such as Silver Ticket, Mimikatz, and kerberoasting attacks. They may also attempt to use custom methods and social engineering tactics to achieve these goals.
Clean-Up
Once the pen tester has completed the analysis and impact assessment, they will clean up any created accounts and modified records/settings to return the production environment to its original state. In some cases, this may not be possible. In that event, the client will be notified of any inability to clean up files.
Post-Exploitation Considerations
Post-exploitation is essential because it ensures that the engagement does not leave any residual effects that malicious actors could exploit. The pen tester will also document all actions taken during the test and provide a detailed report to the client, including recommendations for addressing the identified vulnerabilities.
It's essential to note that the post-exploitation phase should be done with the utmost care and attention to detail, as it can significantly impact the client's network and systems. The pen tester should have a clear and detailed plan to return the environment to its original state and test it before the actual execution of the engagement. Doing so will avoid unintended consequences or damage to the client.
Reporting Phase
The reporting phase is the final step in the process. During this phase, the tester will analyze the results and prepare a detailed report. The report will provide recommendations for addressing the identified vulnerabilities, as well as a prioritized action plan for remediation. Some of the elements of the post-exploitation phase include:
Data Analysis
This process involves analyzing the data collected during the external pen test, including the results of the reconnaissance, vulnerability assessment, and exploitation phases. This information is used to identify the most critical vulnerabilities, as well as understand their impact on the client's systems and networks.
Reporting
The cybersecurity team prepares a detailed report summarizing the findings of the engagement. The report will often include a description of the identified vulnerabilities, their impact, and the cybersecurity team's steps to exploit them. It should also include specific recommendations for addressing the identified vulnerabilities, including best practices and security controls that the client can implement to mitigate the risk.
Remediation
This involves working with the client to develop an action plan for addressing the identified vulnerabilities. This can include providing guidance on patching, implementing security controls to mitigate the risk, and training employees to improve security awareness.
Follow-Up
The team conducts a follow-up assessment to verify that the identified vulnerabilities have been addressed and to ensure that the client's systems and networks are secure.
Pen Testing is Cyclical By Nature
The phases of a penetration testing engagement are cyclical by nature. As the cybersecurity team identifies new vulnerabilities during the test, they may need to revisit previous steps to gather additional information, update their testing strategies, or refine their recommendations.
For example, during the exploitation phase, if the tester can gain unauthorized access to a target system, they may discover additional vulnerabilities not previously identified. In this case, the tester may need to revisit the reconnaissance and vulnerability assessment phases to gather additional information about these new vulnerabilities and to update their exploitation strategies.
Similarly, during the post-exploitation phase, if the tester identifies new vulnerabilities or areas of risk that were not addressed during the engagement, they may need to revisit previous steps. The cybersecurity team can then gather additional information or refine their recommendations.
External Pen Testing Timeframe Expectations
The time required for each phase of a typical external penetration testing engagement can vary depending on the complexity and scope of the test. On average, however, the time for each phase can be broken down as follows:
Planning: 10% - Planning includes identifying the scope of the test, determining the objectives, and tailoring the test to the client's specific needs.
Reconnaissance:10% - This includes gathering information about the target systems and networks.
Vulnerability Assessment:20% - Vulnerability Assessment involves using the information gathered during reconnaissance to check the target systems and networks for vulnerabilities.
Exploitation:25% - The cybersecurity team will exploit the vulnerabilities identified during the vulnerability assessment phase using various techniques.
Post-Exploitation: 10% - This includes analyzing the results of the test, further escalating privileges identified, and conducting a clean-up of created accounts and modified records.
Reporting: 25% - Reporting involves documenting all actions taken during the test and preparing a detailed report for the client.
It's important to note that these percentages are approximate, as the time required for each phase can vary. This variation may depend on the specific requirements of the engagement, the size of the organization's network, and the number of vulnerabilities identified.
Selecting the Right Security Provider
When selecting a security provider, a client should consider several factors to ensure that the cybersecurity team is qualified and capable of conducting a thorough and effective penetration test. Some of the factors that a client should look for when selecting a security provider include:
Certifications
The cybersecurity team should hold industry-recognized certifications, such as Certified Penetration Testing Professional (CPENT), Certified Ethical Hacker (CEH), and GIAC Penetration Tester (GPEN). These certifications demonstrate that the pen tester has a certain level of knowledge and skill.
Experience
A client should look for a team with significant penetration testing experience. They should have a proven track record of identifying and exploiting vulnerabilities in real-world environments. A tester with a wide range of experience, including working with different industries and technologies, is more likely to be able to identify and exploit a broader range of vulnerabilities.
Methodology
The cybersecurity team should follow a structured and industry-accepted methodology, such as the OWASP Testing Guide or the Penetration Testing Execution Standard (PTES). This will ensure that the test is thorough and that the team covers all of the critical elements of a penetration test.
Communication and Reporting Skills
Strong communication and reporting skills are essential when selecting a security provider. A comprehensive report and clear communication are essential for the client to understand and address vulnerabilities during the engagement.
Compliance
A client should look for a team with experience and knowledge in the compliance frameworks that the client's industry is subject to, such as ISO 27001, PCI-DSS, HIPAA, and SOC2.
Professionalism
Finally, the team should show high professionalism and integrity by committing to conducting the test ethically and responsibly.
Conclusion
In conclusion, external penetration testing is an integral part of a successful cybersecurity program. It helps businesses of all sizes identify and address vulnerabilities in their systems and networks. By simulating a real-world attack, organizations gain insight into their defenses from an attacker's perspective. This allows them to identify vulnerabilities that may have gone unnoticed.
In addition, it's important to mention that a professional and experienced cybersecurity team should perform these engagements to conduct the test in compliance with relevant laws and regulations. It's also important to remember that this is just one aspect of a comprehensive security strategy. It should be combined with other security measures such as internal penetration testing, vulnerability management, and incident response planning.
Overall, external penetration testing is essential in protecting your business from cyber threats. It allows you to identify and address vulnerabilities before hackers can exploit them. It also provides you with the necessary information to make informed decisions about prioritizing and addressing vulnerabilities. By conducting regular external penetration testing, you can ensure that your business is protected and that your sensitive data is secure.
Chris Kolezynski
Author
Chris is a Senior Cybersecurity Engineer and Licensed Attorney in the State of Ohio. He has passed the written and practical Certified Ethical Hacker (CEH) exams, Certified Penetration Testing Professional (CPENT) exam, and is published in the Journal of Law and Cyberwarfare.
