How to set the scope of penetration test engagements

TL-DR: Setting scope of penetration test engagements is tricky. I tried to put forward a sound approach and list the tips that I accumulated over the years.

Why Set a Scope?

Your company may have many web applications, servers & services exposed to the Internet. This exposure creates your company’s primary risk surface in terms of cyber security. If the exposure is limited, scoping the penetration test would be easy, but it gets more challenging as the risk surface grows.

Obviously, the best coverage and thus the most assurance would come by leaving out no systems or apps. However, a large penetration test could be expensive, could take a long time to complete and its results could be difficult to manage. Even when 100% coverage is strongly favored, scoping out some components will still make more sense, due to some systems being marked for demise or planned to be migrated to the cloud. Performing a security test on such systems will be wasteful, since they would either change or die before they can even be fixed.

Furthermore, penetration test budgets are often limited, and a sound scoping approach is needed for getting the best bang for the buck.

The Approach

Here are the steps for a simple & clear approach for determining the scope of your next penetration test:

Create an inventory
Filter out unnecessary items
Prioritize
Finalize

1. Create the inventory of your Internet-exposed systems

Ensuring a proper coverage relies on having proper visibility. Having an up-to-date inventory of Internet-facing systems helps achieve that.

It is quite common for organizations to lack an up-to-date inventory of their IT assets. While larger organizations are more likely to have initiated the effort at some point, organizational changes, unresponsive counterparties, and errors & negligence often kick in, too. For reasons of this nature, IT asset inventories are likely to become obsolete sometime after they are created.

Whether outdated or non-existent, the lack of a good IT asset inventory that provides a 360-degree view, it becomes harder to keep track of what exposures the organization could have in terms of information security risks.

The inventory allows filtering based on certain criteria and shortlisting the applications/systems for the penetration test scope. Also, an additional benefit is to aid in assigning and dispatching the reported vulnerabilities to the internal and external stakeholders.

While every company’s needs can be different, the sample table provided below can be a good starting point for building your inventory of Internet-facing systems:

Name	IPs / URLs	Type	Purpose	Business Owner	Regulation	Vendor Type	Dev. Team	Sys. Admin
		Web App, API, Other Service			GDPR, APEC PCI-DSS, HIPAA, …	In-house, Vendor, Package Software
Internet Banking App	bank.com	Web App	Transactional online banking system of retail banking LOB	Retail Banking Director John Doe	Banking Law, GDPR, PCI-DSS	In-house	IntBank Dev Team	Distributed Systems Admin Team
RecruitMe	recruitme.com	Web App	Recruitment platform for the whole bank	HR Coordinator Jane Williams	GDPR	Vendor	Jet Web Apps LLC	Distributed Systems Admin Team

2. Cross-out the systems/services/applications that will be demised, changed, or migrated within a year or less

Once the inventory is completed, reach out to the stakeholders, and ask whether it is in their plans within the calendar year (or within a reasonable time frame for the near future), to make any significant changes to the listed items. Significant changes could be:

Demise of the system,
Migration to/from the cloud, changing the platform provider,
Major hardware or software upgrades,
Changing the authentication mechanism,
New integrations with 3^rd party systems.

While reaching out, it would be good to ask if they have any specific security related concerns, as well as to ask if they have additional systems or applications that are missing out in the list.

3. Prioritize: Make a risk assessment for the remaining systems/services/applications

Obviously, the first item to add to the scope are the applications and systems related to the core functions of the organization. For a bank, these would be the transactional systems that are either in part or completely exposed to the Internet, such as Internet and mobile banking systems, online investment platforms, cheque processing applications, and credit application systems.

It is a known fact that heavily regulated industries, such as banking & finance, health and telecommunications have a lot more to lose in the face of a security breach. However, the potential cost of a data breach is exacerbated by regulations. GDPR, for example, is a far-reaching regulation that applies to any organization in any country, as long as they collect EU citizens’ personally identifiable information. As such, it may be wise to include in scope of the penetration test the applications and systems that are subjected to a regulation.

However, once the core and regulation-related applications are added, choosing what else to add is often where everyone gets a bit lost. One idea that security engineers are often inclined to, is to include the systems that have not been covered in a recent penetration test. Such shortcut thinking may not be ideal, since there could be a good reason for those systems to be out scoped from the previous tests: a static web site that offers no interaction to the end-user, that is hosted on a provider’s server, isolated from the organization’s systems should be less likely to land in the shortlist. So let’s think a bit more and try to come up with more sound ideas to filter the applications.

Companies operating in highly competitive industries, as well as those that are open to innovation often tend to have a high number of applications and services for their users. Creating a high number of applications/services is also likely to accumulate forgotten applications/services. There may be projects that failed to deliver the expected results, but the related apps/systems may not be killed off for some reason. Since such low priority items are less likely to get regular updates and/or maintenance, they can be the low-hanging fruits that attackers love to get their hands on. In the inventory, these would likely correspond to the systems that system admin teams, development teams or business owners cannot be located. The worst mistake would be to out scope a system, simply because the responsible parties cannot be located, for those are often goldmine of security vulnerabilities!

Similarly, development and test instances of web applications or other services are often a treasure-trove for attackers. Subdomain enumeration, SSL certificate examination, Shodan and Google searches, port scanning on the organization’s external IP blocks and similar methods may help locate such systems. Once identified, their lack of security countermeasures (I have never seen a WAF in front of a UAT or DEV server), their detailed error messages being enabled, their lack of MFA (Multi-Factor Authentication) integration, their unmaintained, unpatched servers may let the attackers obtain a foothold into the internal network segments in a super easy manner. Feel free to ask our penetration test team to run a preliminary reconnaissance session to identify them.

Internal or invite-only applications live under the protection of the imposed access restrictions. They are isolated from the wild Internet and thus are inaccessible to most attackers. Our experience has been that the highest number of critical vulnerabilities are identified in this type of closed applications. Open applications, as in the concept of “natural selection”, are exposed to attackers, and their weaknesses get identified & exploited, which in turn create huge problems, and they eventually end up with their vulnerabilities fixed. On the other hand, a closed application is like a person who is never exposed to any microbes, therefore does not have a strong immune system: such an app is healthy as long as it remain in its confined, hygienic space, but the moment that it gets out, trouble begins. The most dangerous case for closed applications is when management decides to make them accessible to the public. A thorough penetration test must be conducted before rolling it out to the wide public audience, to avoid a potential catastrophe.

Internal vs external penetration test is yet another decision point. While an external penetration test could identify the potential entry points into the internal network segments, an internal scan can demonstrate how far a real-world attacker can go, if they breach the perimeter security. In this regard, it is wise to provide a thorough coverage of the perimeter, before proceeding with testing the internal network segments.

If at this point the budget still has room for additional systems, here are some more thoughts on how to make sound additions:

Check the previous penetration test and vulnerability scan reports. If there are outstanding critical vulnerabilities and you cannot track down any evidence of effort for their fixes, add them to the list.
Check security products logs for unusual traffic or high number of errors. If any system is attracting malicious activities, it is worth giving a detailed look.
If there are any unprotected/unmonitored applications, consider adding them to the list.
Locate the applications that are custom developed by 3rd parties. The support contract might have ended (or the vendor simply might have gone out of business). Out of support applications should land on the shortlist.

4. Review, document and finalize

It is always a good idea to retain the documentation created during the preparation and scope setting phase of the penetration test. This would not only ease the process the next time, but also help set comparable scopes for consecutive penetration tests. Then, comparing the results of two or three consecutive tests can highlight the problematic systems that generate critical security risks or point teams that strongly prioritize against cyber security risks.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.