There is a surplus of companies that offer penetration testing services today, but not all provide like-for-like services. To help simplify the decision-making process and help consumers find the best-fit supplier, this article outlines 5
- The skills and knowledge of the tester on your engagement
- Certifications (as required)
- Testing Methodologies
- Testing Scope
- Company Culture
The most important question is one you will have to answer yourself: what are your requirements? Throughout the process, it’s crucial to know if your supplier can work as your partner and can understand your requirements for a penetration test that meets your needs.
There are many different ways of conducting a penetration test and many different layers that can influence the level of assurance a supplier can give you. Although these considerations may not be necessary for your requirements, they should be explored, and the security firm should work as a partner to ensure the best fit is found. Finding the best fit solution requires pen test suppliers to understand the entire ecosystem for assessment.
1. "How will the team be identified and scheduled?"
The team behind the test is a central aspect of the assessment. It is critical to find a vendor that can meet your specific needs.
But senior, technical consultants aren’t always the best choice.
Being a technical expert often requires specialization, so you’ll want to ensure that your supplier’s niche align with your assessment needs. If timing is critical, consultants with a broader set of experience may have more availability.
But oftentimes, organizations don’t know the niche or level of experience that their assessment requires and lean on their suppliers to help them understand their specific needs. . For example, suppliers that can identify when a job is not niche, can assign consultants with the right level of experience in a shorter timeframe.
For more niche tests that require a specific skillset, you may want to ask potential suppliers about the number of consultants that possess that qualification, and how involved they will be. The more qualified consultants that a company has, the more likely these consultants will be available in a shorter amount of time and involved in the project from start to finish. This is particularly important for schemes such as CHECK in the UK, and the new mandates required under the US Proactive Cyber Initiatives Act of 2022.
2. “What Methodology Will be Followed?”
When discussing options with your penetration test supplier, it is crucial to understand the methodology that will be followed. To understand each company’s methodology, you might ask:
- Is testing to be done in a logical order?
- Will it use any well-known benchmarks?
These benchmarks, though extremely useful, are not definitive. In web app testing, most companies will test for the OWASP top 10. Although a great benchmark, the OWASP top 10 only accounts for the most common vulnerabilities from the latest reported data. A strong company, therefore, will test against OWASP while covering many more areas in a methodical approach.
Companies that perform thorough tests will also ask for additional access or information (e.g., administrative credentials or code for an application). While you may be skeptical about providing this kind of information, this data will ultimately make for a more efficient test. First and foremost, it will give suppliers a better understanding of the application or environment and its use. This level of understanding makes it easier for consultants to find subtle vulnerabilities or test filters and allows them to provide a more in-depth report on the system’s security.
3. “What is the Scope?”
Understanding how and why a scope exists can provide valuable information about whether the company is a strong choice as a partner. Assessing the number of days assigned for a test and asking how that conclusion was reached ensures that a thorough test will be completed.
Remember that the cheapest may not mean the best value for money. Cheaper options often reduce the number of days assigned, thereby not covering a system sufficiently. In addition, cheaper day rates often get passed on in consultant wages. In a highly competitive market, this can often mean that when a price is significantly lower or seems too good to be true, the assigned consultants are often less experienced and new to the industry.
4. “What is the Company Culture Like?”
Though often overlooked, the supplier’s company culture can significantly impact the outcomes of a test and the service provided.
An excellent way to understand a company’s culture is to understand its training practices. These questions will help you understand how a company’s training regime aligns with your testing requirements. Some questions to consider:
- How well are the consultants trained?
- How much time is set aside for training?
- How broad and deep is the training?
- What kind of on-the-job training is provided?
If a supplier utilizes ‘on-the-job’ training, you’ll want to understand how this type of training will align with your testing requirements. Many companies train consultants’ on-the-job’, which can be effective for standard jobs but can result in gaps in knowledge should a specific vulnerability or technology appear.
You’ll also want to understand the type and level of training needed. If training only exists in ‘core’ practices, such as application or infrastructure, this may not translate into the same testing standard in a mobile or cloud test.
Outside of training, you’ll also want to understand the supplier’s breadth of offering and knowledge-sharing practices by asking the following questions:
“Is there a process for consultant knowledge sharing?” Another aspect of culture to consider is if and how knowledge gets shared across the business. A company with highly experienced consultants in various disciplines will likely have a wealth of knowledge available. If this information gets shared amongst the team, and there is a culture of assisting each other, everyone benefits.
“What processes does the vendor have in place for their own security?” Lastly, understanding how a company manages its internal security (e.g., what controls and mitigations are in place?) can be a perfect indicator of a company’s maturity and how well it understands security. Not only does it give assurance on the safety of any shared data, but a company that takes care of its own house has the kind of culture that trickles into all areas and influences some of the points above (i.e., how seriously they take training, research and security in general).
5. What Other Services Are Available?
Most suppliers can easily arrange testing for a specific environment or application. However, sometimes, you will discover in ongoing conversations with your vendor that you need more than a simple penetration test. A mature company can offer a range of services that will better meet your requirements. In addition, a mature company produces the results you’re looking for overall by advising on what additional services are needed.
For example, while a test on a web application can be straightforward, a more in-depth test can be done by performing a code review alongside it or a code-assisted test. If testing discovers any vulnerabilities, there may be a benefit in having a gap analysis of the secure development life cycle. Or if the vulnerabilities are complex, it may be that assistance in understanding and remediating could be helpful, or for consultants to attend meetings to explain the overall risk to senior executives.
It may be the case that these services are not required. Still, an understanding of precisely what a company can provide means there is an understanding of what is available and the ability to pick the best service for the requirements.
In Summary
There are many ways to choose a penetration testing supplier and partner to provide assurance and advice on security and many variables that can influence that choice. Focusing on a few key questions (see the summarized list below) will help you select a supplier with a holistic approach to testing. Considering systems as a whole rather than disjointed parts will ensure that security advice is relevant and achieves the best coverage that meets your needs.
A Short List of Key Questions to Ask Suppliers
- How will the team be scheduled?
- What methodology will be followed?
- What is the scope?
- What is the company culture like?
- What other services are available?
Global Practice Lead, Code Review and Native Application Security, NCC Group
As a Global Practice Lead at NCC Group, Katy is responsible for growing the code review and native application practice globally. This includes understanding regional differences and individual challenges, ensuring products meet customer needs and developing and implementing career pathways to ensure consultants have the skills and support required to deliver work of the highest quality. Learn more about Katy in our Colleague Stories blog post.
Are you looking for more on pen test suppliers?
Whether you are in the discovery and education phase or ready to pick a supplier – NCC Group is here to help. Our Resource Hub has a collection of posts focused on all the different aspects of penetration testing and its benefits. Or, if you’re ready to pick a supplier, speak with one of our experts today to see if we’re the right partner for your organization.