What is this post all about?
Two weeks ago, I started a poll asking how much you have paid/would have been quoted for a simple WEB / API pentest engagement. The poll results are surprising, and like in any other industry, they reflect that some vendors do not or are unwilling to understand the AppSec market dynamic.
Although the poll finished four days-ish ago, and in theory, I would already have written this, I decided to take an extra breath before deep diving into this subject.
Poll Results
Chapter I – Quick Intro
Supposing until this point, almost everyone has an idea about what a pentest is and its benefits, and I will go through this quickly. A penetration test (also known as a pen test or a pentest) is an authorized cyberattack against a company’s network or application. And, it is a highly disciplined process, following a methodology. In 2009, one of the first testing methodologies was defined by Pentest-Standard, and surprisingly, it still contains some gems. However, today, an AppSec pentest is currently following OWASP Top 10 Testing Methodology, a comprehensive resource.
So, going back to the poll results, we see some companies expecting to pay USD 10k+ for a one-week external WEB/API AppSec test. Combining the middle responses with the USD 10k+ category, we can split the answers into two sections, under USD 6k and above.
Note: Sometimes, those engagements are called “grey box” or “black box”, depending on the type of details the client provides. If no details are provided, it is “black box” testing; otherwise, it is “grey box” testing. If the company shares the source code, it is a “white box” pentest type.
The pentest engagement’s deliverable is usually represented by a client’s report for compliance needs or similar. However, during the last 2-3 years, things changed a little. The report’s quality represents 55-65% of a pentest added value. You can have an “ACE” technical team to picture it out better, but if the report is hard to follow and misses the key points, you lose the audience faster than the “Gone in 60 Seconds” movie main action.
Somehow, I am still ashamed when I think about my first pentest report. I have seen hundreds of pentest reports from many companies during my career, and to draw a quick narrator line, writing an excellent pentest report is challenging. Luckily, I had some fantastic people who managed to guide me on how to do it properly.
Chapter II – Going further
If you are not familiar with it, here is a generous public GitHub repository containing several professional pentesting reports compiled by different companies. I strongly advise you to find some time and read them. It will help compare the narrative style differences between them.
Now, by unwrapping a WEB/API report examples from that repository, we will notice the same layout pattern outlining six significant points:
Removing noise, in most situations, a business reads a pentest report from two distinct perspectives: executive summary and its technical side. With this, the whole picture reduces from the initial six bullet points to only two known as catching the eye first, Summary of Findings and Detailed Technical Issues.
I hope you have been able to follow me so far.
Okay, but how would this influence the price dynamic of a simple WEB/API pentest engagement? As I’ve mentioned before, OWASP Web Security Testing Guide does line up in a clear way all the steps a professional pentester, in theory, should follow. Looking closer would mean a lot of effort and checks, roughly say around 140. Besides, the web apps gravitate around a few well-known categories like e-Commerce, Brochure, Fintech/Banking, SaaS. As a result, all the apps used today are just variations of the same few types, meaning only a subset of checks will apply roughly for each, NOT all of them.
As an example, consider that you have a fintech/banking app that needs to be assessed, and you are asking a vendor for a quote. Here’s a high-level example of the process that happens in the background.
I am sure you have noticed the differences because there are a couple. Overall, the biggest the vendor is, the higher the price you will get because many variables play in the equation. I will avoid going into too many details, but the initial value is multiplied with a value between 2 – 4, sometimes even more.
Simply put, a well-established brand will charge you for all the ingredients from the menu, although you have ordered just a soup. A smaller company will give you the soup with a fantastic taste and at least something extra because they cannot afford to screw it up. In this line of business, trust is something hard to earn, and once it is lost, it might be lost for good.
Chapter III – Tips
Without further ado, here are my six tips you should consider before signing off the Statement of Work:
1. Does the vendor have your app-specific testing experience?
It is advisable to ask the vendor what his experience is in testing your app-specific deployment. For example, we are often asked about the top three potential issues affecting the scoped target during the discovery call. This way, you can quickly make a first impression about the vendor’s technical maturity and prepare your next moves.
2. How many technical people will effectively do the testing?
Some companies can provide more than one tester to test the same target. Sometimes this is called a bond team because it operates similar to a spotter and a marksman concept. It is not an uncommon practice, and it has multiple benefits. Ask me more if you are keen to deep dive in here.
3. What’s their level of seniority? Entry-level, Senior, Principal?
As a client, you expect the best from the best to work on your project. However, 72% from all the time proved that is not the case. While everyone has to start from somewhere and build up his professional experience, this should be communicated upfront from the vendor’s side as part of a transparent business relationship.
A senior would have the proper experience and broad automatization knowledge of various tooling arsenal to maximize his effort focusing on impactful findings based on the client business model and application type, rather than spending valuable time chasing low impact or informational results.
A pentest company does have a limited pool of technical resources and even less senior+ technical resources that are not tired. There are a couple of exceptions tho, and you can ask about them if you like.
4. Do they have a profile page or something similar you can consult?
If a vendor provides you with the tester profile, they have a solid company culture and values their people adhere to. I have yet to see a security tester screened out by a client executing a poor job. For the majority of the veterans in this field, it is a matter of pride and honor, knowing their names are narrowed down somewhere with job deliverables.
5. Is there any Quality and Assurance process in place? Does the vendor have a sample pentest report to provide?
This is very important. A Q&A process reflects that someone else is checking things along the way and ensuring consistency.
6. Would the vendor be able to prove the testing coverage?
Another important aspect. Suppose the vendor will provide you details about his coverage assurance process. In that case, that means the vendor is willing to take accountability, work upon if something went sideways because it might happen, and improve.
Chapter IV – Conclusions
The subject is much broader and includes edge situations that I did not detail here for the sake of keeping things simple.
However, suppose you are a results-oriented business looking to get the most from your cyber dollars spent. In that case, all the hints summarized above should provide a decent start in the process of justifying and negotiating an AppSec pentest quote.
With this, thank you for reading, and that will be all from my end for now. If you have any questions, please let me know.
P.S. You’re going to think this is a huge imposition, but if you feel those hints brought some light for you, we’d love to hear from you! We are specialized in working with companies just like yours. If you’d like to find out how we do it, feel free to reach out via securityhubs.io/contact.
