At Cobalt we are on a mission to make pen testing not suck. But what is it that “sucks” about application pen testing today and what improvements need to be made? Below I give my view on this.
Traditional Pen Testing
To understand the need for a better pen test model, one needs to look at the traditional pen testing options. Traditionally, on a very high level, there have been two ways to get manual pen testing of your applications.
- Hire a consultancy
- Hire an internal pen testing team
I do see the value in filling a skills gap with a consultant. However, for pen tests, which are becoming more commoditized, a call for a more effective solution is needed. Below I list some of the challenges with working with consultancies:
Not integrated with SDLC: Typical deliverable is a large PDF report, which is hard to digest and communication does not flow easily
Limited talent pool: One needs to book a pen test a long time in advance and matching will therefore sometimes be done based on availability and not skills
High variance in quality: Companies shop around for different consultancies for each test
High cost: This is especially bad if there is a low ROI
Internal Pen Testing Teams
Hiring an internal security staff makes sense, but how you apply their skills is critical, especially given the security talent shortage.
Listed below are some of the challenges I see with hiring internal staff to perform pen testing:
High cost (at least if you hire in US)
Limited talent pool available locally
Motivation and creativity drops when testing the same applications again and again
The onsite talent could be used better for driving other security initiatives than pen testing
An internal team does not satisfy the requirements for 3rd party testing (vendor compliance, PCI etc.)
Pen Testing in General
Besides from the sourcing challenge, there are also key challenges in building an organisation where pen tests are handled effectively. A few of these are:
Hard to plan pen tests to ensure adequate coverage
Hard to drive urgency of fixes — Feature delivery takes priority over security patches
Hard to get overview of the pen tests done and the finding status for all pen tests
A Better Model
At Cobalt we understand these issues with the traditional models and are continuously working on building something better. Below I have highlighted some of the key areas that we believe makes for the best pen test experience.
Globally Sourced Talent from a Vetted and Diverse Pool
Most web, mobile, API and ext. network pen tests can be done remotely. Thus limiting yourself to a local talent pool might create a higher level of trust, but it will significantly limit your ability to get the right skills at the right time. In turn, by dipping into the global gig economy, one can source skilled security freelancers worldwide who are highly motivated to test applications for companies that they previously did not have access to. With a large and diverse talent pool, you can also be more picky on who you select for the engagements.
Using emails and PDF files is not an effective way of communicating around a test. Similar to how developers use tools like Slack, Jira, Clubhouse or Github, an effective pen test model needs a modern platform with standardized workflows, structure, and an easy flow of real-time communication. The platform should be integrated with the development tools previously mentioned to ensure that questions and findings are quickly pushed to the right developer for effectively identifying and fixing vulnerabilities.
By using standardized platform-supported workflows and modern vetting/rating mechanisms (similar to services like Lyft or Airbnb), one can build a pen test model built on trust and credibility of the people performing the tasks, as well as ensure consistent quality of the deliverables. The vetting does the initial filtering, and a well-thought-out workflow and rating system ensures that pen testers keep delivering quality.
On-Demand Aligned with Agile Development
Modern businesses use agile development and security testing needs to follow along. Being closely aligned with the software development lifecycle (SDLC) is of high importance to a successful pen test model. If the SDLC is agile, scheduling should not take 4–6 weeks, but be on-demand and communication around the findings should happen in close to real-time so the developers can work effectively.
Less Time Spent on Reporting, more on Finding
By automating parts of the reporting process (like suggested fixes, methodology, etc.), you ensure consistent quality, but you also take the workload off the security researchers doing the testing. Thus they can focus on what really matters, covering more of the scope and finding as much as possible.
Overview of Status Across Multiple Teams/Apps and Past Pen Tests
How many pen tests did you do last year? What is the status of fixing the issues found? These questions are not necessarily easy to answer for most businesses. A modern pen test model should provide an easy overview of all previous pen tests and also allow businesses to see trends and plan for future testing.
Where is Cobalt on this journey?
Since 2013 we have been working on building a platform that can support a better pen test model as well as a talented and vetted community of security researchers (The Cobalt Core). We have built the baseline for most the things mentioned above and are constantly working on to make it even better and more agile. A big part of building this new model is getting feedback from security researchers and existing/potential customers. Thus we greatly appreciate any feedback/input and thoughts and we will send a T-shirt to anyone with valuable ideas.
If you are interested in learning more about the Cobalt model, feel free to reach out to us directly.