Cobalt Crowdsourced Application PentestCobalt Crowdsourced Application PentestCobalt Crowdsourced Application Pentest


As a security provider to MSPs, Datto needed a pentest partner that could support its goal of delivering world-class security to its customers. Learn more from Justin Bacco, Datto's Application Security Manager, and Jeremy Galindo, Datto's Security Engineer.

Justin Bacco & Jeremy Galindo

How Datto Ensures World-Class Product Security with On-Demand Pentesting

As a security provider to MSPs, Datto needed a pentest partner that could support its goal of delivering world-class security to its customers.

Datto is a technology, business continuity, and disaster recovery provider for IT Managed Service Providers (MSPs). The company’s integrated, world-class products and services provide MSPs with a comprehensive toolkit to manage their business, drive efficiency and growth, and expertly protect their customers. As a provider of secure cloud, SaaS, and file protection services, Datto places cybersecurity at the heart of its business.

As Application Security Manager, Justin Bacco manages all of Datto’s offensive security initiatives and strategies. His team thinks and behaves like bad actors do, proactively using ethical hacking, red teaming, exploit research and development, and other offensive techniques to identify security weaknesses. His colleague Jeremy Galindo, a security engineer, focuses on exploit development and security R&D and is heavily involved in day-to-day security testing.

The Challenges

Initially, the main driver for Datto’s pentesting program was SOC 2 compliance. The company previously worked with a handful of third-party vendors who delivered pentesting services. However, their approach to vulnerability reporting was challenging to integrate with Agile development workflows.

“We were working with companies that use the old-school ‘email-and-PDF’ style of reporting,” Bacco explains. “We would kick off a two-week pentest and then have to wait two more weeks before we finally get the PDF report. Then it would have mistakes, and we’d have to push back on it. There wasn’t much communication, and that created real challenges for us.”

Lacking a channel for real-time communication with pentesters, Datto ran into several problems. Most notably, the company’s engineers weren’t receiving vulnerability reports in a usable format. Not only were they unable to seek clarification where needed, there were also issues with feeding reports into established engineering sprints. Combined, these issues made it difficult to obtain full value from each pentest.

Over time, Datto wanted to expand its program to support a more rigorous testing approach in line with its commitment to industry-leading cybersecurity. At this point, it became clear the traditional method wasn’t providing the high-quality, full-coverage testing Datto needed. Instead of having the same 2-3 pentesters working on the same assets year after year, Datto needed a more diverse pentesting approach.

The Solutions

To take its security program to the next level, Datto needed a pentest partner that could deliver four things:

  1. High-quality testing with broad and clearly-defined asset coverage
  2. Real-time communication between engineering, security, and pentesters
  3. Better integration of pentest findings with engineering workflows
  4. Access to a diverse range of pentesters to avoid diminishing returns

Cobalt’s Pentest as a Service (PtaaS) platform immediately delivered on all four requirements. The platform made it easy for Datto to expand its testing program and focus on delivering world-class security. When asked what it was about Cobalt that made it the right choice, Bacco and Galindo were clear:

“Having instant communication with testers is a driving factor in why we chose Cobalt and continue to use them,” explains Bacco. “Talking to the researchers in real time and getting instant feedback on issues, or answering their questions, just makes for a better quality pentest. If a researcher is stuck on something and we can answer their question within 10 minutes, that really helps dig out vulnerabilities.”

Essential real-time communication wasn’t the only benefit Cobalt provided. Cobalt’s community approach to pentesting -- in which vetted, seasoned testers are matched to client engagements based on expertise -- gave Datto access to a broad range of testing skills. Executing a comprehensive pentest program on an annual cycle allowed Datto to flexibly switch pentesters between engagements, or request new testers and ensure full coverage for critical and frequently updated assets. When it came time to deliver pentest findings to engineering, the Cobalt platform came through again.

“The platform is just more conducive to involving different people at the company in investigating results, triaging them, and getting them into our ticketing system,” Bacco continues. “These are all things that were difficult in the past but are much more easily facilitated through a platform like Cobalt.”

It was clear from the outset that Cobalt’s PtaaS platform was far better suited to Datto’s needs than the traditional approach. Almost immediately, the company took the opportunity to expand its pentesting program to better serve its customers’ need for security. Justin Bacco explains:

“Once we got involved with Cobalt, our pentesting program went from mostly being a SOC 2 compliance effort to having a larger and larger scope as time went on. We strive for excellence as best we can, particularly when it comes to security. We find Cobalt is a company that matches the high expectations we set for ourselves.”

Cobalt also simplifies the process of involving Datto’s developers early in vulnerability management. This ‘shift left’ helps the company to action pentest findings more quickly and enables developers to play a more proactive security role than they could in the past.

Ultimately, the quality of a pentest comes down to its results. With Cobalt, Datto not only receives a more reliable stream of confirmed vulnerabilities, they also have a more detailed understanding of the coverage they receive from each pentest.Bacco explains,

“With Cobalt, we have a portal where we can see every specific spot that the testers looked at and the different types of attacks they've tested. That makes us comfortable knowing that even if nothing was found, we know we got good coverage of the application and this approach feels very thorough. If you compare it to the alternative, where you get a PDF report that says there were no findings, but you don’t know how much of the application was covered, there’s no comparison. With Cobalt researchers, we know exactly what they’ve looked at and which attacks they have tested.”

What’s Next for Datto?

After seeing first-hand the value Cobalt’s platform adds to Datto’s security mission, Justin Bacco has his sights set on the future.

“We’ve already expanded our pentesting program to take full advantage of the Cobalt platform. Moving forward, I can see Datto almost entirely outsourcing all of its penetration testing work to Cobalt, while we focus on different components of offensive security.”

The Challenges

  • ‘Email-and-PDF’ vulnerability reporting is too slow and cumbersome for Agile development
  • Lack of communication between engineers and pentesters slows down the remediation process
  • Having assets tested repeatedly by the same testers leads to diminishing returns

The Results

  • Datto can switch existing testers or request new ones for every pentest, allowing the team to conduct periodic testing as part of a comprehensive pentest program
  • Having vulnerabilities retested takes minutes, not weeks
  • The PtaaS platform enables real-time interaction and results
  • Confirmed vulnerabilities go directly to Datto’s Slack and Jira workflows