Cobalt Crowdsourced Application PentestCobalt Crowdsourced Application PentestCobalt Crowdsourced Application Pentest



Optimizely is the leading mobile and website experience optimization platform. Their customers include major high-profile brands such as Sony, Microsoft, and NBCUniversal.

With a focus on customer experience, Optimizely frequently releases new features and updates to ensure that every experiment is powerful. Security is also a top priority for the team. With more than 8,000 customers, they are dedicated to creating a safe and trustworthy technology for running on their customers’ websites.

Kyle randolph
Kyle Randolph
Staff Security Engineer

Connecting to the security community

In an effort to find ways to further enhance the security review of their applications in a cost-effective and scalable way, Optimizely decided to explore launching a bug bounty program. Kyle Randolph, Staff Security Engineer at Optimizely, states,

"We decided to run a bug bounty program as a sustainable and continuous security testing solution to ensure that we are protecting our customers as best we can."

Kyle and his team discovered the benefits of bug bounty programs after researching the security methods of tech giants like Google and Facebook. They understood the value of running a public bug bounty program, but were not ready to take on the complexity and cost of setting up their own responsible disclosure approach. So, Kyle sought a reliable third party partner to help the team implement and maintain a customized program.

Kyle’s team was looking for:

  • Ability to quickly set-up and host a bug bounty program
  • Visibility to all incoming reports through a web-based platform, allowing them to directly communicate with contributors and quickly fix bugs
  • A service that could provide resources to help manage incoming vulnerability reports.

After researching different solutions, they selected Cobalt as it satisfied all the requirements listed above.

A Managed Program

One of the key reasons the Optimizely team chose Cobalt, was the possibility of getting a managed bounty program to simplify the process of managing and triaging incoming security reports. With this option, they have a dedicated security engineer to directly monitor all incoming vulnerabilities reported from the crowd and provide a first level response.

Kyle Randolph explains:

"Not only does Cobalt deliver a great platform, but the managed service where you can get skilled security engineers to triage incoming reports, frees up time for us to focus on the most important thing - fixing the issues."

JIRA Integration and Labeling

The Optimizely team is especially satisfied with Cobalt Central’s collaboration features and integration options. JIRA integration helps the team easily coordinate vulnerability reports into their development process. To organize reports, the labeling feature allows vulnerabilities to be categorized with tags like "triaging" and "awaiting response." This feature streamlines the workflow between researchers and the security team, which means that bugs can be fixed quickly.


Using a crowdsourced security solution, Kyle and his team at Optimizely have successfully been able to:

  • Adopt and maintain an ongoing model for security testing their platform in a controlled manner
  • Organize and respond to incoming reports with the help of Cobalt’s managed program option
  • Effectively integrate the bounty program to internal processes
  • Collaborate directly with security researchers to quickly fix any bugs

"Cobalt has made it easy for us to get valuable feedback from security researchers and respond to them in a timely manner."

Cobalt bug hunters@2xChallenges

  • Implement a continuous security solution to remediate vulnerabilities without managerial overhead
  • Adopt a tailored bug bounty program that is easy to use and maintain

Icon  checklistSolution

  • Deploy a Cobalt managed public bounty program to find vulnerabilities on their platform on an ongoing basis

Cobalt certified researchers@2xBenefits

  • A structured bug bounty program integrated into the development processes
  • Quick access to and communication with a diverse set of security researchers
  • Lower $ cost per vulnerability found