Many CEOs and boards have a zero-tolerance policy regarding their CIOs and CISOs. In a recent survey from Osterman Research, 38 percent of companies thought that a data breach that was made public was a fireable offense.
From a “CYA” perspective, but more importantly for security infrastructure and policy optimization, providing industry-wide benchmarking data could play a critical role in helping IT security teams better understand whether they have put best practices in place – and provide better job security. At this point, this data is hard to get, beyond the ad-hoc sharing among industry peers. But exciting new products and development efforts – driven by machine learning – are on the horizon that offer specific security benchmarks relative to a company’s stance, and provide recommendations for improvements to configurations, products and policies.
The benefits of industry benchmarks
In general, one can think of CISOs as baseball GMs and managers, that every year spend millions of dollars on new products (players), provision these products to interoperate at some expected optimal level with the legacy products (Spring training to foster teamwork with the players returning from the prior season), and then hope that they prevent all external and internal threats (winning the World Series!) Just as most baseball teams don’t win the World Series, most CISOs don’t have teams that prevent all the inevitable attacks. And then those that fail, either in baseball or security, tend to ultimately lose their jobs. Joe Girardi almost made it to the World Series, but even that didn’t allow him to keep his Yankees managerial job after their loss to the Astros in the ALCS.
Just like in baseball, even those with the largest budgets – like the Yankees – are not always successful. So how can better information be provided to security executives and their teams to make better decisions, and help save their jobs when bad things inevitably happen?
It’s an easy case to make that pooling information about security is good for the industry and for CISOs. First, it would help with threat detection. If a mid-market bank suffered a breach from hackers, it could inform others in the same industry category that an attack has occurred, where the attack originated from and how the attack was carried out. It could also detail how the firewall and other security devices and products were provisioned in a certain way and detail what steps are needed to better prevent it. This would help better defend the industry and help the bank under attack better understand what they might have done better if anything.
The other benefit is self-preservation. CISOs want to be able to demonstrate that relative to their peers, they’re doing everything humanly possible to deter an attack. Over time, security hygiene could become a board-tracked initiative, like compliance. On the other hand, the board would be able to better determine if the IT team didn’t take the right security precautions. They could see that although they spent $200 million on devices and services last year, their IT department didn’t think to update their Linux servers when a vulnerability was exposed a month before. The board could look at how the CISO’s peer group reacted and determine whether they lagged behind.
This is a compelling business case. Companies would be willing to spend for this data because it would help to make their organizations more secure. It would also let them know if they’re spending more than their peers for better or worse results. A recent SANS survey showed that only 22 percent of companies currently benchmark their IT spending and at least one survey respondent complained about the lack of existing benchmark metrics. It’s no wonder that startups like Smart Hive are eyeing this space.
The ‘first client’ problem
The limitation of this approach is that for it to work, everyone — or at least a critical mass of companies — needs to take part. But companies are loath to disclose such sensitive information.
One way around this is to offer different pricing tiers. A company that disclosed its own information on an anonymized basis might get a huge discount. But those who want data but don’t want to disclose their own would pay a significantly higher price.
The idea of pooling data to create industry benchmarks isn’t unprecedented. Back in the late 1990s, Keynote Systems and Service Metrics made a business out of providing benchmarks for website performance.
Back then, the guys who ran IT at retail financial institutions like Fidelity got pumped on being at the top of the leaderboard relative to Schwab or Waterhouse or the other web trading platforms. Literally, they tried every month to be at the top of the leader board, by demonstrating the fastest website load times and web application transaction time. If they made the top, they ultimately had a chance of being compensated more generously. If they were at the bottom, they risked getting fired. They loved it, but they were also scared to death. This was what then helped drive adoption for benchmarking platforms.
There’s a similar need for metrics around security. The question is whether individual companies will submit to parting with some information for the greater good. That will take some doing, but there’s no doubt that in this case what’s good for the industry at large will also help protect the CIOs and CISOs jobs.
Benchmarking is Inevitable
CISO’s don’t go to bed wondering if other organizations have learned from them, they go to bed worrying if their organization is not aware of a threat that others have already stopped. Benchmarking their security against others in real-time allows an organization to have situational awareness.
Today, the only situational awareness an organization has is limited to what their security operations team can see. This is limited to events and data generated by the tools deployed at the organization. A threat actor takes advantage of this limited awareness. The threat actor tries various attacks on an organization until one is successful. Eventually, either one of the deployed tool or someone in operation team will identify the attack and stops it. This can happen minutes after the attack or hours and weeks after the attack. Once the attack is stopped both the organization and the hacker move on. The hacker moves on to its next target while the organization moves to stop the next attack. This puts the organization in a permanent defensive position.
This scenario is repeated over and over at company after company. What if the first organization to stop an attack was able to tell others what actions it took to stop the attack? What if it did this within minutes of stopping the attack? The situational awareness for organizations sharing data vs. those not sharing data will be dramatically different.
This is where benchmarking and data sharing platforms can provide extreme value as organizations will be able to see in real time what attacks they have against them vs. others.
This real-time benchmarking would drive decisions including:
- Do I have the right tools in place?
- Are my tools configured correctly?
- How effective is my team compared to other companies like me?
A CISO can use real-time benchmarking data when presenting to the board of directors. A CISO can easily show if they are better than their peers at stopping threats, as good as their peers or far behind and with necessary investment needed to improve. Benchmarking can also be used in a breach situation. An organization can show that they did everything they could to stop a breach as well as everything their peers were doing.
Organizations will share data if they feel the information they are receiving in return is a lot more valuable than the data they are sharing. Value can be measured by asking, is the information I am getting back, real-time, relevant and actionable?
This problem can only be solved by leveraging machine learning to fingerprint the security products and configurations, while also identifying the attempted and successful attacks on the company’s infrastructure. Fortunately, this technology is now ready for prime time, and promising solutions leveraging it will be in the market shortly to help solve this industry-wide problem.
