Each day we escape an infinite number of scenarios that could have unfolded, but didn’t. The damage they might have wrought is what justifies security defence, but the cost of ‘what if’ scenarios isn’t easy to calculate, and neither is return on investment (ROI). Assessing a security product’s ROI boils down to a basic (but problematic) formula:
(final value – cost of investment) / cost of investment * 100 = ROI
For example:
(1000 – 150) / 150 * 100 = 566(ish)
What’s so problematic about that? Well, the main fly in the ointment lies in determining the ‘final value’. There is no obvious final value for consumers of security, only a net loss if defenders lose some battles against threats. So, the crux of assessing ROI for Cyber Threat Intelligence (CTI), or any other security product, is trying to figure out the value of something that did not happen.
Some models attempt to address this paradox. Here’s a basic one:
(average cost of a security incident * estimated number of incidents over a set timeframe) – cost of security investment = ROI
There are others, and some are more complex. None yields an airtight justification for a purchase, however they all share a common central premise – security defenders ignore at their own peril: The cost of deploying a security feature directly reduces the loss incurred by cumulative security incidents.
Cyber-security staging: Service in the spotlight, tools backstage
A formulaic approach to calculating security ROI is workable and logical, however may not always be appropriate when subjectively assessing loss from a security incident. Formulas also do not distinguish ‘security services’ from ‘security tools’. So, what is the difference between the two and why is this important?
A service is something like a threat intelligence feed or an outsourced security operations center (SOC); in other words, it’s some combination of people, information, and technology working together to deliver an outcome. By contrast, a tool is purely a piece of technology used within the context of security operations.
Common CTI tools include classics like IBM’s i2 Analyst Notebook, Paterva’s Maltego, and our own tool, READ. These tools are critical for the modern CTI analyst. So, why is it so often difficult to calculate the ROI they offer, in discussions with organisations’ senior decision makers?
Tools are often overlooked in ROI calculations because they are not typically visible at the strategic level of the business compared to, for example, a 24hr online security subscription. Moreover, when tools are included, the cost of acquisition can shut down conversations with decision makers as tools can be an expensive outlay, running into many hundreds of thousands of dollars/pounds of investment.
‘Missing link: The trickle-down effects of the wrong tools’?
Because it’s tricky to include CTI tools in a conventional security ROI formula, there’s a strong possibility that your security teams are missing opportunities to integrate game-changing tools into their threat operations. The potential effects are both numerous and sobering, with two of the biggest being:
Human impact: You’ve heard about the ‘right tool for the right job’? This has never been truer in the professional practice of CTI. It’s rarely the case that a CTI analyst lacking the right tool can’t do the job; more often, the analyst is forced to do the job without the right tool.
This can have a huge impact on the quality of work and morale of the analyst. We’ve seen it turn a brilliant and promising role into a grinding obligation that can quickly lead to attrition. Elemendar’s READ. tool eliminates the burden on an analyst by automating the tedious task of annotating unstructured text with MITRE ATT&CK and STIX 2.1 entities.
Missed details: In CTI – more than in other threat intelligence disciplines – the devil truly lies in the details. The difference between a mitigated incident and a critical failure can be as small as a missed indicator of compromise.
Successful CTI tools focus on details that, once identified, can feed into other cyber-security functions such as threat hunting activities, therefore enabling greater attention to detail. Intelligence analysis tools such as i2, or collection tools, such as Maltego or READ. provide capabilities that validate an enormous ROI: a cog in the wheel of comprehensive security operations.
Concluding Thoughts
Calculating ROI for cyber security is complex, given the strategic nature of an ROI calculation and the tactical nature of tool usage. However, the cost of giving up on the effort is often severe. Security practitioners at all levels of an organisation are likely to suffer the effects of missing or inappropriate tools, including attrition and tension within the team. With that being said, the success of security operations themselves often reflect the presence or absence of appropriate tools that can collect, categorise, and focus attention on critical details.
