CISOs, unis, investors turn to richer metrics as security training evolves
SA&T is evolving fast. It needs to...
The modern cyber crisis, as UK security awareness and training (SA&T) firm Immersive Labs puts it, is an “all-encompassing organisational trauma”. Take a trip down memory lane to the ransomware attacks on the world’s largest shipping firm, A.P. Møller-Maersk, for one vivid illustration of that claim -- or more recently, revisit the cancelled operations and organisational turmoil across Ireland’s healthcare sector after an attack in 2021.
CISOs know, equally vividly, just how “all encompassing” cybersecurity is -- as an organisational and profoundly human challenge. The security awareness and training (SA&T) world is evolving rapidly as a result, not least as the wholesale shift to remote work triggered by the pandemic woke up those security leaders working with a narrowly technical solutions-based mindset to the fact that a security culture needs to be deep and borderless.
That has driven real change in the security awareness and training solutions world (as research house Forrester puts it, the pandemic caused “well-needed disruption in a long-stagnant market”) and increased investor activity too – witness SA&T provider KnowBe4 this week agreeing a $4.6 billion deal with Vista Equity Partners.
It’s also caused a rethink in cybersecurity training outside of the SA&T world.
A case in point is Airbus’s collaboration with Cardiff University: The two this week launched the Airbus Centre of Excellence in Human Centric Cyber Security at Cardiff University. Critically, that is led by Professor Phillip Morgan from the University’s School of Psychology -- whose research background has included work on the "effects of visual and auditory interruption and distraction on performance of tasks involving attention, short-term memory and problem solving" as well as work on "adapting features of computer interfaces to encourage deeper cognitive processing of key information and thereafter protect against negative effects of interruption."
Professor Morgan said: “Optimising people is the key to cybersecurity. We may have the best hardware and software, but we realised more had to be done on human centric factors within cybersecurity, within industry, within workplaces – we recognised seamless cyber security is impossible unless we get the people factor right.”
A focus on psychology and a more human-centric cybersecurity strategy along with the ongoing evolution in the SA&T realm is being warmly welcomed by security professionals.
As security consultant James Bore (who previously operational security for a major UK telco) puts it to The Stack: “The classic CBT [computer-based training] many companies rely on is almost a joke in security departments. Its main benefit is that it is easy to measure, and cheap to deploy, so it meets compliance goals and makes auditors happy."
He adds: "Fortunately, a number of vendors are moving to much much more engaging, better-designed, micro-training making use of sound pedagogical principles."
Not all of these vendors, of course, get it.
As Forrester's Q1 2022 SA&T “Wave” report puts it: “Traditional SA&T program metrics such as training completion rates, quiz performance, and engagement metrics are fundamentally flawed… our briefings were full of vendors paying lip service to awareness, behavior, and culture change. This is pleasing in that it shows that they got the memo. However, many had a limited vision of how to change behavior or instill a culture and quickly reverted to describing their content and quizzes as ways to measure employee engagement.”
(For what it’s worth Forrester tracks 11 key players: Cofense, CybSafe, Elevate Security, Infosec Institute, Kaspersky, KnowBe4, Living Security, Mimecast, Proofpoint, SANS, and Terranova Security – putting Knowbe4, Cybsafe, Living Security and Proofpoint among the leaders in its report earlier this year.)
Security awareness and training (SA&T) and bad phishing
Competent CISOs have long recognised the importance of inculcating a security culture – and making sure it is one that is not driven by an old-fashioned approach of penalties for failing phishing simulations, or worse, self-congratulatory performance results for such exercises, which can be tempting to manipulate.
As Veeam CISO Gil Vega, a former federal CISO with extensive experience in this, earlier told The Stack “any company is potentially one click away from an existential event” – and getting security culture right is about ensuring that employees understand that they’re part of the solution: “We created this competitive spirit amongst employees to find security vulnerabilities in our infrastructure — people really loved being rewarded for identifying these problems; they got really competitive about trying to get the newest trophy!”
Yet too many companies continue to over-index on things like phishing simulation emails; often controversially seen as fundamentally malicious by non-technical staff, not least when they include things like “congratulations, you’ve been awarded a bonus” or other lures designed to ensure a click-inducing dopamine hit.
Indeed, the UK’s NCSC this week warned companies that too much emphasis was being placed on phishing tests, saying: “Responding to emails and clicking on links is an integral part of work.
"Attempting to stop the habit of clicking is not only extremely difficult, but is it what you want? Asking users to stop and consider every email in depth isn't going to leave enough hours in the day to do work.
As the NCSC’s “Kate R” puts it: “Phishing simulations aren't just about training. They are also popular because they produce a metric (e.g. 'Last week 60% of people fell for our phish, this week only 35% fell for it'). It appears really positive and encouraging… Metrics are extremely difficult to come by in the security space, and having a clear, quantitative metric that can show progress... can be really seductive…
“The risk of living or dying by this single metric is”, she warns, that you can “get any result you want by adjusting the emails you send out, which is hardly an objective measure of your defences. And if you are on the receiving end of a metric that shows a vast improvement, you should be asking some very probing questions about how the simulation was designed..."
Metrics and exercises are evolving rapidly
To the UK’s Immersive Labs – whose customers include HSBC and Goldman Sachs – generating positive engagement around building a more human-centric cybersecurity approach is critical, but exercises and metrics really need to transcend the CBT or phishing training norm. As CEO James Hadley puts it to The Stack: “Too many organizations rely on race to the finish elearning or videos to educate the wider non-technical workforce.
“However this doesn't prove or measure the current knowledge, skills and most importantly judgement that the workforce has. We perform micro exercises to see what decisions individuals would make and record things like confidence and rationale, and dynamically change the training based on decisions made. This tailors the feedback, ensuring those that are making good decisions with high levels of confidence can complete it quickly, whilst those making poorer or uncertain decisions get in the moment information and nudges to help improve security awareness. Most importantly CISOs can see which categories of exercises are creating the most uncertainty or poor decision making enabling better prioritisation of future security spend…”
As Hadley adds: “Security moves on so quickly, the suggestion that being certified or attended an out of date course a year ago means you've got the in-house capabilities based on today’s threat isn't good enough.”
Quietly, many CISOs and other security leaders mutter to The Stack that some of the most egregious awareness issues come from the top of the company; something that problematic when you are trying to encourage both cultural change and get those same people to sign off on budgets to help tackle the problem.
One CISO who has nailed that is Veeam's Gil Vega, who earlier told us that his position had been set up to report directly into the CEO, with an independent path directly to the board of directors: "I am part of the CEOs leadership team, and the executive team, and that allows me to provide a lot of input and guidance on the company’s strategic plans; where we’re going to spend money, where we’re going to invest, where we’re going to open new business... [that setup] really reflects a state-of-the-art governance model; because there’s still a lot of large companies out there that don’t quite take this issue [cybersecurity], as importantly as they should."
CEOs reading this and wondering how they can change security culture should consider not just new SA&T techniques and tools but think about how their organisational reporting lines are structured.
That broader organisational awareness and understanding is critical to SA&T success, says Mario Platt -- an experienced former interim CISO who is now Director, Governance, Risk Management and Compliance at LastPass -- because "the problem with most traditional approaches to security awareness, is that it makes the gross oversimplification that the 'awareness' needs are the same across different business units, when they’re not.
There are some common themes that everyone needs to be aware of, for instance knowing how to spot a phishing event. IIn order to 'do' awareness right, we need to understand the operational constraints and trade-offs which are an integral part of the work performed by each business unit, and understand how it affects their work."
But, he adds to The Stack: "General awareness is not a substitute for understanding the context of each team and how risks may materialise in the context of their role and work. It can also become a red herring, and frame as problems of awareness what is really an issue of conflicting goals and limited resources, so directs enquiry to 'feel good activities' such as watching videos, when we should be really focusing on re-designing work systems that aren’t capable of identifying and handling these goal conflicts..."
nb: The NCSC's new phishing guidance emphasises the importance of developing multi-layer defences so you have multiple opportunities to stop a phishing attack causing serious damage.
