Anti-Phishing: Measuring Phishing Awareness Training Effectiveness

Do you train your employees to understand what phishing is, what it looks like and what to do if they realize they’ve been targeted?

As you’re about to find out, you better be doing all three. Phishing is a scam that is growing in terms of numbers and sophistication.

However, aside from training your people to understand these attacks, you also need to make sure your efforts are effective.

Why It’s Important to Track the Effectiveness of Your Phishing Awareness Training

Hopefully, we don’t need to spend too much time on this topic because you already appreciate that training your employees to avoid falling for phishing scams is of paramount importance.

Nonetheless, it may be helpful to review some statistical data that drives this point home. Some of you might be working in IT and need evidence to build your case to management about the importance of proper phishing awareness training, as well.

• 85% of companies have already been the victims of phishing attacks. If yours hasn’t already, just know it’s probably coming. Sadly, you may have been successfully targeted already without even knowing it.
• 60% of organizations reported that the rate of attacks they’re seeing is increasing. So, again, if you don’t already fall into that 85%, expect that you’ll be attacked this year.
• In fact, during the first quarter of 2016, phishing attack incidents skyrocketed 250%. While increases are normal during the holidays, this happened after a sustained surge that built up over the course of three months.
• 30% of phishing emails are opened by their recipients. This doesn’t necessarily lead to a successful attack, but this should show you why training is so important. Even with the software most companies now use to send phishing emails right to junk folders, many still get through.
• Roughly 90% of phishing emails carry ransomware. This type of attack is meant to block off an essential part of a company’s network. It’s only restored to the organization once they have paid a ransom for it.
• The average cost of a phishing attack is now more than $1 million. For the specialized versions known as spear phishing, the average damage done goes up to $1.6 million.
• About 33% of companies have been successfully targeted with CEO fraud emails. This is why it’s so important that executives are included in training. Many scam artists focus on them because their credentials make them such profitable targets.

The big takeaway here is that if you don’t take phishing awareness training seriously, your company is going to be victimized, perhaps numerous times. These are not small-time scams, either.

Many hackers now use phishing as a way to then install viruses on a company’s network. When Target was hacked (which happened because an HVAC company they worked with was first victimized), they had to pay around $50 million in settlements, the CEO resigned and they were even investigated by the Secret Service.

While this is probably an extreme example, the point is that the price victims have to pay has greatly ramped up. Ransomware aside, many companies simply get their money stolen outright and all victims generally need to undertake some level of PR to restore their reputation and keep their business alive.

How Can You Measure Phishing Awareness Training Effectiveness?

In the next section, we’ll look at how you can develop, collect and analyze metrics related to the effectiveness of your phishing awareness training.

For now, though, we’ve listed metrics you can use to assess how effective your training has been and then the unit we suggest you utilize for the actual measurement.

• Successful Attacks – How Many You Receive in Six Months to a Year
• Phishing Emails Reported – How Many Your IT Team Receives in Six Months to a Year
• Results of Simulated Phishing Attacks – How Well Your People Avoid Becoming Victims

Now, there are a number of ways you can develop and collect these metrics. Let’s take a look at all three in a bit more detail.

The first one is clearly the main metric you’ll want to use to see if your employees are getting closer to or further away from becoming proficient in defending against phishing attacks.

Unfortunately, it’s also a metric that you really don’t ever want to actually have to use. Hopefully, you don’t have to deal with too many of these attacks and none of them are remotely successful.

Still, it’s an important metric to keep track of. Whether you want to choose six months or a year for your timeframe, you absolutely need to make every effort to see this number go down over time.

The second metric is how many phishing emails get reported to your IT security team.

This measurement must work in tandem with your last one. That’s because you may see the number of attacks that get reported go down, year after year. As long as you’re not succumbing to successful phishing scams, though, this isn’t remotely a bad thing.

Of course, the sad truth is that this is unlikely. There is every reason to believe that the barrage of phishing attacks every company sustains will continue to increase year after year.

This is why that first metric should go down while the second one goes up. If it does, this is a very good sign that you are doing a fantastic job with your phishing awareness training.

Keep in mind, too, that you can actually hire third-party security companies that specialize in sending out “phishing” emails to their clients’ employees and then reporting back which ones were found lacking.

In lieu of actual phishing emails, this is a great way to administer this metric and collect the results.

Finally, you should be regularly sending questionnaires out to your staff to make sure they understand what to look for in phishing emails. You can do multiple-choice questions about what traits give these emails away.

Score them and then see what the average is for your employees. Again, this score should improve over time. However, you also need to factor the average score against the average length of time people have been with your company.

If you’ve just hired a lot of new employees, your average score may initially suffer.

Developing, Collecting and Analyzing Metrics

Earlier, we looked at how to measure phishing awareness training effectiveness. In this section, you’ll get the actual how-to’s, so you can develop metrics unique to your company, test your staff on them, collect data and analyze it to make sure your organization is safe.

Developing metrics related to phishing scams should revolve around three major analytics:

• Are People Becoming Victims? This is the most important one. The others can be used to help inform your training, but if staff members are still clicking on           fraudulent links or otherwise falling for these attacks, nothing else matters.
• Are People Reporting Attempts? Not falling for a phishing scam is fantastic, but also reporting these attempts when they happen would be better. You want your security team to be able to react and alert the rest of your staff, as well, making it           easier for them to avoid becoming victims.
• Are People Aware of New Types of Attacks? Phishing scams are constantly evolving. While they will always rely heavily on social proof, that doesn’t mean    your people – even the well-trained ones – will immediately recognize future       versions. Your analytics should take this into consideration. This can be done by       measuring participation rates in training, for example.

Customize these three metrics as you see fit and, add your own if necessary. However, make sure that you then move on to the next step, which is collecting data about these metrics.

The first one is relatively easy. Over time, you should notice that less and less phishing attacks are succeeding. Using simulated attacks will make this much easier to track as, sadly, you could be victimized by actual phishing scams and not know it for some time.

Simulated attacks will also be a great way to measure whether or not people are reporting them when they happen. This can be a lot harder to track with real attempts because you only know when people are reporting them and not how many times they’re simply ignoring these attempts.

As we mentioned above, one way to collect data about your employees’ abilities to recognize trends in phishing is through training. How often do you hold training? Are all your employees attending? Most importantly, does an increase in this training correlate with improved scores for the other two metrics?

This is really what metric analysis comes down to: is the rate successful phishing attacks (even simulated ones) trending down while the rate of reporting these attempts goes up?

Sadly, we’re probably never going to live in a world without phishing. However, if you use analytics correctly, you can live in one where your company isn’t victimized.

Case Studies and Statistical Data

We started out with statistics at the beginning, but we want to mention one more here before we look at examples of how to undertake phishing awareness training.

97% of people can’t identify phishing emails.

Granted, this takes into consideration a whole spectrum of people and the percentage is probably much lower amongst your employees, but this should still have you thinking about the importance of measuring the effectiveness of your training and looking for ways to develop it further.

With that in mind, let’s look at two common “case studies” you could use to educate your employees about possible phishing scams they may someday receive in their inbox.

One of the first ever phishing scams to be reported was back in 2004. Interestingly, many of the tactics haven’t changed much in the decade-plus since this incident. That just goes to show you how many people can successfully launch phishing attacks if they want.

The Boyles owned a window treatment company and received an email claiming to be from their bank. It asked them to update their information right away. Ironically, the reason given for this request was because of new anti-fraud measures.

Later, after this initial scam was effective in stealing their money, Mrs. Boyle received another email. This time it claimed to be from eBay and asked for more sensitive information because of potentially fraudulent activity the alleged company detected on their account. More money was stolen.

This is a good case study because, again, it shows how simple these tactics are, yet how devastating their results can be.

If you want to see how effective your phishing awareness training is, use real incidents like this and ask your employees to point out all the signs of a phishing attack.

In this case, it would be the urgency used to ensure the victims don’t have time to think about what they’re doing. The links provided in the emails were to fraudulent sites, as well, that wouldn’t have matched the company the message was supposed to be from.

The language in the email may have been broken English, too. Phishing is often undertaken from all over the planet, including many places where English isn’t a first language. Obviously, a bank or major corporation like eBay would have impeccable English in their emails.

Sophie Curtis has a great phishing story. She’s a reporter with the Daily Telegraph who volunteered to let hackers try take over her system.

As Curtis is an expert on the subject of cyber security, she was fairly confident the hackers wouldn’t be successful.

Furthermore, she had taken smart precautions to keep herself safe. For example, her Facebook account was locked to outsiders.

It didn’t matter.

The hackers used everything from family tree sites to figure out her birth date to obscure photos on her Twitter, which showed them seemingly unimportant information about the phone she used, the cigarettes her fiancé smoked and more.

While she ignored their first attempt at a phishing scam (they pretended to be an accountant who wanted to talk to her about a story), the second one landed. It was made to look like a LinkedIn email from someone who wanted to connect. Without thinking, she accepted.

This spelled the beginning of the end. The phishing scam let them know important information about her network (yes, just by clicking on the false LinkedIn button to connect).

They then emailed her again. This time they appealed to the journalist in her. Instead of using the information they had learned about her private life, they simply informed her of a big scoop they had and that part of it was in an attached document. She took the bait and her attackers were in.

Don’t let your company suffer because an employee fell for one of these malicious messages. Instead, invest in proper phishing awareness training and then test to ensure it’s effective.

Sources:

https://phishme.com/use-metrics-measure-improve-effectiveness-security-awareness/

https://securingthehuman.sans.org/resources/metrics

https://www.infosecurity-magazine.com/blogs/effective-phishing-assessment/

https://www.secureworldexpo.com/industry-news/how-do-you-measure-the-effectiveness-of-your-security-awareness-training

https://www.researchgate.net/post/How_can_I_measure_information_security_awareness

https://blog.barkly.com/phishing-statistics-2016

http://resources.infosecinstitute.com/category/enterprise/phishing/the-phishing-landscape/phishing-data-attack-statistics/

https://blog.returnpath.com/13-spear-phishing-stats-to-build-your-case-for-email-fraud-protection/

http://www.trustedia.com/news/how-to-run-effective-phishing-assessment-and-training-campaigns-emplyees-dont-hate

http://www.nbcnews.com/id/46789454/ns/technology_and_science-security/t/phishing-gang-steals-victims-life-savings-m/

Be Safe

Section Guide

Ryan
Fahey

View more articles from Ryan

Infosec IQ awareness and training empowers your employees with the knowledge and skills to stay cybersecure at work and at home. Teach employees to outsmart cyberthreats with over 2,000 awareness resources and phishing simulations.

Request Demo