Phishing

The Phish Scale: How NIST is quantifying employee phishing risk

May 25, 2021 by Greg Belding

With the relatively recent uptick in phishing around the globe (due in part to Covid-19 and other factors), experts at the National Institute of Standards and Technology (NIST) have been working hard to create a new way to quantify phishing risk for organizational employees. This new way is called the Phish Scale. If phish and scales have you thinking more of the messy work associated with processing fish to eat, this article will give you a better smelling impression of the phonetic term. 

What is Phish Scale?

Released by NIST in 2020, Phish Scale is a breath of fresh air in this age of ever-increasing phishing instead of the aquatic stench the name might suggest. Phish Scale was created as a method by which CISOs can quantify the phishing risk of their employees. It quantifies this information by using the metrics of “cues” and “context,” which makes the data generated by training simulations to be more insightful. In essence, it allows organizations to better categorize actual threats (for better detection) and to better determine the effectiveness of their phishing training program.

You may be wondering why this is a significant development and it is probably more significant than you think for those that see its value in determining program effectiveness. Before Phish Scale, the traditional metrics organization used were click-rate, which is not always reporting rates and reporting times. By adding cues and context to the mix, organizations will have a more accurate view of where they stand regarding phishing detection.

Phish Scale
Number of cues Context Detection difficulty
Few High Very difficult
Medium Very difficult
Low Moderately difficult
Some High Very difficult
Medium Moderately difficult
Low Moderately to least difficult
Many High Moderately difficult
Medium Moderately difficult
Low Least difficult

Above is a visual depiction of the Phish Scale. It uses the metrics of the cues present in the phishing emails and the context of the information contained in the email about the organization which is referred to as premise alignment by NIST (simplicity is king so context it is).

Metrics: Cues and context

Cues refer to the characteristics of the phishing email that may tip off, or cue, the recipient into thinking that the email is legitimate. There are five types of cues to look out for, presented below:

Type of cue Cues
Error Spelling and grammar irregularities
Inconsistency
Technical indicator Attachment type
Sender display name and email address
URL hyperlinking
Domain spoofing
Visual presentation indicator No/minimal branding and logos
Logo imitation or out-of-date branding/logos
Unprofessional looking design or formatting
Security indicators and icons
Language and content Legal language/copyright info/disclaimers
Distracting detail
Requests for sensitive information
Sense of urgency
Threatening language
Generic greeting
Lack of signer details
Common tactic Humanitarian appeals
Too good to be true offers
You’re special
Limited time offer
Mimics a work or business process such as a legitimate email
Pose as a friend, colleague, supervisor or authority figure

Context, or Premise Alignment, is the other Phish Scale metric. There are two methods to categorizing context. The first method uses three rating levels low, medium and high for how closely the context aligns with the target audience. The second method uses five elements, rated on a five-point scale to measure workplace/premise alignment called the alignment rating. This helps the phishing trainer at the organization score the phishing exercise as being of low, medium or high difficulty based upon the data gathered of the phishing simulation. The Phish Scale implementor can choose either method they like and this article will focus on the five-element method.

The five context elements are:

  1. Mimics a workplace process or practice
  2. Has workplace relevance
  3. Aligns with other situations or events, including external to the workplace
  4. Engenders concern over consequences for not clicking
  5. Has been the subject of targeted training, specific warnings or other exposure (not scored)

Only elements 1-4 are added up when scored with the fifth element being subtracted from the score. The five-point scoring system used to rate each element is based upon even numbers of 0-8:

8 = Extreme applicability, alignment or relevancy

6 = Significant applicability, alignment or relevancy

4 = Moderate applicability, alignment or relevancy

2 = Low applicability, alignment or relevancy

0 = Not applicable

Application

NIST tested Phish Scale by using 10 exercises on organizational employees. These exercises were emails that focused on different angles to trick the recipient. Below are the angles used in each exercise:

  • E1. Safety requirements email
  • E2. Weblogs (unauthorized web site access)
  • E3. Unpaid invoice
  • E4. Scanned file
  • E5. New voicemail
  • E6. Valentine “eCard delivery”
  • E7. Order confirmation
  • E8. Security token
  • E9. Gift certificate
  • E10. Adobe update

To highlight the disconnect between click-rate percentage and the actual difficulty level of detecting the phishing exercise, let’s take a look at how one exercise rated very difficult with few cues and high premise alignment, scanned file (E4). This exercise was conducted with 62 participants taking part.

Context element Alignment rating Actual click-rate percentage
Mimics a workplace process or practice 6  
Has workplace relevance 4  
Aligns with other situations or events, including external to the workplace 6  
Engenders concern over consequences for not clicking 4  
Has been the subject of targeted training, specific warnings or other exposure -2  
Total 18 19%

This phishing email exercise used a message referring to a shared scanning and printing device, a common device in organizational settings. Despite a high level of difficulty based mostly upon a mimicked workplace practice that aligns with workplace situations significantly, there was only a 19% click rate.

Utilizing NIST to categorize phishing threats

NIST has released the Phish Scale method for CISOs (and organizations generally) to better categorize actual threats and to determine if their phishing program is effective. It allows implementers to use other metrics aside from the traditional click-rate percentage to do this, which will positively impact cybersecurity in the face of an increasing number of phishing attempts.

 

Sources:

Categorizing human phishing difficulty: a Phish Scale. Oxford Academic Journal of Cybersecurity

4 Things to Know About the NIST Phish Scale, Mindpointgroup.com

The Phish Scale: NIST-Developed Method Helps IT Staff See Why Users Click on Fraudulent Emails. NIST

Posted: May 25, 2021
Articles Author
Greg Belding
View Profile

Greg is a Veteran IT Professional working in the Healthcare field. He enjoys Information Security, creating Information Defensive Strategy, and writing – both as a Cybersecurity Blogger as well as for fun.