The Phish Scale: How NIST is quantifying employee phishing risk
With the relatively recent uptick in phishing around the globe (due in part to Covid-19 and other factors), experts at the National Institute of Standards and Technology (NIST) have been working hard to create a new way to quantify phishing risk for organizational employees. This new way is called the Phish Scale. If phish and scales have you thinking more of the messy work associated with processing fish to eat, this article will give you a better smelling impression of the phonetic term.
What is Phish Scale?
Released by NIST in 2020, Phish Scale is a breath of fresh air in this age of ever-increasing phishing instead of the aquatic stench the name might suggest. Phish Scale was created as a method by which CISOs can quantify the phishing risk of their employees. It quantifies this information by using the metrics of “cues” and “context,” which makes the data generated by training simulations to be more insightful. In essence, it allows organizations to better categorize actual threats (for better detection) and to better determine the effectiveness of their phishing training program.
You may be wondering why this is a significant development — and it is probably more significant than you think for those that see its value in determining program effectiveness. Before Phish Scale, the traditional metrics organization used were click-rate, which is not always reporting rates and reporting times. By adding cues and context to the mix, organizations will have a more accurate view of where they stand regarding phishing detection.
|Number of cues||Context||Detection difficulty|
|Low||Moderately to least difficult|
Above is a visual depiction of the Phish Scale. It uses the metrics of the cues present in the phishing emails and the context of the information contained in the email about the organization which is referred to as premise alignment by NIST (simplicity is king so context it is).
Metrics: Cues and context
Cues refer to the characteristics of the phishing email that may tip off, or cue, the recipient into thinking that the email is legitimate. There are five types of cues to look out for, presented below:
|Type of cue||Cues|
|Error||Spelling and grammar irregularities|
|Technical indicator||Attachment type|
|Sender display name and email address|
|Visual presentation indicator||No/minimal branding and logos|
|Logo imitation or out-of-date branding/logos|
|Unprofessional looking design or formatting|
|Security indicators and icons|
|Language and content||Legal language/copyright info/disclaimers|
|Requests for sensitive information|
|Sense of urgency|
|Lack of signer details|
|Common tactic||Humanitarian appeals|
|Too good to be true offers|
|Limited time offer|
|Mimics a work or business process such as a legitimate email|
|Pose as a friend, colleague, supervisor or authority figure|
Context, or Premise Alignment, is the other Phish Scale metric. There are two methods to categorizing context. The first method uses three rating levels — low, medium and high for how closely the context aligns with the target audience. The second method uses five elements, rated on a five-point scale to measure workplace/premise alignment called the alignment rating. This helps the phishing trainer at the organization score the phishing exercise as being of low, medium or high difficulty based upon the data gathered of the phishing simulation. The Phish Scale implementor can choose either method they like and this article will focus on the five-element method.
The five context elements are:
- Mimics a workplace process or practice
- Has workplace relevance
- Aligns with other situations or events, including external to the workplace
- Engenders concern over consequences for not clicking
- Has been the subject of targeted training, specific warnings or other exposure (not scored)
Only elements 1-4 are added up when scored with the fifth element being subtracted from the score. The five-point scoring system used to rate each element is based upon even numbers of 0-8:
8 = Extreme applicability, alignment or relevancy
6 = Significant applicability, alignment or relevancy
4 = Moderate applicability, alignment or relevancy
2 = Low applicability, alignment or relevancy
0 = Not applicable
NIST tested Phish Scale by using 10 exercises on organizational employees. These exercises were emails that focused on different angles to trick the recipient. Below are the angles used in each exercise:
- E1. Safety requirements email
- E2. Weblogs (unauthorized web site access)
- E3. Unpaid invoice
- E4. Scanned file
- E5. New voicemail
- E6. Valentine “eCard delivery”
- E7. Order confirmation
- E8. Security token
- E9. Gift certificate
- E10. Adobe update
To highlight the disconnect between click-rate percentage and the actual difficulty level of detecting the phishing exercise, let’s take a look at how one exercise rated very difficult with few cues and high premise alignment, scanned file (E4). This exercise was conducted with 62 participants taking part.
|Context element||Alignment rating||Actual click-rate percentage|
|Mimics a workplace process or practice||6|
|Has workplace relevance||4|
|Aligns with other situations or events, including external to the workplace||6|
|Engenders concern over consequences for not clicking||4|
|Has been the subject of targeted training, specific warnings or other exposure||-2|
This phishing email exercise used a message referring to a shared scanning and printing device, a common device in organizational settings. Despite a high level of difficulty based mostly upon a mimicked workplace practice that aligns with workplace situations significantly, there was only a 19% click rate.
Utilizing NIST to categorize phishing threats
NIST has released the Phish Scale method for CISOs (and organizations generally) to better categorize actual threats and to determine if their phishing program is effective. It allows implementers to use other metrics aside from the traditional click-rate percentage to do this, which will positively impact cybersecurity in the face of an increasing number of phishing attempts.
Categorizing human phishing difficulty: a Phish Scale. Oxford Academic Journal of Cybersecurity
4 Things to Know About the NIST Phish Scale, Mindpointgroup.com