21 cybersecurity products to combat APT29: MITRE weighs in
Introduction
MITRE, a not-for-profit organization based in the US, is best known for its globally accessible knowledge base of cyber adversary strategies and techniques popularly referred to as the ATT&CK frame. Recently, the organization conducted an independent set of evaluations on 21 cybersecurity products to help the industry and government make well-informed decisions in the battle against cyber threats. The results of the evaluations have now been released to the general public.
Assessing cybersecurity products with APT29 simulations
Using its ATT&CK framework, MITRE chose to emulate the methods and tactics of APT29, a hacker group that many industry analysts believe works on the behalf of the Russian government. The evaluations involved 58 adversary techniques in several kill chain categories. Unlike past evaluations where product capabilities are assigned a certain score, these focused on highlighting how detections take place.
The evaluations, which were sponsored by cybersecurity companies, comprised products from:
- BlackBerry Cylance
- Bitdefender
- Broadcom (Symantec)
- CyCraft
- Cybereason
- CrowdStrike
- Elastic (Endgame)
- FireEye
- F-Secure
- GoSecure
- Kaspersky
- HanSight
- McAfee
- Microsoft
- Malwarebytes
- Secureworks
- SentinelOne
- ReaQta
- Palo Alto Networks
- VMware (Carbon Black)
- Trend Micro
Although the focus of this evaluation was endpoint detection and response (EDR), MITRE simulated APT29 end-to-end and across various attack vectors, allowing cybersecurity companies to benefit from visibility beyond endpoint security.
One of the main reasons behind MITRE’s selection of APT29 is that it offered the opportunity to evaluate the cybersecurity products from different vendors against an adversary that utilizes sophisticated tactics through custom-built malware and alternative executions techniques, like WMI and PowerShell.
How was APT29 emulated?
MITRE shared two scenarios that emulate the ATP29’s publicly reported operational flows.
The first scenario involves the execution of a payload delivered by a spearphishing campaign, which is followed by the gathering and exfiltration of certain file types. Then, after the initial data theft, the adversary realizes the value of the target and deploys an additional, stealthier toolkit to attack the target network. Meterpeter and Pupy are some of the tools used to execute the initial payload.
In the second scenario, the adversary performs a highly methodical and targeted breach. It starts with the injection of a uniquely designed payload that scrutinizes the target environment before its execution. The attack then continues with the gradual takeover of the first target and then the whole domain.
MITRE stated that both scenarios involved the execution of established persistence mechanisms from the past after a “simulated time lapse” to extend the scope of the attack. Also, they were divided into six stages (Compromise, Collection & Evasion, Reconnaissance, Expand Access, Exfiltration & Cleanup) that were completed using the selected TTP (Tactics, Techniques and Procedures) mapped in the ATT&CK framework.
Outcome of MITRE ATT&CK Evaluations
MITRE’s ATT&CK Evaluations methodology and results are available online. Users visiting the ATT&CK evaluations site will find a tool that allows them to choose cybersecurity companies and view a side-by-side comparison of how their products identified each attack method. In addition, the website features a data analysis tool that users can leverage to evaluate how the 21 cybersecurity products handled those methods.
The presentation of the results was also different from how MITRE displayed previously evaluated cybersecurity products from SentinelOne, CrowdStrike and other security vendors against the threats posed by the Chinese hacker group APT3.
In addition to the evaluation results, MITRE has also introduced a DIY APT29 evaluation that enables users to assess cybersecurity products in their own environments against the same threat actors through CALDERA. For those unfamiliar with it, CALDERA is an automated Red Team system built by MITRE using the ATT&CK knowledge base. This can be especially useful for companies who don’t have the budget to hire a Red Team.
Conclusion
After MITRE publicly announced the ATT&CK Evaluations results, almost every cybersecurity company narrated its own interpretation of the outcome and how it excelled in the tests. While this is a standard practice in a highly competitive market like EDR, it is also aligned with the fact that 21 tools trigger slightly different detections when put against APT29. That being said, the evaluations are based on real TTPs, which are becoming critical for mapping detections and observations in MITRE ATT&CK.
Security-focused organizations are highly recommended to review these evaluations. Doing so will enable users to access how successful certain cybersecurity products might be against sophisticated attack vectors throughout their life cycle. Rather than reporting failures to identify specific activities, reporting from Red Teams and pentests can deliver better context to apply the activities directly to integrated cybersecurity tools.
Sources
- APT29 Emulation, MITRE
- ATT&CK Evaluations: Understanding the Newly Released APT29 Results, Frank Duff (Medium)
- Kaspersky Announces Results of MITRE ATT&CK Evaluation, Toolbox