Results Overview: 2022 MITRE ATT&CK Evaluation – Wizard Spider and Sandworm Edition

Threat actor groups like Wizard Spider and Sandworm have been
wreaking havoc over the past few years – developing and deploying
cybercrime tools like Conti, Trickbot, and Ryuk ransomware. Most
recently, Sandworm (suspected to be a Russian cyber-military unit)
unleashed cyberattacks against Ukranian infrastructure targets.

To ensure cybersecurity providers are battle ready, MITRE
Engenuity uses real-world attack scenarios and tactics implemented
by threat groups to test security vendors’ capabilities to protect
against threats – the MITRE ATT&CK Evaluation. Each vendor’s
detections and capabilities are assessed within the context of the
MITRE ATT&CK
Framework.^[1]

This year, they used the tactics seen in Wizard Spider and
Sandworm’s during their evaluation simulations. And MITRE Engenuity
didn’t go easy on these participating vendors. As mentioned before
– the stakes are too high, and risk is growing.

The 2022 results overview

To think about it simply, this MITRE ATT&CK Evaluation
measured protection capabilities of 30 endpoint protection
solutions. Two key measurements that are generated from the testing
are Overall Detection and Overall Protection.

As one participating vendor, Cynet, explained in a blog post reviewing the
results^[2], “Overall Detection
(What MITRE refer to as “Visibility”) is the total number of attack
steps detected across all 109 sub-steps. Overall Prevention (What
MITRE refer to as “Protection”) measures how early in the attack
sequence the threat was detected so that subsequent steps could not
execute. Both are important measurements and are indicative of a
strong endpoint detection solution.”

The graph below shows the 2022 participating vendors’ overall
detection and protection performance:

And here are the results in the form of a summary table:

How it works

MITRE ATT&CK uses a unique approach, testing 30 security
vendors this year for their ability to protect against attacks that
are currently happening in the wild. They do it by putting these
vendors through a simulation in a controlled environment, creating
an unbiased assessment of each vendor’s platform and capabilities
to detect and respond to threats.

The results of these evaluations are released at the end of
every March and are intended to be used by security teams looking
to bolster their security program, which often entails identifying
a cybersecurity provider. The MITRE ATT&CK Evaluation tests
specific capabilities using a public-facing methodology and
provides an objective assessment without ranking each vendor’s
performance.

The interpretation of the results and determining which vendor
performed the best is up to the reader to decide. And that’s where
things get tricky.

The MITRE ATT&CK Evaluation results are meant to be a
helpful resource, and it behooves security leaders and executives
to learn how to leverage these results. The difficulty is
understanding what these results mean within the context of other
vendors’ performance.

The 2022 MITRE ATT&CK Evaluation Results
Webinar

As many security experts will tell you, interpreting this data
is easier said than done. Cynet, one of the vendors that
participated in this year’s evaluation aims to bring some clarity
to the confusion. The goal is to help organizations looking for a
security provider use these results to assess which participating
vendor’s capabilities best align with their needs.

Cynet’s CTO, Aviad Hasnis, will host this webinar series,
starting on April 7, 2022. He’ll explain how you can use the MITRE
ATT&CK Evaluation results as a tool in your search for a
security vendor in addition to sharing details specific to Cynet’s
performance. Find out more and sign up here.^[3]