Machine Learning for Email Threat Protection: What It Is. How It Works.

Machine Learning and Artificial Intelligence are Everywhere

If you attended the RSA conference, or any other recent security conference, you’ve probably noticed that many vendors pitch machine learning and artificial intelligence as the latest and greatest ways to defend against cyber threats. Unfortunately, it’s rarely explained how these concepts work to actually help keep you safe. Understanding these terms is an important part of determining how they should fit into your overall email threat protection strategy.

AI vs Machine Learning

 Machine learning is the ability to teach a machine how to learn from experience, something that comes naturally to humans. A child that sees a cat for the first time and then sees a dog, may confuse the two. But after the child sees several cats, they learn to distinguish cats from dogs. In a similar way, machine learning systems improve the accuracy of their pattern identification over time, without human intervention. These systems are built to focus on and improve at a specific task. Let’s take, for example, the identification of cat pictures.  The more images of cats we use to train the machine, the more accurately it will be able to identify future images of cats and distinguish them from images of dogs.

Artificial intelligence, on the other hand, is the ability to teach a machine to mimic human behavior or intelligence. This requires the machine to perform multiple tasks and adapt to its surroundings while performing its tasks. For example, with AI we can build a robot that takes care of household chores, like walking your dog.  To accomplish this task, the robot would use machine learning to identify your dog, your house, the sidewalk, other dogs and potential street traffic, and then adapt based on what is going on around it. This means machine learning is a component of an AI system but AI is a broad field of technology.

Although machine learning and artificial intelligence are often used interchangeably, most threat detection solutions rely primarily on machine learning.  So how are the two types of machine learning, supervised learning and unsupervised learning, used to detect email threats?

Supervised Machine Learning for Threat Detection

With supervised machine learning, a system is trained with predefined data. The system uses this data to create a predictive model that can in turn be used to identify future data. To return to our previous example, supervised learning can be used to identify cat pictures by feeding the system with cat images and labeling them as cats. The system analyzes image attributes and builds a predictive model that allows it to look at future images and assign a probability as to whether the image is a cat or not.  With enough data (images of cats), the machine will learn to accurately identify images that contain cats and images that do not contain cats, for example an image of a dog.

In the world of email threat protection, supervised machine learning is used to detect malicious or unwanted emails. The protection software is trained using known bad emails. A predictive model is then built by looking at all available email attributes (body, attachments, HTML and MIME) of the malicious or spam messages. The more example emails used to train the system, the more accurately it’s able to predict whether a new message is malicious or spam. To ensure accuracy over time, the system is re-trained with new email samples as threats evolve.

Unsupervised Machine Learning for Threat Detection

 With unsupervised learning the machine is trained using data that has not be predefined. The system analyzes the data and groups or categorizes it.  Using our previous example of cat images, the system would be given various images of cats without being told what the images contain. The system would identify and use similarities in these cat images and group them together. If one of the images was a dog, it would determine that the image was not in the same category as the cats. This form of machine learning is very useful in detecting data anomalies.

While unsupervised machine learning is not as commonly used in threat protection solutions as supervised machine learning, there are systems that use it to detect email attacks.  In this scenario the system is fed regular email communication data. Over time this system can learn what normal email communication looks like for the organization and its users. This means it can detect when an email purportedly from the CEO is actually from an email address that has never been used by the CEO, or when the content doesn’t match the CEO’s normal writing style. This means that unsupervised machine learning can be particularly helpful in identifying business email compromise or other impersonation attacks.

Machine Learning as Part of a Layered Defense Strategy

 Prior to machine learning, email threat protection solutions depended primarily on signatures of known spam and malware to block the unwanted emails – we could tell the system that a specific email is known to be bad, so if you see this email again, block it. This technique was, and still is, important in preventing known threats. But hackers have figured out that small modifications to their attack campaigns can get through signature based systems.

Over time, other methods of identifying malicious emails have also been developed, such as blacklisting URLs and IPs, attachment sandboxing and DMARC sender authentication. All of these methods are useful in preventing some attacks but the attacks keep evolving. Machine learning now adds another layer of defense to protect against advanced threats. The ability to assign a threat probability to an email, in combination with information from other layers of defense, provides the best possible level of protection. Additionally, the ability to automate threat identification and feed that data into a SIEM platform, allows limited human IT resources to focus on identifying threats that may come in through email but then permeate other systems across the organization.

While there is no silver bullet that can stop all threats, using machine learning as part of a layered defense can reduce the number of attacks that get through, and help identify when they do. This combined, layered approach represents the best available strategy to keep your organization safe.