There is no shortage of indicator data these days.  Large numbers of providers, both commercial and free, have set up shop to help fight the cause.  The problem is, a lot of it has a shelf life.  Web sites come and go, IP addresses get re-assigned, compromised hosts get fixed, and so on.  If this data sits forever in your security infrastructure, your analysts will be swamped with alerts that are potentially no longer timely, accurate, or relevant.

The ThreatQ threat intelligence platform, in version 2.2, provides a robust and deeply customizable indicator expiration system.  Rather than making broad generalizations about when an indicator should be expired (e.g expire all indicators 60 days after creation), the ThreatQ platform enables you to decide on expiration per source and Indicator type.

The platform also understands how to deal with multiple sources when it comes to an indicator and allows them to work together to make a more dynamic expiration date.  All sources should be able to participate in deciding when that indicator is expired.  Of course, an analyst can manually override an indicator’s expiration date or prevent it from ever being expired.

Once an indicator is expired, its status is changed from “active” to “expired”.  This will result in the indicator being removed from your security infrastructure provided the device supports removal.  This will keep the ThreatQ threat intelligence platform in-sync with your infrastructure with minimal effort on the part of your analysts.  That leaves them with more time to investigate the actual alerts instead of chasing down an old IP address that is no longer relevant.

In this post we will go through some scenarios of setting up expiration policies.

Simple

Once a feed is enabled, it will appear in the “Indicator Management” page.  Here is where you can set its expiration policy.  As you can see below, I enabled the “Emerging Threats” Compromised IP’s feed.  I chose the “Automatically Expire Indicators” option, which defaults to 14 days.  This is configurable, but if you want to go with the default all you have to do is hit save.  Any new indicators from this source will expire in 2 weeks.

Now that we have some indicators from this source, we can check out one of the IP addresses’ details page and see what it looks like.

On the information bar under the header, you can see it is set to expire on 2/25/2017.  Notice the extend link, using this will allow you to:

• bump the expiration date for this indicator
• remove the expiration date
• set this indicator to never be expired

What happens if this feed reports this indicator again?  It will extend the expiration date, but it will never extend beyond the maximum date specified of 14 days.

Short Fuse Indicators

Some indicators have a very short shelf life.  An example of these would be Fast Flux/DGA domain names.  Some malware families will generate a set of domain names each day to use for command and control.  The malware controller also knows which set will be generated on each day and can pick one of the domain names to use.  The domain names are only good for that day and there could be thousands of possibilities.

We want our security infrastructure to look for connections to those domains, but only for a day or two.  After that, those domain names are irrelevant (unless there is some time skew) so we don’t need them deployed anymore.  Expiration is great for this situation.  Bambenek Consulting provides a number of free feeds which generate these DGA domain names for given malware families.  As you can see below, we added it to ThreatQ and gave it a short shelf life of 2 days.  This will keep our security infrastructure from being swamped with DGA domain names.

Customize

Each security infrastructure is different, as are the opinions of analysts when it comes to which kind of indicator they want to keep longer.  The ThreatQ threat intelligence platform allows you to set an expiration policy per indicator type, per feed.  Let’s take iSight as an example, as they provide all sorts of indicator data.

With the above policy, we are telling ThreatQ to expire iSight indicators in 14 days, except for what is listed below.  MD5 hashes are valid for longer than a URL, for example.  We can expire those indicators in 60 days, while expiring the URLs from iSight in 20 days.  This allows for a lot of flexibility as it can be done for every feed.  An organization can make an expiration policy that suits their needs.

Teamwork

Another important aspect of ThreatQ expiration is how it works when multiple feeds report about the same indicator.  One of the core design decisions behind expiration was that all feeds should participate in deciding when the indicator will expire.  The algorithm takes it into account by adjusting the expiration date based on all the sources reporting it while still preventing runaway dates.  For example,  if an OSINT feed reports an indicator and has it set to expire in 7 days, then 5 days in CrowdStrike reports it, the expiration date will be extended based on the combined policies of the two feeds.

Changing the expiration policy is still all up to the organization, of course.  Based on the policies set, certain feeds can have a much longer expiration policy than others.