Sunday, December 4, 2016

Red vs. Blue: Monitoring User-Agent Strings

The Purpose

Security teams can only go so far when it comes to trying to keep control over what browsers or applications tenants on their networks are using. Often, there are attempts to limit users' rights to install software. This typically works pretty well with standard users, but what about the administrators? Sure, lots of larger organizations have host-based solutions to prevent even administrators from modifying their systems in efforts to bypass corporate policy, but what about those organizations that do not yet have this implemented due to cost or expertise? Below I will go back and forth between different "Red" tactics and the corresponding "Blue" response to try to capture those abusers of our policy (or, better yet, detect compromised systems).

The Environment

All of the following machines will be running in Virtualbox on a Windows 10 host machine. I will be using the same Security Onion virtual machine as my first post and a Windows 7 virtual machine acting as our malicious insider. Additionally, I am setting up a web server the easiest way I can think of: a Xubuntu Linux client that will be serving a "web page" via the following command:

sudo python -m SimpleHTTPServer 80
  • Run as an administrator
  • Execute Python
  • Import module, SimpleHTTPServer
  • Listen on port 80 (thus, requiring administrative rights)

Note: Most (if not all) popular Linux distributions come with Python and this corresponding SimpleHTTPServer module. By default, the above command can be run as a standard user if attempting to listen on an ephemeral port (TCP port greater than 1023).

What will be looked at?

Most of this article will be focused on user agent strings. User agent strings are simply what a browser or application uses to identify itself and the host to the web server. Based upon the user agent string, a web server may decide the content or format that is displayed to the browser. For example, this is how web servers know when to use the mobile version of their site. Below is an example of the user agent string that I am using on my host machine that is typing this blog post:

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36

Looks to be pretty much as I expected... a 64-bit Windows 10 host running Google Chrome 54.0. The rest of fields are simply browser information and capabilities. This link has a funny read if you're interested in understanding the different user agent string verbiage.

Why look at user agent strings? 

It's a simple indication that, unless the malicious actor is a little sophisticated, the user agent will differ from what should be seen on the network if they would be using an unapproved browser or application to access a web server. I'll cover a couple of these "more sophisticated" means that could be used a little later.

The "Corporate Baseline"

Let's assume that this sample corporation in the following scenarios has a policy that allows only for those applications in the baseline to be used on the corporate machines. The corporation utilizes Linux servers and Windows 7 workstations. As a best practice and part of the baseline, the only machines that can access the internet are the Windows clients and they must use Internet Explorer 11. I'll admit... a lame policy, but it's what I'm going with to keep it simple.

First, the Security Onion virtual machine will use Wireshark to listen to the network and capture some "internet" traffic from the sample Windows client to the Xubuntu server. I'll accomplish this by:
  • Security Onion VM: open wireshark from the command line and listen to my sniffing port (in my case, eth1)
  • Windows 7 VM: Open IE 11 and navigate to the Xubuntu "web server" (in my case,
  • Security Onion VM: Stop the Wireshark capture
Now I have what a "normal" user agent string would look like, but I do have a lot of other data. A good way to find user agent strings in Wireshark would be to look at GET requests by using the filter, frame contains "GET".

That's the GET request from the client ( to the server ( To get what the actual user agent string is, I have to drill into at the HTTP header in the Packet Details pane:

This is the user agent string that the Windows 7 machine is sending (you'll notice Windows NT 6.1... Microsoft had made numbering a bit confusing until recently as seen here) running Internet Explorer 11.0.

Blue Team Response: Snort

Now that I know what "normal" looks like, a simple Snort rule can take it from here and show what doesn't match the approved user agent string. This actually took a bit of Googling as 99% of the time when developing a Snort rule, you're looking for a positive match, not a negative one. After a bit of trial and error, I added this rule to /etc/nsm/rules/local.rules on the Security Onion VM:

alert tcp $HOME_NET any -> any $HTTP_PORTS (msg: "Unapproved User-Agent"; content: "User-Agent"; content: !"Mozilla/5.0 (Windows NT 6.1\; WOW64\; Trident/7.0\;  rv\:11.0) like Gecko"; sid: 4000005; rev:1;)
  • Send an alert (which will be viewed in Sguil)
  • Only look at the home network headed to a web server on HTTP ports
  • Message to be sent to the analyst
  • Look for packets that contain the string, "User-Agent"
  • Highlight any of the filtered packets that do not have the approved user agent string (Note the "\" escapes that are required if the string contains a ";" or ":")
  • Snort ID and revision number
After this update, I'm going to run sudo rule-update to implement the new rule.

Monitor with Sguil

Now to test. I'm going to launch Sguil on the Security Onion VM and rerun the baseline test above with the Windows 7 and Xubuntu client. If an alert comes up, the alert was likely typed wrong as it should not fire if the user agent string matches.

The next test for this rule is to install an alternate browser and rerun the test again. Hopefully, I'll see an alert (which should contain the user agent string of the misconfigured system). Here's what was found in Sguil after installing Google Chrome on the Windows VM and connecting to the Xubuntu VM again:

Success! It seems that the alert caught a Windows 7 machine running Google Chrome 55.0. So, this should be fine, right?

Red Team Tactic: Downloading with Powershell

The above rule would certainly catch administrators that install unapproved web browsers and simply start using them, but there's another scenario that may prove a bit more malicious and would completely evade this rule.

In this scenario, there is a malicious insider that wants to download unapproved files, but does not want to have a trail of Internet Explorer browsing logs. Before actually running the command below, though, I will bring up Wireshark again on the Security Onion VM to watch this traffic. On the Xubuntu "web server", I'm hosting a file named FreeMusicPlayer.exe. Let's see if our rule catches it if the insider uses the following Powershell command to download the file:

(New-Object System.Net.WebClient).DownloadFile("
  • Create a new Web Client
  • Download a file from the Xubuntu host and name it "FreeMusicPlayer.exe" inside the current directory
After the Powershell command is executed, I would expect to have my alert count increase to 3 (it was 2... and it still is 2). So what's going on? Let's take a look at the new "GET" request in Wireshark:

That's strange... no User-Agent field. It seems that Powershell, by default, does not send a user agent string when downloading a file! This tells me that a user agent string is not required and, after digging into RFC 2616, there's a lot of SHOULDs but no REQUIREDs in regards to user agent strings. So that means, our pre-requisite that we look for "User-Agent" before doing the check against our whitelisted string is not met. Also by default on the system, there are no Powershell command logs to be found when digging through Event Viewer. FireEye has a great article on Powershell logging that should be implemented in any Windows shop to really see what Powershell users are really doing.

Blue Team Response: Another Snort Rule

This is a rather simple fix. Since this was a "GET" request, a second rule can be made to catch users that are trying to download via Powershell. Sure, the first rule could be modified to key on "GET /" instead of "User-Agent", but I would have to generalize my message to an analyst... something like "Unapproved download method". I like to stay a little more descriptive, so that's why I'm creating a second rule named "GET Request without User-Agent String Detected". That's nice and descriptive. I've also realized this time around, that everything being looked for is in the HTTP header -- not the data, so both the previous and new alerts can be made more efficient by adding "http_header". The new alert looks like this:

alert tcp $HOME_NET any -> any $HTTP_PORTS (msg: "GET Request without User-Agent String Detected"content: "GET /"content: !"User-Agent"http_headersid: 4000006; rev:1;)
  • New message to be sent to the analyst
  • Look for packets that contain the string, "GET /"
  • Highlight any of the filtered packets that do not have a user agent string
  • Restrict the search to the HTTP header fields
  • Snort ID and revision number
Now, I'm going to re-run sudo rule-update and try the Powershell download again...

Voila! We caught our guy/gal!

Red Team Tactic: Modified User Agent String

We're not out of the woods yet. Powershell, wget, curl, and even web browsers offer the ability to change the default user agent strings. I will be running Wireshark again to see the frame and Sguil to check to see if any alerts pop up. Below is an example of Powershell downloading a file after setting a custom user agent string:

$wc = New-Object System.Net.WebClient
$wc.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1\; WOW64\; Trident/7.0\;  rv\:11.0) like Gecko")
$wc.DownloadFile("", "FreeMusicPlayer.exe")
  • Create a new WebClient variable named $wc
  • Add the User-Agent header information
  • Download the file using the WebClient

Looking back at Sguil I see... nothing new! What was seen in Wireshark?

Looks like IE 11.0 to me... and to Snort.

Red Team Tactic: Spoofing the Web Browser

This is actually pretty easy to do in Chrome as well using Developer tools (no third-party plugins needed):

  • Open Chrome to a blank page so no GETs are sent
  • Open Developer Tools (Ctrl+Shift+I in Windows)
  • Click on the three vertical dots >> More tools >> Network conditions
  • Here, you can set custom and default  user agent strings
  • Navigate to desired site and appear as IE 11.0

Running the test again, no new alerts! Wireshark again verifies that the user agent is successfully spoofed:

Blue Team Response: ?

This is quite the cat and mouse game! There's even several factors that would keep any rule-writing analyst a bit busy which could take this article down many paths:
  • What if the insider is using TCP ports not defined in Snort (right now, I have only 80 and 8080)?
  • If a user is spoofing user agent strings, what are they trying to hide? What should be looked for?
  • What other mechanisms could be implemented on the network to catch some of this?
  • What about potential false positives? Example: Java applications can create their own user agent strings as well
The intent was to get security team members thinking, so please comment below to add your thoughts as I just scratched the surface on this topic.


  1. Fortunately those ADT home security systems solutions are actually simple to use. It gives me using the convenience realizing that she’s safe, although not only does it guard her from potential burglars. I really, truly take pleasure in that! Thanks, ADT!

  2. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free.

  3. Great Article
    Cyber Security Projects

    projects for cse

    Networking Security Projects

    JavaScript Training in Chennai

    JavaScript Training in Chennai

    The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training


Note: Only a member of this blog may post a comment.