Wednesday, November 23, 2016

Blue Team: Conversations with Multiple Tools


There are often times during network analysis where we would really love to get a good sense of communications between endpoints. Sometimes we don't have all the fancy tools in our environment, so I'll show a few ways to accomplish what would seem to be a simple task.

The Environment

This one's easy to replicate on your own as I am simply using a Security Onion 14.04 virtual machine in Virtualbox and the included packet capture located at /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap.

The virtual machine setup is rather simple as well. I allocated just 2GB RAM, 2 processors, and two virtual NICs (one for management and one for sniffing). I followed the Security Onion wiki to set up a production, standalone instance using best practices (a lot of next -> next -> next...).

Viewing "2014-12-05-phishing-email-traffic.pcap" in Wireshark

Wireshark does give us the capability to view conversations sorted by relative time (default).

As shown below, it does not allow us to view all TCP and UDP conversations together in an attempt to write the story, so we would need to bounce back and forth between tabs. 

Also, once we get into a mass amount of traffic, we may find ourselves constantly adjusting a display filter and then selecting "Limit to display filter" which, if we have a large number of packets in our capture, adds time.

What about tshark?

tshark does make this process a little bit faster, but has the same limitations as Wireshark... and more. First, when viewing conversations in tshark, you must select which type of traffic you want to see. Below are the available options (pulled from man tshark):

"eth"   Ethernet
"fc"    Fibre Channel
"fddi"  FDDI
"ip"    IP addresses
"ipx"   IPX addresses
"tcp"   TCP/IP socket pairs  Both IPv4 and IPv6 are supported
"tr"    Token Ring
"udp"   UDP/IP socket pairs  Both IPv4 and IPv6 are supported

Again, we are limited by taking multiple steps to see all conversation data that we're interested in. We have to do significant command-line Kung Fu with"awk" and "sort" to put this into a timeline.

Let's say we just want to see TCP and UDP communications from our packet capture:

First, we must capture all of the TCP conversations...

tshark -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap -z conv,tcp -q -n | awk '!/[=A-Za-z]/ {print $10,"\011","TCP","\011", $1,"\011",$2,"\011",$3,"\011",$9}' > tsharkout.txt
  • Read in packet capture
  • Show TCP conversations
  • Do not show packet details before conversation list or do name resolution
  • Remove lines with equals signs or alpha characters, then print the relative timestamp, TCP, source IP/port, destination IP/port, and total bytes transmitted
  • Write to a temporary file
Then, we must capture all UDP conversations...

tshark -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap -z conv,udp-q -n | awk '!/[=A-Za-z]/ {print $10,"\011","UDP","\011", $1,"\011",$2,"\011",$3,"\011",$9}' >> tsharkout.txt
  • Read in packet capture
  • Show UDP conversations
  • Do not show packet details before conversation list or do name resolution
  • Remove lines with equals signs or alpha characters, then print the relative timestamp, UDP, Source IP/port, Destination IP/port, and total bytes transmitted
  • Append to our temporary file
Now, we have all of our data in a file, but it's severely out of order. tshark sorts conversations by highest number of frames and is not customizable. On top of that, all of our TCP conversations were written prior to the UDP conversations. The next step is rather simple to fix this...

cat tsharkout.txt | sort -n > conv-sorted.txt
  • Display our temporary file
  • Sort the data in numeric order
  • Write to our final conversation list

Whew! That was a lot of typing. What if we want to filter our data on an IP of interest (example: We'd have to edit both tshark commands by adding | grep "" just before both file redirections (> and >>) and redoing our sort:

tshark -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap -z conv,tcp -q -n | awk '!/[=A-Za-z]/ {print $10,"\011","TCP","\011", $1,"\011",$2,"\011",$3,"\011",$9}' | grep "" > tsharkout.txt

tshark -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap -z conv,udp-q -n | awk '!/[=A-Za-z]/ {print $10,"\011","UDP","\011", $1,"\011",$2,"\011",$3,"\011",$9}' | grep "" >> tsharkout.txt

cat tsharkout.txt | sort -n > conv-sorted-177-124-228-4.txt

    Enter: Argus

    Of course, all of this could be scripted, but Argus, by default, does allow for all of this as well as easily add searching (via egrep) just by adjusting our commands. Here's our example, again, using Argus:

    First, convert the pcap to something the ra command can work with...

    argus -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap 
    -w out.argus
    • Read in the packet capture
    • Write to an .argus file
    Next, we'll use the ra command to get the output that we're looking for. In our case, we want the same fields that tshark gave us...

    ra -n -r out.argus -s stime proto saddr sport daddr dport bytes > argus-conv.txt
    • Do not resolve names and read in our .argus file
    • Choose the relative start time, protocol (this would even include non-TCP and non-UDP data), source IP/port, destination IP/port, and total bytes transferred
    • Write to argus-conv.txt

    That was much easier and highly customizable if you want to see more/less fields and look, it's default sorted by relative time! The available fields are listed in the argus man page (man argus) under the -s flag (WAY too many to list here).

    Just like our Wireshark example, I'll show how simple it is to filter by an IP of interest. We can leave our argus command alone and just modify the ra command by adding an egrep command.

    ra -n -r out.argus -s stime proto saddr sport daddr dport bytes | egrep "(Start|" > argus-conv.txt

    The egrep command is simply displaying all lines that match the string "Start" (so we get our top line) or the IP of interest. This can easily be adjust by looking for other strings (port numbers, for example).

    Last, but not least... Bro

    Bro is just as useful and also requires two separate commands to get the job done. It's also very easy to use:

    First, let's run the packet capture with bro (which writes out many log files in our current directory)...

    sudo bro -r /opt/samples/mta/2014-12-05-phishing-email-traffic.pcap
    • Run as root (it'll still work without, but you get some errors)
    • Read in the packet capture file
    Now we should have quite a few .log files in our current directory. The one we're interested in is conn.log. Next, we'll run bro-cut to pull the same data as the last examples...

    cat conn.log | bro-cut -d ts proto id.orig_h id.orig_p id.resp_h id.resp_p orig_bytes resp_bytes > bro-conv.txt
    • conn.log will be the input to bro-cut
    • Display time as human-readable (otherwise it's epoch time)
    • Show us the timestamp, protocol used, source IP/port, destination IP/port, and bytes sent from our source and destination
    • Write results to bro-conv.txt

    Just like with Argus, our results are ordered by time by default. If you want to know what other columns bro-cut can handle, simply look at the beginning of conn.log. You can also conduct bro-cut against other logs, but their columns may be different, so looking at the top of the log, again, will help you out with using bro-cut.

    Also like Argus, filtering by an IP is very easy. We'll just rerun our bro-cut command with grep tacked on before the file write...

    cat conn.log | bro-cut -d ts proto id.orig_h id.orig_p id.resp_h id.resp_p orig_bytes resp_bytes | grep "" > bro-conv.txt

    Wrap up

    I'm sure there's a ton of other tools that are out there in the wild, but, like I said earlier, sometimes you're limited by what you have available (in this case, Security Onion). Feel free to leave any comments if you have any better approaches to tracking conversations in a packet capture.

    1 comment:

    1. Great Article
      Cyber Security Projects

      projects for cse

      Networking Security Projects

      JavaScript Training in Chennai

      JavaScript Training in Chennai

      The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training


    Note: Only a member of this blog may post a comment.