Automating PCAP Parsing with Linux CLI, Bash & Security Onion
Introduction
In a landscape where we see odd traffic all the time in networks, pulling PCAP’s and remembering your Wireshark expressions can sometimes be a bit of a nightmare… That never ending list in your notes of expressions.
I’ve experimented with Security Onions tools such as ELK, Bro, Squill, Snort, Squert etc. and now integrated these with some nice lightweight bash scripting and CLI aliases to make for a very quick analysis platform to get an early verdict on your traffic.
CLI Aliases
I’ve setup a few ‘aliases’ in my Security Onion machine, this means if I type “SuspectedMalware” into the CLI, the CLI will recognise this as an actual command, moreover, this command will actually use Bro-Cut to parse out Malware related artefacts from your PCAP. Here are some examples below:
ParseFiles — Simply parses out all files
alias ParseFiles=”cat files.log | bro-cut -d ts fuid tx_hosts rx_hosts filename total_bytes”
SuspUA — This will parse out User Agents so you can spot anomalous ones
alias SuspUA=”cat http.log | bro-cut -d ts user-agent method url uid id”
If you want to create your own aliases to expand on this concept: https://alvinalexander.com/blog/post/linux-unix/create-aliases/
Bash scripts
Bash script primer: A bash script is just a file containing a series of commands. they’re really good for automating long and repetitive tasks which involve several commands.
Now let’s make it even easier with a few bash scripts…
I’ve written this simple bash script that will pull out interesting artefacts for early triage such as non RFC-1918 IP addresses (I’ve also excluded some generally ok CIDR ranges), interesting destination ports, files etc.
This can be used for example, to guide your analysis inside ELK on Security Onion. My other use case for this is ELK and the services that run on Security Onion can be very hardware intensive if you’re on a VM. So by using the below script and above aliases, you should be on the path to bro-cut ninja in a few hours.
Note: CD to the directory where your PCAP is first...
REM@remnux: so-import-pcap incident123.pcap
REM@remnux: bro -Cr incident123.pcap
REM@remnux: chmod +x quick.sh
REM@remnux: ./quick.sh#!/usr/bin/env bash#Gives you a quick view of user agents to potentially spot something anomalous
SuspUA(){
cat http.log | bro-cut user_agent | sort -u > UniqueUserAgents.txt}#This is the file parser, it simply gives you a list of files, along with source, destination etc. You will only get a filename if it was available. SourceIP is the first column!
ParseFiles() {
cat files.log | bro-cut tx_hosts rx_hosts mime_type filename | grep -E -v 'application/x-x509-user-cert|application/x-x509-ca-cert|application/ocsp-response|application/font-woff2' > FileList.txt}#ExtInbIPAddr will give you a list of unique external source (inbound) IP's and Ports
ExtInbIPAddr() {
cat conn.log | bro-cut id.orig_h proto service | grep -E -v '10.*.|192.168.*.|172.16.*.' | sort -u > ExtInbIPAddrs.txt}#ExtOutbIPAddr will give you a list of unique external destination (outbound) IP's and Ports
ExtOutbIPAddr() {
cat conn.log | bro-cut id.resp_h id.resp_p proto service | grep -E -v '10.*.|192.168.*.|172.16.*.' | sort -u >ExtOutbIPAddrs.txt}SuspUA
ParseFiles
ExtInbIPAddr
ExtOutbIPAddr
You can now review the results i.e. take your external IP’s, do some OSINT to work out the bad ones. Assess the files, user agents etc.
This is just an example of what you could do, to speed up your triage.
Make your own Bro Aliases and Scripts
The cheat sheets for Bro: https://github.com/corelight/bro-cheatsheets
Book to read: The Practice of Network Security Monitoring by No Starch Press
Shoutout to darkdefender’s post, this got me thinking about bro-cut (zeek-cut): https://medium.com/@melanijan93/https-medium-com-melanijan93-analysing-pcaps-with-bro-zeek-33340e710012
