Automating PCAP Parsing with Linux CLI, Bash & Security Onion

Source: https://dribbble.com/shots/11988083-Server-Icons

Introduction

I’ve experimented with Security Onions tools such as ELK, Bro, Squill, Snort, Squert etc. and now integrated these with some nice lightweight bash scripting and CLI aliases to make for a very quick analysis platform to get an early verdict on your traffic.

Zeek — Formally Bro

CLI Aliases

ParseFiles — Simply parses out all files

SuspUA — This will parse out User Agents so you can spot anomalous ones

If you want to create your own aliases to expand on this concept: https://alvinalexander.com/blog/post/linux-unix/create-aliases/

Bash scripts

Now let’s make it even easier with a few bash scripts…

I’ve written this simple bash script that will pull out interesting artefacts for early triage such as non RFC-1918 IP addresses (I’ve also excluded some generally ok CIDR ranges), interesting destination ports, files etc.

This can be used for example, to guide your analysis inside ELK on Security Onion. My other use case for this is ELK and the services that run on Security Onion can be very hardware intensive if you’re on a VM. So by using the below script and above aliases, you should be on the path to bro-cut ninja in a few hours.

Note: CD to the directory where your PCAP is first...
REM@remnux: so-import-pcap incident123.pcap
REM@remnux: bro -Cr incident123.pcap
REM@remnux: chmod +x quick.sh
REM@remnux: ./quick.sh
#!/usr/bin/env bash#Gives you a quick view of user agents to potentially spot something anomalous
SuspUA(){
cat http.log | bro-cut user_agent | sort -u > UniqueUserAgents.txt
}#This is the file parser, it simply gives you a list of files, along with source, destination etc. You will only get a filename if it was available. SourceIP is the first column!
ParseFiles() {
cat files.log | bro-cut tx_hosts rx_hosts mime_type filename | grep -E -v 'application/x-x509-user-cert|application/x-x509-ca-cert|application/ocsp-response|application/font-woff2' > FileList.txt
}#ExtInbIPAddr will give you a list of unique external source (inbound) IP's and Ports
ExtInbIPAddr() {
cat conn.log | bro-cut id.orig_h proto service | grep -E -v '10.*.|192.168.*.|172.16.*.' | sort -u > ExtInbIPAddrs.txt
}#ExtOutbIPAddr will give you a list of unique external destination (outbound) IP's and Ports
ExtOutbIPAddr() {
cat conn.log | bro-cut id.resp_h id.resp_p proto service | grep -E -v '10.*.|192.168.*.|172.16.*.' | sort -u >ExtOutbIPAddrs.txt
}SuspUA
ParseFiles
ExtInbIPAddr
ExtOutbIPAddr

You can now review the results i.e. take your external IP’s, do some OSINT to work out the bad ones. Assess the files, user agents etc.

This is just an example of what you could do, to speed up your triage.

Make your own Bro Aliases and Scripts

Book to read: The Practice of Network Security Monitoring by No Starch Press

Shoutout to darkdefender’s post, this got me thinking about bro-cut (zeek-cut): https://medium.com/@melanijan93/https-medium-com-melanijan93-analysing-pcaps-with-bro-zeek-33340e710012

Twitter: @mikecybersec

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store