Bro Quickstart: Cluster Edition

In my last post, I described the very basic commands you need to get a very minimal Bro IDS instance up and running for proof-of-concept purposes.  In this post, I'll show you how to take Bro a step further with Bro cluster.

Bro Cluster
There are several reasons for running Bro in a cluster.  A single Bro instance will be overwhelmed when it encounters more than about 80 Mb/sec of traffic.  You can run multiple Bro instances on the same machine (to take advantage of multiple cores) or multiple instances spread across distributed machines to cut the load each instance sees down to the 80 Mb/sec mark.

Another reason for using Bro cluster is the ease of managing Bro processes and logging.  There is a comprehensive log rotation framework integrated into Bro cluster, as well as seamless configuration changes on-the-fly.

Installation
This install quickstart will be written for Ubuntu Server 10.04 LTS, but aside from the prerequisite installation, the steps should be largely the same on any Linux distribution.  To run multiple Bro instances on the same local machine, we will use PF_RING's amazing ability to perform efficient, per-flow software pcap load-balancing.

# install prereqs
sudo apt-get install swig python-dev libmagic-dev libpcre3-dev libssl-dev cmake git-core subversion ruby-dev libgeoip-dev flex bison
# uninstall conflicting tcpdump
sudo apt-get remove tcpdump libpcap-0.8
cd ~/
# install PF_RING
svn export https://svn.ntop.org/svn/ntop/trunk/PF_RING/ pfring-svn
cd pfring-svn/kernel
make && sudo make install
cd ../userland/lib
./configure --prefix=/usr/local/pfring && make && sudo make install
cd ../libpcap-1.1.1-ring
./configure --prefix=/usr/local/pfring && make && sudo make install
echo "/usr/local/pfring/lib" >> /etc/ld.so.conf
cd ../tcpdump-4.1.1
./configure --prefix=/usr/local/pfring && make && sudo make install
# Add PF_RING to the ldconfig include list
echo "PATH=$PATH:/usr/local/pfring/bin:/usr/local/pfring/sbin" >> /etc/bash.bashrc
cd ~/
# Get and make Bro
mkdir brobuild && cd brobuild
git clone --recursive git://git.bro-ids.org/bro
BRODIR=/usr/local/bro-$(date +"%Y%m%d")
./configure --prefix=$BRODIR --with-pcap=/usr/local/pfring && cd build && make -j8 && sudo make install
sudo ln -s $BRODIR /usr/local/bro
cd /usr/local/bro


Now we need to edit the Bro config for your environment.  For the purposes of this quickstart, I will assume that you are monitoring a single interface, eth2.  Edit /usr/local/bro/etc/node.cfg to look like this (assuming your local host is 192.168.1.1 and you want to run 4 instances):

[manager]

type=manager

host=192.168.1.1

[proxy-0]

type=proxy

host=192.168.1.1

[worker-0]

type=worker

host=192.168.1.1

interface=eth2

lb_method=pf_ring

lb_procs=4

Now we need to edit share/bro/site/local.bro to add some options that I recommend to prevent some of the more verbose logs from filling up the drive.  Append the following text:


  event bro_init()

       {

       Log::disable_stream(HTTP::LOG);

        Log::disable_stream(Syslog::LOG);

        Log::disable_stream(Conn::LOG);

        Log::disable_stream(DNS::LOG);

        Log::disable_stream(Weird::LOG);

       }

Now we're all ready to start Bro cluster:

/usr/local/bro/bin/broctl

> install

> start

Finally, in an enterprise, you will want to get these logs into your log management or SIEM solution.  You can do this very easily out of the box in Ubuntu, which uses rsyslog by default.  Create /etc/rsyslog.d/60-bro.conf:

$ModLoad imfile #
$InputFileName /usr/local/bro/logs/current/ssl.log
$InputFileTag bro_ssl:
$InputFileStateFile stat-bro_ssl
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/smtp.log
$InputFileTag bro_smtp:
$InputFileStateFile stat-bro_smtp
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/smtp_entities.log
$InputFileTag bro_smtp_entities:
$InputFileStateFile stat-bro_smtp_entities
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/notice.log
$InputFileTag bro_notice:
$InputFileStateFile stat-bro_notice
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/ssh.log
$InputFileTag bro_ssh:
$InputFileStateFile stat-bro_ssh
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/ftp.log
$InputFileTag bro_ftp:
$InputFileStateFile stat-bro_ftp
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
# check for new lines every second
$InputFilePollingInterval 1
local7.* @central_syslog_server

Apply changes:

restart rsyslog