Monthly Archives: April 2015

Installing OpenAppID with Snort 2.9.7.x on Ubuntu 14

UPDATE: Snort 2.9.9.x has been released. Please see the updated of article here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install OpenAppId in Snort 2.9.7.2 on Ubuntu 14. If you want a more in-depth explanation of the install steps for Ubuntu (without OpenAppID), as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort on Ubuntu. If you want to test the new 3.0 alpha version of Snort, please see my article: Installing Snort 3 Alpha in Ubuntu.

Preparing to Install Snort

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev

Next we want to install the pre-requisites that are specific to OpenAppID:

sudo apt-get install -y libluajit-5.1-dev pkg-config libssl-dev

Disable LRO and GRO (notes on this in the The Snort Manual), to ensure that packets do not get truncated by the network card:

sudo ethtool -K eth0 gro off
sudo ethtool -K eth0 lro off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.4.tar.gz
tar -xvzf daq-2.0.4.tar.gz	
cd daq-2.0.4
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Installing Snort

Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/snort-2.9.7.2.tar.gz
tar -xvzf snort-2.9.7.2.tar.gz
cd snort-2.9.7.2
./configure --enable-sourcefire --enable-open-appid
make
sudo make install

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

We need to a few configuration things to prepare Snort for use. We create a snort user and group, and copy some files from the Snort source. More detailed information on the steps below can be found here.

First create the user and group:

sudo groupadd snort
sudo useradd snort -r -s /sbin/nologin -c SNORT_IDS -g snort

Then create the necessary directories and empty files:

sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo touch /etc/snort/rules/white_list.rules
sudo touch /etc/snort/rules/black_list.rules
sudo touch /etc/snort/rules/local.rules
sudo mkdir /etc/snort/preproc_rules

sudo mkdir /var/log/snort

sudo mkdir /usr/local/lib/snort_dynamicrules

sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

sudo chown -R snort:snort /etc/snort
sudo chown -R snort:snort /var/log/snort
sudo chown -R snort:snort /usr/local/lib/snort_dynamicrules

Finally copy some files:

sudo cp ~/snort_src/snort-2.9.7.2/etc/*.conf* /etc/snort
sudo cp ~/snort_src/snort-2.9.7.2/etc/*.map /etc/snort

Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time):

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Next we need to edit the /etc/snort/snort.conf Snort configuration file.  I have included the line numbers after the hash so you can more easily find the setting:

var RULE_PATH /etc/snort/rules					# line 104
var SO_RULE_PATH /etc/snort/so_rules			# 105
var PREPROC_RULE_PATH /etc/snort/preproc_rules	# 106

var WHITE_LIST_PATH /etc/snort/rules			# 113
var BLACK_LIST_PATH /etc/snort/rules			# 114

Download and Extract the Application Detector Package

Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but may have changed:

cd ~/snort_src
wget https://snort.org/downloads/openappid/1516 -O snort-openappid.tar.gz
tar -xvzf snort-openappid.tar.gz

The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:

sudo cp -r ~/snort_src/odp/ /etc/snort/rules/

Editing snort.conf to enable OpenAppID

We need to enable the OpenAppID pre-processor, then we need to have snort output the AppID data. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf). You should add the following line before the commented-out section 6 (line 512 for me):

preprocessor appid: app_stats_filename appstats-u2.log, \
   app_stats_period 60, \
   app_detector_dir /etc/snort/rules

This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.

While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6 ):

output unified2: filename snort.log, limit 128, appid_event_types

this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.

Now test the Snort configuration file:

sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0

you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.

Collecting OpenAppID Data

Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:

sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0
ctrl-c

To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is passing through the interface you are listening on (from the same computer, for example).

Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.

An example of this processing:

user@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 
statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894"
statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384"
statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490"
statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490"
statTime="1428300780",appName="https",txBytes="777",rxBytes="1444"
statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116"
statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684"
statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294"
statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282"
statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912"
statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907"
statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327"
statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694"
statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912"
statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625"
statTime="1428300720",appName="dns",txBytes="380",rxBytes="538"
statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943"
statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0"
statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0"
statTime="1428300780",appName="dns",txBytes="482",rxBytes="936"
statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591"
statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974"
statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0"
statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372"
statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"

If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.

If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.

Installing Snort 3 Alpha in Ubuntu

NOTE: this article is out of date and has been replaced with a newer article: Installing Snort++ (Snort 3 Alpha 4 build 237) in Ubuntu.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install Snort 3 alpha build 144 on Ubuntu. This install has been tested on the x64 version of Ubuntu 14 (but will probably also work on the x86 version, as well as Ubuntu 12 and 13, although I haven’t tested those systems and architectures yet). If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality for the 2.9.7.x version of Snort, see my series on installing Snort 2.9.7.x on Ubuntu.

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev

Install the Snort pre-requisites that are new for the 3.0 version of Snort:

sudo apt-get install -y libluajit-5.1-dev pkg-config

Disable LRO and GRO for the network interface that Snort will listen on (notes on this in the The Snort Manual):

sudo ethtool -K eth0 gro off
sudo ethtool -K eth0 lro off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data AcQuisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.4.tar.gz
tar -xvzf daq-2.0.4.tar.gz	
cd daq-2.0.4
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Now we are ready to install Snort from source. This command downloads and installs the 3.0.0 Alpha 1, build 144 version of Snort. If a newer version is available from the Snort website, make the necessary changes. If you want to install all the snort directories under a single directory, see the section at the bottom of this document titled Changing the install location of Snort. To install as normal:

cd ~/snort_src
wget https://www.snort.org/downloads/snortplus/snort-3.0.0-a1-144-auto.tar.gz
tar -xvzf snort-3.0.0-a1-144-auto.tar.gz
cd snort-3.0.0-a1
./configure
make -j 8
sudo make install

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

Snort 3 requires a few environmental variables, we store them temporarily in the current session so we can continue working, and save them permanently to the /etc/environment file:

export LUA_PATH=/usr/local/include/snort/lua/\?.lua\;\;
export SNORT_LUA_PATH=/usr/local/etc/snort

sudo sh -c "echo 'LUA_PATH=/usr/local/include/snort/lua/\?.lua\;\;' >> /etc/environment"
sudo sh -c "echo 'SNORT_LUA_PATH=/usr/local/etc/snort' >> /etc/environment"

to ensure that these two environmental variables are available when using sudo, we need to add them to the /etc/sudoers file:

sudo visudo

in the editor, add the following to to the bottom of the file:

Defaults env_keep += "LUA_PATH SNORT_LUA_PATH"

use ctrl-x to exit, save when prompted by pressing y, then press enter to save the file to /etc/sudoers.tmp (which will get copied automatically to /etc/sudoers).

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snort3:~$ snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0-a1 (Build 144) from 2.9.6-9
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2015 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.5.3
           Using LuaJIT version 2.0.2
           Using PCRE version 8.31 2012-07-06
           Using ZLIB version 1.2.8


user@snort3:~$

Now let’s test snort with the default configuration file and ruleset:

user@snort3:~$ /usr/local/bin/snort -c /usr/local/etc/snort/snort.lua -R /usr/local/etc/snort/sample.rules
--------------------------------------------------
o")~   Snort++ 3.0.0-a1-144
--------------------------------------------------
Loading /usr/local/etc/snort/snort.lua:
	back_orifice
	ssh
	classifications
	ftp_data
	ftp_server
	stream_tcp
	http_inspect
	telnet
	rpc_decode
	port_scan
	perf_monitor
	arp_spoof
	stream_icmp
	stream_ip
	stream
	ftp_client
	references
	stream_udp
	wizard
	dns
Finished /usr/local/etc/snort/snort.lua.
Loading rules:
Loading /usr/local/etc/snort/sample.rules:
Finished /usr/local/etc/snort/sample.rules.
Finished rules.
--------------------------------------------------
rule counts
       total rules loaded: 3974
               text rules: 3974
            option chains: 3974
            chain headers: 187
--------------------------------------------------
rule port counts
             tcp     udp    icmp      ip
     src    1685       4       0       0
     dst    1927     232       0       0
     any     118      30      28      26
      nc       1       0       1       0
     s+d       0       1       0       0
--------------------------------------------------
flowbits
                available: 1024
                     used: 124
              not checked: 9
                  not set: 2
--------------------------------------------------
pcap DAQ configured to passive.

Snort successfully validated the configuration.
o")~   Snort exiting
user@snort3:~$ 

If you have output similar to the above, then Snort 3.0.0 Alpha is installed and works.

A note on install locations:

When Snort is installed as above, a few binaries are installed to /usr/local/bin:

  • snort : The Snort binary.
  • snort2lua : Tool to convert a Snort 2.9.7.x configuration file into a 3.x configuration file. More notes here.
  • u2boat : U2boat is a tool for converting unified2 files into different formats.
  • u2spewfoo: U2SpewFoo is a lightweight tool for dumping the contents of unified2 files to stdout.

Additionally, the following folders are created / used:

  • /usr/local/bin : Binaries for Snort and supporting software.
  • /usr/local/etc/snort : The configuration files for Snort.
  • /usr/local/include/snort : All include files for Snort.
  • /usr/local/lib/pkgconfig : The pkgconfig file for Snort (compilation details for Snort).
  • /usr/local/share/doc/snort : The documentation for the installed version of Snort.

Changing the install location of Snort

If you would rather have all these folders under one folder, add ‑‑prefix=/usr/local/bin/snort to the ./configure command when preparing to build Snort. This will install all these folders under the path you choose. You also need to modify some of the other paths detailed above, so if you decide to install in that manner, you should follow the install instructions detailed in the Snort blog.

Where to go from here

If you want to learn more about how to run the 2.9.7.x version of Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu.

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.