Installing OpenAppID with Snort 2.9.8.x on Ubuntu
UPDATE: Snort 2.9.9.x has been released. Please see the updated of article here.
I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The instructions below show how to install OpenAppId in Snort 2.9.8.x on Ubuntu 14.
If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality, see my in-depth series for installing Snort on Ubuntu, or my Quick Install Guide for Snort 2.9.8.x on Ubuntu.
If you want to test the new alpha version of Snort (Version 3.0 Alpha 2), please see my articles: Installing Snort 3 Alpha in Ubuntu 14, or Ubuntu 12.
Let Us Get Started
So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:
sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev ethtool
Next we want to install the pre-requisites that are specific to OpenAppID:
sudo apt-get install -y libluajit-5.1-dev pkg-config openssl libssl-dev
Disable LRO and GRO for all interfaces Snort will listen on under /etc/network/interfaces. using ethtool. An explanation of LRO and GRO are in the The Snort Manual). Use an editor to edit the network interfaces file:
sudo vi /etc/network/interfaces
and for every interface that Snort will listen on (one interface for simple setups, multiple interfaces for more complex setups), add the following two lines, changing eth0 to match the interface:
post-up ethtool -K eth0 gro off post-up ethtool -K eth0 lro off
for example, my /etc/network/interfaces file looks like this:
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet dhcp post-up ethtool -K eth0 gro off post-up ethtool -K eth0 lro off
Reboot the system and verify that LRO and GRO are off:
user@snortserver:~$ ethtool -k eth0 | grep receive-offload generic-receive-offload: off large-receive-offload: off
Next we will create a directory to save the downloaded tarball files:
mkdir ~/snort_src cd ~/snort_src
Download and install Data Acquisition library (DAQ) from the Snort website:
cd ~/snort_src wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz tar -xvzf daq-2.0.6.tar.gz cd daq-2.0.6 ./configure make sudo make install
Installing Snort
Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:
Now we are ready to install Snort from source:
cd ~/snort_src wget https://snort.org/downloads/snort/snort-2.9.8.0.tar.gz tar -xvzf snort-2.9.8.0.tar.gz cd snort-2.9.8.0 ./configure --enable-sourcefire --enable-open-appid make sudo make install
Run the following command to update shared libraries:
sudo ldconfig
Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:
sudo ln -s /usr/local/bin/snort /usr/sbin/snort
We need to a few configuration things to prepare Snort for use. More detailed information on the steps below can be found here .
Create the needed directories and empty files:
# Create the Snort directories: sudo mkdir /etc/snort sudo mkdir /etc/snort/rules sudo mkdir /etc/snort/rules/iplists sudo mkdir /etc/snort/preproc_rules sudo mkdir /usr/local/lib/snort_dynamicrules sudo mkdir /etc/snort/so_rules # Create some files that stores rules and ip lists sudo touch /etc/snort/rules/iplists/default.blacklist sudo touch /etc/snort/rules/iplists/default.whitelist sudo touch /etc/snort/rules/local.rules sudo touch /etc/snort/sid-msg.map # Create our logging directories: sudo mkdir /var/log/snort sudo mkdir /var/log/snort/archived_logs # Adjust permissions: sudo chmod -R 5775 /etc/snort sudo chmod -R 5775 /var/log/snort sudo chmod -R 5775 /var/log/snort/archived_logs sudo chmod -R 5775 /etc/snort/so_rules sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules
Finally copy some files:
cd ~/snort_src/snort-2.9.8.0/etc/ sudo cp *.conf* /etc/snort sudo cp *.map /etc/snort sudo cp *.dtd /etc/snort cd ~/snort_src/snort-2.9.8.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/ sudo cp * /usr/local/lib/snort_dynamicpreprocessor/
Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time) by running the following command:
sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf
Next we need to edit the /etc/snort/snort.conf Snort configuration file.
Line 45 of /etc/snort/snort.conf: the variable HOME_NET should match your internal (defended) network. In the below example our HOME NET is 10.0.0.0 with a 24-bit subnet mask (255.255.255.0):
ipvar HOME_NET 10.0.0.0/24
Still editing snort.conf, next we need to modify some file paths to match the lines below, beginning at line 104:
var RULE_PATH /etc/snort/rules var SO_RULE_PATH /etc/snort/so_rules var PREPROC_RULE_PATH /etc/snort/preproc_rules var WHITE_LIST_PATH /etc/snort/rules/iplists var BLACK_LIST_PATH /etc/snort/rules/iplists
Still editing snort.conf, next we need to modify the whitelist and blacklist path, beginning at line 511:
whitelist $WHITE_LIST_PATH/default.whitelist, \
blacklist $BLACK_LIST_PATH/default.blacklist
Once you have saved your edits to snort.conf, you should test that snort can load this configuration file without any errors. You do this by running snort with the -T flag to tell snort to test the file, the -c flag to identify the path of the snort.conf file, and the -i flag for a network interface that Snort will listen on. This is shown below. Output has been truncated to the final few lines to show success:
user@snortserver:~$ sudo snort -T -i eth0 -c /etc/snort/snort.conf (...) Snort successfully validated the configuration! Snort exiting user@snortserver:~$
Download and Extract the Application Detector Package
Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but will probably have changed, as the Snort team is updating regularly:
cd ~/snort_src wget https://snort.org/downloads/openappid/3192 -O snort-openappid.tar.gz tar -xvzf snort-openappid.tar.gz
The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:
sudo cp -r ~/snort_src/odp/ /etc/snort/rules/
and create one folder for third-party developed application detectors:
sudo mkdir /usr/local/lib/thirdparty
Editing snort.conf to enable OpenAppID
We need to enable the OpenAppID pre-processor, then we need to have Snort output the AppID data. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf), and add the following line before the commented-out section 6 (line 513 for me):
preprocessor appid: app_stats_filename appstats-u2.log, \ app_stats_period 60, \ app_detector_dir /etc/snort/rules
This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.
While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6, around line 526 ):
output unified2: filename snort.log, limit 128, appid_event_types
this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.
Now test the Snort configuration file to verify there are no errors:
sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0
as above, you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.
Collecting OpenAppID Data
Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:
sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0 ctrl-c
To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is visible to the interface that snort is listening on, either by passing that data directly through the Snort interface, or by ensuring that your network infrastructure copies network traffic to the Snort server (span port, port mirroring, or promiscuous mode, for example).
Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.
A simple example of this processing:
noah@snort:~$ sudo u2openappid /var/log/snort/appstats-u2.log.1449426302 statTime="1449426240",appName="HTTP",txBytes="0",rxBytes="8152" statTime="1449426300",appName="HTTP",txBytes="0",rxBytes="9542" statTime="1449426240",appName="DNS",txBytes="301",rxBytes="0" statTime="1449426240",appName="__unknown",txBytes="12376",rxBytes="1118" statTime="1449426300",appName="DNS",txBytes="761",rxBytes="0"
In the above example, I used curl over the same interface snort was listening on to request www.xkcd.com. The various application detectors show the amount of traffic for each detector, DNS, HTTP, and the like.
An more complex example of this processing (from an older version of OpenAppID, but still valid):
noah@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894" statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384" statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490" statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490" statTime="1428300780",appName="https",txBytes="777",rxBytes="1444" statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116" statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684" statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294" statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282" statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912" statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907" statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327" statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694" statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912" statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625" statTime="1428300720",appName="dns",txBytes="380",rxBytes="538" statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943" statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0" statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0" statTime="1428300780",appName="dns",txBytes="482",rxBytes="936" statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591" statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974" statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0" statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372" statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074" statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074" statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"
If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.
If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my in-depth series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.
Comments are Disabled