Posts Tagged: OpenAppID

Installing Snort++ in Ubuntu (Version 3.0 Alpha 4 build 240)

The instructions below show how to install Snort 3 alpha 4 build 240 on Ubuntu. This install has been tested on Ubuntu 14 and 16, for both the x86 and x64 architectures. For an outdated Ubuntu 12 version of these instructions, please go here. Note that Snort 3 is Alpha software, and therefore has bugs and issues, and should be installed for testing purposes only (not on production systems).

Snort 3 Alpha 4 Build 40 was released on November 1st, 2017, and this guide has been tested with that version (releases after this specific release may not follow the same steps). Generic build instructions, prerequisites, and detailed notes are available in the manual.

If you want a more in-depth explanation of the install steps, which are very similar to the 2.9.9.x version of Snort, as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort 2.9.9.x on Ubuntu.

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential autotools-dev libdumbnet-dev libluajit-5.1-dev libpcap-dev libpcre3-dev zlib1g-dev pkg-config libhwloc-dev cmake

Install the optional (recommended) software:

sudo apt-get install -y liblzma-dev openssl libssl-dev cpputest libsqlite3-dev uuid-dev

Install tools required for compiling the source from github:

sudo apt-get install -y libtool git autoconf

Install the DAQ pre-requisites:

sudo apt-get install -y bison flex

If you want to build the documentation as well (not really needed, unless you want it, usually about 700 MB of libraries):

sudo apt-get install -y asciidoc dblatex source-highlight w3m

If you want to run Snort in inline mode using NFQ, install the required packages (not required for IDS mode or inline mode using afpacket). If you’re unsure, you should install this package.

sudo apt-get install -y libnetfilter-queue-dev

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

First and install safec for runtime bounds checks on certain legacy C-library calls (this is optional but recommended):

cd ~/snort_src
wget http://downloads.sourceforge.net/project/safeclib/libsafec-10052013.tar.gz
tar -xzvf libsafec-10052013.tar.gz
cd libsafec-10052013
./configure
make
sudo make install

One of the Snort recommended prerequisites is Hyperscan 4.4.0. From their webpage: “Hyperscan is a regular expression engine designed to offer high performance, the ability to match multiple expressions simultaneously and flexibility in scanning operation.” Hyperscan needs Ragel 6.9 and the Boost header libraries.

Install Ragel 6.10 from source:

cd ~/snort_src
wget http://www.colm.net/files/ragel/ragel-6.10.tar.gz
tar -xzvf ragel-6.10.tar.gz
cd ragel-6.10
./configure
make
sudo make install

Download the Boost 1.65.1 libraries, but do not install:

cd ~/snort_src
wget https://dl.bintray.com/boostorg/release/1.65.1/source/boost_1_65_1.tar.gz
tar -xvzf boost_1_65_1.tar.gz

Install Hyperscan 4.6.0 from source, referencing the location of the Boost source directory:

cd ~/snort_src
wget https://github.com/intel/hyperscan/archive/v4.6.0.tar.gz
tar -xvzf v4.6.0.tar.gz
mkdir ~/snort_src/hyperscan-4.6.0-build
cd hyperscan-4.6.0-build/

cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_65_1/ ../hyperscan-4.6.0

make
sudo make install

If you want to test that Hyperscan works, from the build directory, run:

cd ~/snort_src/hyperscan-4.6.0-build/
./bin/unit-hyperscan

The unit tests will run (this takes a few minutes).

Snort has an optional requirement for flatbuffers, A memory efficient serialization library:

cd ~/snort_src
wget https://github.com/google/flatbuffers/archive/master.tar.gz -O flatbuffers-master.tar.gz
tar -xvzf flatbuffers-master.tar.gz
mkdir flatbuffers-build
cd flatbuffers-build

cmake ../flatbuffers-master

make
sudo make install

Download and install Data AcQuisition library (DAQ) from the Snort website (note that DAQ for Snort 3 is a different DAQ than for the 2.9.9.x series of Snort):

cd ~/snort_src
wget https://www.snort.org/downloads/snortplus/daq-2.2.2.tar.gz
tar -xvzf daq-2.2.2.tar.gz
cd daq-2.2.2
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Now we are ready to install Snort from source. This command downloads and installs the latest version of Snort 3 (currently 3.0.0 Alpha 4, build 239, but as the codebase is updated, you’ll get a newer version). If you want to specifically download the version used in this guide, use this URL instead with wget below: https://github.com/snortadmin/snort3/archive/BUILD_239.tar.gz.

If you want to install all the snort directories under a single directory, see the section at the bottom of this document titled Changing the install location of Snort. Here we choose to install the entire Snort directory structure to a single folder under /opt/:

cd ~/snort_src
wget https://github.com/snortadmin/snort3/archive/master.tar.gz
tar -xvzf master.tar.gz
cd snort3-master/
autoreconf -isvf

./configure --prefix=/opt/snort

make
sudo make install

Since the Snort installation places the Snort binary at /opt/snort/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /opt/snort/bin/snort /usr/sbin/snort

Snort 3 requires a few environmental variables, we store them temporarily in the current session so we can continue working, and save them permanently to the ~/.bashrc file (you’ll need to do this for every user profile):

export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;
export SNORT_LUA_PATH=/opt/snort/etc/snort
 
sh -c "echo 'export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;' >> ~/.bashrc"
sh -c "echo 'export SNORT_LUA_PATH=/opt/snort/etc/snort' >> ~/.bashrc"

to ensure that these two environmental variables are available when using sudo, we need to add them to the /etc/sudoers file:

sudo visudo

in the editor, add the following to to the bottom of the file:

Defaults env_keep += "LUA_PATH SNORT_LUA_PATH"

use ctrl-x to exit, save when prompted by pressing y, then press enter to save the file to /etc/sudoers.tmp (which will get copied automatically to /etc/sudoers).

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snort3:~$ snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0-a4 (Build 240) from 2.9.8-383
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2017 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.2
           Using LuaJIT version 2.0.4
           Using OpenSSL 1.0.2g 1 Mar 2016
           Using libpcap version 1.7.4
           Using PCRE version 8.38 2015-11-23
           Using ZLIB version 1.2.8
           Using FlatBuffers 1.7.0
           Using Hyperscan version 4.6.0 2017-11-04
           Using LZMA version 5.1.0alpha

user@snort3:~$

A note on install locations:

When you install snort to /opt/snort, you get the following folder structure:

user@snort3x86:/opt/snort$ tree /opt/snort -L 3
/opt/snort
├── bin
│   ├── fbstreamer
│   ├── snort
│   ├── snort2lua
│   ├── u2boat
│   └── u2spewfoo
├── etc
│   └── snort
│       ├── file_magic.lua
│       ├── snort_defaults.lua
│       └── snort.lua
├── include
│   └── snort
│       ├── actions
│       ├── codecs
│       ├── daqs
│       ├── decompress
│       ├── detection
│       ├── events
│       ├── file_api
│       ├── flow
│       ├── framework
│       ├── hash
│       ├── log
│       ├── lua
│       ├── main
│       ├── managers
│       ├── mime
│       ├── packet_io
│       ├── profiler
│       ├── protocols
│       ├── pub_sub
│       ├── search_engines
│       ├── sfip
│       ├── stream
│       ├── time
│       └── utils
├── lib
│   ├── pkgconfig
│   │   └── snort.pc
│   └── snort
│       └── daqs
└── share
    └── doc
        └── snort

38 directories, 8 files

The /opt/snort/bin folder contains the following Snort binaries:

  • fbstreamer : A utility for accessing the statistics generated in flatbuffer format.
  • snort : The Snort binary.
  • snort2lua : Tool to convert a Snort 2.9.8.x configuration file into a 3.x configuration file. More notes here.
  • u2boat : U2boat is a tool for converting unified2 files into different formats.
  • u2spewfoo: U2SpewFoo is a lightweight tool for dumping the contents of unified2 files to stdout.

Additionally, the following folders are created / used:

  • /opt/snort/bin : Binaries for Snort and supporting software.
  • /opt/snort/etc/snort : The configuration files for Snort.
  • /opt/snort/include/snort : All include files for Snort.
  • /opt/snort/lib/pkgconfig : The pkgconfig file for Snort (compilation details for Snort).
  • /opt/snort/share/doc/snort : The documentation for the installed version of Snort.

Changing the install location of Snort

If you would rather have all these folders install to a more normal location (/usr/local) , add ‑‑prefix=/usr/local/ to the ./configure command when preparing to build Snort. This will install all these folders under the path you choose. You also need to modify some of the other paths detailed above, so if you decide to install in that manner, you should follow the install instructions detailed in the Snort blog.

Snort Rules

Snort3 rules have more options than Snort 2 rules, and while the normal rules downloaded with PulledPork or manually will work, for testing you will probably want to download the set of community rules specificallycreated for snort3. You can manually download snort3 specific community rules from the snort website:

cd ~/snort_src/
wget https://www.snort.org/downloads/community/snort3-community-rules.tar.gz
tar -xvzf snort3-community-rules.tar.gz
cd snort3-community-rules
sudo mkdir /opt/snort/etc/snort/rules
sudo cp snort3-community.rules /opt/snort/etc/snort/rules/
sudo cp sid-msg.map /opt/snort/etc/snort/rules/

now test that snort can load these rules:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules

your output should contain something similar:

Loading rules:
Loading /opt/snort/etc/snort/rules/snort3-community.rules:
Finished /opt/snort/etc/snort/rules/snort3-community.rules.
Finished rules.
--------------------------------------------------
rule counts
total rules loaded: 3462
text rules: 3462
option chains: 3462
chain headers: 264
--------------------------------------------------
...

you may want to run Snort with the following flags to detect issues: the warn-all and pedantic flags. From the Snort3 manual:
Warnings are not emitted unless –warn-* is specified. –warn-all enables all warnings, and –pedantic makes such warnings fatal

Installing OpenAppID

OpenAppID allows for the identification of application layer traffic. The Snort team has put together a package of detectors, with assistance from the community that you can download and install, called the Application Detector Package which needs to be installed. First download the OpenAppID detector package:

cd ~/snort_src/
wget https://www.snort.org/downloads/openappid/5759 -O OpenAppId-6329
tar -xzvf OpenAppId-6329
sudo cp -R odp /opt/snort/lib/

Now we need to edit our snort configuration file to point to this odp directory:

sudo vi /opt/snort/etc/snort/snort.lua

At line 113 (yours line number may be slightly different) you will see the appid= entry. You will want to add the app detector dir option here, pointing to the parent folder of the odf folder. It should look like this:

appid =
{
    app_detector_dir = '/opt/snort/lib',
}

note that you must have four spaces (not a tab) for the indented line. Now we want to test the configuration file loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua --warn-all

you should see output similar to:

...
Snort successfully validated the configuration (with 0 warnings).
o")~ Snort exiting

Now to load Snort with the OpenAppID detectors, as well as all rules (we omit the pedantic flag, since the rules will throw flowbit warnings that are non fatal and can be ignored:)

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules --warn-all

Create a simple rule to test that OpenAppID is working correctly:

sudo touch /opt/snort/etc/snort/rules/local.rules
sudo vi /opt/snort/etc/snort/rules/local.rules

with the following content:

alert tcp any any -> any any ( msg:"Facebook trafic Seen"; appids:"Facebook";sid:10000001; )

test to make sure the rule loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules --warn-all

You should see one rule loaded successfully. Now let’s run snort in detection mode on an interface (change eth0 below to match your interface name), printing alerts to the console:

sudo /opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules -i eth0 -A alert_fast -k none

the -k none flag tells Snort to ignore bad checksums. the Stream and Frag decoders will drop packets that have bad checksums, and the packets will not get processed by the OpenAppID detectors. By including this flag, we ensure that a packet with a bad checksum still gets processed. Now from another window on that computer (open a new terminal window or a second ssh session), use wget
to connect to facebook:

wget facebook.com

from the first console window you will see alerts output similar to the following:

08/05−19:13:45.451834 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 157.240.1.35:443 −> 10.0.0.104:33882
08/05−19:13:45.451842 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 10.0.0.104:33882 −> 157.240.1.35:443

use ctrl-c to stop Snort.

Note: if you are collecting packets with a larger MTU that the standard MTU for your adapter (VLAN tagged packets, MPLS Packets, packets from a different network type with a larger MTU), you may need to use the –snaplen flag to adjust snort to process larger packets)

Where to go from here

If you want to learn more about how to run the 2.9.9.x version of Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you want to develop Snort plugins, please see my guide: Installing Snort++ Example Plugins.

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Installing Snort++ in Ubuntu (Version 3.0 Alpha 4 build 239)

NOTE: this article is out of date and has been replaced with a newer article: Installing Snort++ (Snort 3 Alpha 4 build 240) in Ubuntu

The instructions below show how to install Snort 3 alpha 4 build 239 on Ubuntu. This install has been tested on Ubuntu 14 and 16, for both the x86 and x64 architectures. For an outdated Ubuntu 12 version of these instructions, please go here. Note that Snort 3 is Alpha software, and therefore has bugs and issues, and should be installed for testing purposes only (not on production systems).

Snort 3 Alpha 4 Build 239 was released on July 28, 2017, and this guide has been tested with that version (releases after this specific release may not follow the same steps). Generic build instructions, prerequisites, and detailed notes are available in the manual.

If you want a more in-depth explanation of the install steps, which are very similar to the 2.9.9.x version of Snort, as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort 2.9.9.x on Ubuntu.

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential autotools-dev libdumbnet-dev libluajit-5.1-dev libpcap-dev libpcre3-dev zlib1g-dev pkg-config libhwloc-dev cmake

Install the optional (recommended) software:

sudo apt-get install -y liblzma-dev openssl libssl-dev cpputest libsqlite3-dev

Install tools required for compiling the source from github:

sudo apt-get install -y libtool git autoconf

Install the DAQ pre-requisites:

sudo apt-get install -y bison flex

If you want to build the documentation as well (not really needed, unless you want it, usually about 700 MB of libraries):

sudo apt-get install -y asciidoc dblatex source-highlight

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

First and install safec for runtime bounds checks on certain legacy C-library calls (this is optional but recommended):

cd ~/snort_src
wget http://downloads.sourceforge.net/project/safeclib/libsafec-10052013.tar.gz
tar -xzvf libsafec-10052013.tar.gz
cd libsafec-10052013
./configure
make
sudo make install

One of the Snort recommended prerequisites is Hyperscan 4.4.0. From their webpage: “Hyperscan is a regular expression engine designed to offer high performance, the ability to match multiple expressions simultaneously and flexibility in scanning operation.” Hyperscan needs Ragel 6.9 and the Boost header libraries.

Install Ragel 6.10 from source:

cd ~/snort_src
wget http://www.colm.net/files/ragel/ragel-6.10.tar.gz
tar -xzvf ragel-6.10.tar.gz
cd ragel-6.10
./configure
make
sudo make install

Download the Boost 1.64 libraries, but do not install:

cd ~/snort_src
wget https://dl.bintray.com/boostorg/release/1.64.0/source/boost_1_64_0.tar.gz
tar -xvzf boost_1_64_0.tar.gz

Install Hyperscan 4.5.1 from source, referencing the location of the Boost source directory:

cd ~/snort_src
wget https://github.com/01org/hyperscan/archive/v4.5.2.tar.gz
tar -xvzf v4.5.2.tar.gz
mkdir ~/snort_src/hyperscan-4.5.2-build
cd hyperscan-4.5.2-build/

cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_64_0/ ../hyperscan-4.5.2

make
sudo make install

If you want to test that Hyperscan works, from the build directory, run:

cd ~/snort_src/hyperscan-4.5.2-build/
./bin/unit-hyperscan

The unit tests will run (this takes a few minutes).

Download and install Data AcQuisition library (DAQ) from the Snort website (note that DAQ for Snort 3 is a different DAQ than for the 2.9.9.x series of Snort):

cd ~/snort_src
wget https://www.snort.org/downloads/snortplus/daq-2.2.2.tar.gz
tar -xvzf daq-2.2.2.tar.gz
cd daq-2.2.2
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Now we are ready to install Snort from source. This command downloads and installs the latest version of Snort 3 (currently 3.0.0 Alpha 4, build 239, but as the codebase is updated, you’ll get a newer version). If you want to specifically download the version used in this guide, use this URL instead with wget below: https://github.com/snortadmin/snort3/archive/BUILD_239.tar.gz.

If you want to install all the snort directories under a single directory, see the section at the bottom of this document titled Changing the install location of Snort. Here we choose to install the entire Snort directory structure to a single folder under /opt/:

cd ~/snort_src
wget https://github.com/snortadmin/snort3/archive/master.tar.gz
tar -xvzf master.tar.gz
cd snort3-master/
autoreconf -isvf

./configure --prefix=/opt/snort

make
sudo make install

Since the Snort installation places the Snort binary at /opt/snort/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /opt/snort/bin/snort /usr/sbin/snort

Snort 3 requires a few environmental variables, we store them temporarily in the current session so we can continue working, and save them permanently to the ~/.bashrc file (you’ll need to do this for every user profile):

export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;
export SNORT_LUA_PATH=/opt/snort/etc/snort
 
sh -c "echo 'export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;' >> ~/.bashrc"
sh -c "echo 'export SNORT_LUA_PATH=/opt/snort/etc/snort' >> ~/.bashrc"

to ensure that these two environmental variables are available when using sudo, we need to add them to the /etc/sudoers file:

sudo visudo

in the editor, add the following to to the bottom of the file:

Defaults env_keep += "LUA_PATH SNORT_LUA_PATH"

use ctrl-x to exit, save when prompted by pressing y, then press enter to save the file to /etc/sudoers.tmp (which will get copied automatically to /etc/sudoers).

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snort3:~$ snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0-a4 (Build 239) from 2.9.8-383
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2017 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.2
           Using libpcap version 1.7.4
           Using LuaJIT version 2.0.4
           Using PCRE version 8.38 2015-11-23
           Using ZLIB version 1.2.8
           Using LZMA version 5.1.0alpha
           Using OpenSSL 1.0.2g  1 Mar 2016
           Using Hyperscan version 4.5.2 2017-08-05

user@snort3:~$

A note on install locations:

When you install snort to /opt/snort, you get the following folder structure:

user@snort3x86:/opt/snort$ tree /opt/snort -L 3
/opt/snort
├── bin
│   ├── snort
│   ├── snort2lua
│   ├── u2boat
│   └── u2spewfoo
├── etc
│   └── snort
│       ├── file_magic.lua
│       ├── snort_defaults.lua
│       └── snort.lua
├── include
│   └── snort
│       ├── actions
│       ├── codecs
│       ├── daqs
│       ├── decompress
│       ├── detection
│       ├── events
│       ├── file_api
│       ├── flow
│       ├── framework
│       ├── hash
│       ├── log
│       ├── lua
│       ├── main
│       ├── managers
│       ├── mime
│       ├── packet_io
│       ├── profiler
│       ├── protocols
│       ├── pub_sub
│       ├── search_engines
│       ├── sfip
│       ├── stream
│       ├── time
│       └── utils
├── lib
│   ├── pkgconfig
│   │   └── snort.pc
│   └── snort
│       └── daqs
└── share
    └── doc
        └── snort

36 directories, 8 files

The /opt/snort/bin folder contains the following Snort binaries:

  • snort : The Snort binary.
  • snort2lua : Tool to convert a Snort 2.9.8.x configuration file into a 3.x configuration file. More notes here.
  • u2boat : U2boat is a tool for converting unified2 files into different formats.
  • u2spewfoo: U2SpewFoo is a lightweight tool for dumping the contents of unified2 files to stdout.

Additionally, the following folders are created / used:

  • /opt/snort/bin : Binaries for Snort and supporting software.
  • /opt/snort/etc/snort : The configuration files for Snort.
  • /opt/snort/include/snort : All include files for Snort.
  • /opt/snort/lib/pkgconfig : The pkgconfig file for Snort (compilation details for Snort).
  • /opt/snort/share/doc/snort : The documentation for the installed version of Snort.

Changing the install location of Snort

If you would rather have all these folders install to a more normal location (/usr/local) , add ‑‑prefix=/usr/local/ to the ./configure command when preparing to build Snort. This will install all these folders under the path you choose. You also need to modify some of the other paths detailed above, so if you decide to install in that manner, you should follow the install instructions detailed in the Snort blog.

Snort Rules

Snort3 rules have more options than Snort 2 rules, and while the normal rules downloaded with PulledPork or manually will work, for testing you will probably want to download the set of community rules specificallycreated for snort3. You can manually download snort3 specific community rules from the snort website:

cd ~/snort_src/
wget https://www.snort.org/downloads/community/snort3-community-rules.tar.gz
tar -xvzf snort3-community-rules.tar.gz
cd snort3-community-rules
sudo mkdir /opt/snort/etc/snort/rules
sudo cp snort3-community.rules /opt/snort/etc/snort/rules/
sudo cp sid-msg.map /opt/snort/etc/snort/rules/

now test that snort can load these rules:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules

your output should contain something similar:

Loading rules:
Loading /opt/snort/etc/snort/rules/snort3-community.rules:
Finished /opt/snort/etc/snort/rules/snort3-community.rules.
Finished rules.
--------------------------------------------------
rule counts
total rules loaded: 3462
text rules: 3462
option chains: 3462
chain headers: 264
--------------------------------------------------
...

you may want to run Snort with the following flags to detect issues: the warn-all and pedantic flags. From the Snort3 manual:
Warnings are not emitted unless –warn-* is specified. –warn-all enables all warnings, and –pedantic makes such warnings fatal

Installing OpenAppID

OpenAppID allows for the identification of application layer traffic. The Snort team has put together a package of detectors, with assistance from the community that you can download and install, called the Application Detector Package which needs to be installed. First download the OpenAppID detector package:

cd ~/snort_src/
wget https://www.snort.org/downloads/openappid/5759 -O OpenAppId-5759
tar -xzvf OpenAppId-5759
sudo cp -R odp /opt/snort/lib/

Now we need to edit our snort configuration file to point to this odp directory:

sudo vi /opt/snort/etc/snort/snort.lua

At line 113 (yours line number may be slightly different) you will see the appid= entry. You will want to add the app detector dir option here, pointing to the parent folder of the odf folder. It should look like this:

appid =
{
    app_detector_dir = '/opt/snort/lib',
}

note that you must have four spaces (not a tab) for the indented line. Now we want to test the configuration file loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua --warn-all

you should see output similar to:

...
Snort successfully validated the configuration (with 0 warnings).
o")~ Snort exiting

Now to load Snort with the OpenAppID detectors, as well as all rules (we omit the pedantic flag, since the rules will throw flowbit warnings that are non fatal and can be ignored:)

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules --warn-all

Create a simple rule to test that OpenAppID is working correctly:

sudo touch /opt/snort/etc/snort/rules/local.rules
sudo vi /opt/snort/etc/snort/rules/local.rules

with the following content:

alert tcp any any -> any any ( msg:"Facebook trafic Seen"; appids:"Facebook";sid:10000001; )

test to make sure the rule loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules --warn-all

You should see one rule loaded successfully. Now let’s run snort in detection mode on an interface (change eth0 below to match your interface name), printing alerts to the console:

sudo /opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules -i eth0 -A alert_fast -k none

the -k none flag tells Snort to ignore bad checksums. the Stream and Frag decoders will drop packets that have bad checksums, and the packets will not get processed by the OpenAppID detectors. By including this flag, we ensure that a packet with a bad checksum still gets processed. Now from another window on that computer (open a new terminal window or a second ssh session), use wget
to connect to facebook:

wget facebook.com

from the first console window you will see alerts output similar to the following:

08/05−19:13:45.451834 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 157.240.1.35:443 −> 10.0.0.104:33882
08/05−19:13:45.451842 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 10.0.0.104:33882 −> 157.240.1.35:443

use ctrl-c to stop Snort.

Note: if you are collecting packets with a larger MTU that the standard MTU for your adapter (VLAN tagged packets, MPLS Packets, packets from a different network type with a larger MTU), you may need to use the –snaplen flag to adjust snort to process larger packets)

Where to go from here

If you want to learn more about how to run the 2.9.9.x version of Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you want to develop Snort plugins, please see my guide: Installing Snort++ Example Plugins.

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Installing OpenAppID with Snort 2.9.9.x on Ubuntu

The instructions below show how to install OpenAppId in Snort 2.9.9.x on Ubuntu 14 and Ubuntu 16.

If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality, see my in-depth series for installing Snort on Ubuntu, or my Quick Install Guide for Snort 2.9.9.x on Ubuntu. If you want to test the new alpha version of Snort (Version 3.0 Alpha 4), please see my article: Installing Snort 3 Alpha in Ubuntu.

Let Us Get Started

So let’s get started. First we need tocreate a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Next we install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev

Snort 2.9.9.x needs the development libraries for Nghttp2. On Ubuntu 16 this is simple:

# Ubuntu 16 only (not Ubuntu 14)
sudo apt-get install -y libnghttp2-dev

On Ubuntu 14, we do this from scratch:

# Ubuntu 14 only (not Ubuntu 16)
sudo apt-get install -y autoconf libtool pkg-config
cd ~/snort_src
wget https://github.com/nghttp2/nghttp2/releases/download/v1.17.0/nghttp2-1.17.0.tar.gz
tar -xzvf nghttp2-1.17.0.tar.gz
cd nghttp2-1.17.0
autoreconf -i --force
automake
autoconf
./configure --enable-lib-only
make
sudo make install

Next we want to install the pre-requisites that are specific to OpenAppID:

sudo apt-get install -y libluajit-5.1-dev pkg-config openssl libssl-dev

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

Installing Snort

Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:

Now we are ready to install Snort from source:

cd ~/snort_src
wget https://snort.org/downloads/snort/snort-2.9.9.0.tar.gz
tar -xvzf snort-2.9.9.0.tar.gz
cd snort-2.9.9.0
./configure --enable-sourcefire --enable-open-appid
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

We need to a few configuration things to prepare Snort for use. More detailed information on the steps below can be found here .

Create the needed directories and empty files:

# Create the Snort directories:
sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo mkdir /etc/snort/rules/iplists
sudo mkdir /etc/snort/preproc_rules
sudo mkdir /usr/local/lib/snort_dynamicrules
sudo mkdir /etc/snort/so_rules

# Create some files that stores rules and ip lists
sudo touch /etc/snort/rules/iplists/black_list.rules
sudo touch /etc/snort/rules/iplists/white_list.rules
sudo touch /etc/snort/rules/local.rules
sudo touch /etc/snort/sid-msg.map

# Create our logging directories:
sudo mkdir /var/log/snort
sudo mkdir /var/log/snort/archived_logs

# Adjust permissions:
sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /var/log/snort/archived_logs
sudo chmod -R 5775 /etc/snort/so_rules
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

Finally copy some files:

cd ~/snort_src/snort-2.9.9.0/etc/
sudo cp *.conf* /etc/snort
sudo cp *.map /etc/snort
sudo cp *.dtd /etc/snort
cd ~/snort_src/snort-2.9.9.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/
sudo cp * /usr/local/lib/snort_dynamicpreprocessor/

Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time) by running the following command:

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Next we need to edit the /etc/snort/snort.conf Snort configuration file as root.

sudo vi /etc/snort/snort.conf

Line 45 of /etc/snort/snort.conf: the variable HOME_NET should match your internal (defended) network. In the below example our HOME NET is 10.0.0.0 with a 24-bit subnet mask (255.255.255.0):

ipvar HOME_NET 10.0.0.0/24

Still editing snort.conf, next we need to modify some file paths to match the lines below, beginning at line 104:

var RULE_PATH /etc/snort/rules
var SO_RULE_PATH /etc/snort/so_rules
var PREPROC_RULE_PATH /etc/snort/preproc_rules

var WHITE_LIST_PATH /etc/snort/rules/iplists
var BLACK_LIST_PATH /etc/snort/rules/iplists

Still editing snort.conf, next we need to enable the local.rules file by un-commenting (remove the hash symbol from the beginning) line 546 so it looks like the following:

include $RULE_PATH/local.rules

Once you have saved your edits to snort.conf, you should test that snort can load this configuration file without any errors. You do this by running snort with the -T flag to tell snort to test the file, the -c flag to identify the path of the snort.conf file, and the -i flag for a network interface that Snort will listen on (Note that Ubuntu 16 can have different interface names, which you can check with the ifconfig command). This is shown below. Output has been truncated to the final few lines to show success:

user@snortserver:~$ sudo snort -T -i eth0 -c /etc/snort/snort.conf
   (...)
   Snort successfully validated the configuration!
   Snort exiting
user@snortserver:~$

Download and Extract the Application Detector Package

Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but will probably have changed, as the Snort team is updating regularly:

cd ~/snort_src
wget https://snort.org/downloads/openappid/4602 -O snort-openappid.tar.gz
tar -xvzf snort-openappid.tar.gz

The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:

sudo cp -r ~/snort_src/odp/ /etc/snort/rules/

and create one folder for third-party developed application detectors:

sudo mkdir /usr/local/lib/thirdparty

Editing snort.conf to enable OpenAppID

We need to enable the OpenAppID pre-processor, then we need to have Snort output the AppID data. The 2.9.9.0 release of snort doesn’t seem to create the default snort.conf correctly (which would include the necessary OpenAppID settings), so rather than enabling them by uncommeting them, we will need to add these lines manually. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf) as root, and add the following lines before the commented-out section 6 (line 513 for me):

preprocessor appid: app_stats_filename appstats-u2.log, \
   app_stats_period 60, \
   app_detector_dir /etc/snort/rules

This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.

While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6, around line 526 ):

output unified2: filename snort.u2, limit 128, appid_event_types

this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.

Now test the Snort configuration file to verify there are no errors:

sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0

as above, you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.

Collecting OpenAppID Data

Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:

sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0
ctrl-c

To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is visible to the interface that snort is listening on, either by passing that data directly through the Snort interface, or by ensuring that your network infrastructure copies network traffic to the Snort server (span port, port mirroring, or promiscuous mode, for example).

Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.

A simple example of this processing:

noah@snort:~$ sudo u2openappid /var/log/snort/appstats-u2.log.1483841898 
statTime="1449426240",appName="HTTP",txBytes="0",rxBytes="8152"
statTime="1449426300",appName="HTTP",txBytes="0",rxBytes="9542"
statTime="1449426240",appName="DNS",txBytes="301",rxBytes="0"
statTime="1449426240",appName="__unknown",txBytes="12376",rxBytes="1118"
statTime="1449426300",appName="DNS",txBytes="761",rxBytes="0"

In the above example, I used curl over the same interface snort was listening on to request www.xkcd.com. The various application detectors show the amount of traffic for each detector, DNS, HTTP, and the like.

An more complex example of this processing (from an older version of OpenAppID, but still valid):

noah@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 
statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894"
statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384"
statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490"
statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490"
statTime="1428300780",appName="https",txBytes="777",rxBytes="1444"
statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116"
statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684"
statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294"
statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282"
statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912"
statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907"
statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327"
statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694"
statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912"
statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625"
statTime="1428300720",appName="dns",txBytes="380",rxBytes="538"
statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943"
statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0"
statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0"
statTime="1428300780",appName="dns",txBytes="482",rxBytes="936"
statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591"
statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974"
statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0"
statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372"
statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"

If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.

If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my in-depth series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.

Installing OpenAppID with Snort 2.9.8.x on Ubuntu

UPDATE: Snort 2.9.9.x has been released. Please see the updated of article here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install OpenAppId in Snort 2.9.8.x on Ubuntu 14.

If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality, see my in-depth series for installing Snort on Ubuntu, or my Quick Install Guide for Snort 2.9.8.x on Ubuntu.

If you want to test the new alpha version of Snort (Version 3.0 Alpha 2), please see my articles: Installing Snort 3 Alpha in Ubuntu 14, or Ubuntu 12.

Let Us Get Started

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev ethtool

Next we want to install the pre-requisites that are specific to OpenAppID:

sudo apt-get install -y libluajit-5.1-dev pkg-config openssl libssl-dev

Disable LRO and GRO for all interfaces Snort will listen on under /etc/network/interfaces. using ethtool. An explanation of LRO and GRO are in the The Snort Manual). Use an editor to edit the network interfaces file:

sudo vi /etc/network/interfaces

and for every interface that Snort will listen on (one interface for simple setups, multiple interfaces for more complex setups), add the following two lines, changing eth0 to match the interface:

post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

for example, my /etc/network/interfaces file looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp
post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

Reboot the system and verify that LRO and GRO are off:

user@snortserver:~$ ethtool -k eth0 | grep receive-offload
generic-receive-offload: off
large-receive-offload: off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

Installing Snort

Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:

Now we are ready to install Snort from source:

cd ~/snort_src
wget https://snort.org/downloads/snort/snort-2.9.8.0.tar.gz
tar -xvzf snort-2.9.8.0.tar.gz
cd snort-2.9.8.0
./configure --enable-sourcefire --enable-open-appid
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

We need to a few configuration things to prepare Snort for use. More detailed information on the steps below can be found here .

Create the needed directories and empty files:

# Create the Snort directories:
sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo mkdir /etc/snort/rules/iplists
sudo mkdir /etc/snort/preproc_rules
sudo mkdir /usr/local/lib/snort_dynamicrules
sudo mkdir /etc/snort/so_rules

# Create some files that stores rules and ip lists
sudo touch /etc/snort/rules/iplists/default.blacklist
sudo touch /etc/snort/rules/iplists/default.whitelist
sudo touch /etc/snort/rules/local.rules
sudo touch /etc/snort/sid-msg.map

# Create our logging directories:
sudo mkdir /var/log/snort
sudo mkdir /var/log/snort/archived_logs

# Adjust permissions:
sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /var/log/snort/archived_logs
sudo chmod -R 5775 /etc/snort/so_rules
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

Finally copy some files:

cd ~/snort_src/snort-2.9.8.0/etc/
sudo cp *.conf* /etc/snort
sudo cp *.map /etc/snort
sudo cp *.dtd /etc/snort
cd ~/snort_src/snort-2.9.8.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/
sudo cp * /usr/local/lib/snort_dynamicpreprocessor/

Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time) by running the following command:

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Next we need to edit the /etc/snort/snort.conf Snort configuration file.

Line 45 of /etc/snort/snort.conf: the variable HOME_NET should match your internal (defended) network. In the below example our HOME NET is 10.0.0.0 with a 24-bit subnet mask (255.255.255.0):

ipvar HOME_NET 10.0.0.0/24

Still editing snort.conf, next we need to modify some file paths to match the lines below, beginning at line 104:

var RULE_PATH /etc/snort/rules
var SO_RULE_PATH /etc/snort/so_rules
var PREPROC_RULE_PATH /etc/snort/preproc_rules

var WHITE_LIST_PATH /etc/snort/rules/iplists
var BLACK_LIST_PATH /etc/snort/rules/iplists

Still editing snort.conf, next we need to modify the whitelist and blacklist path, beginning at line 511:

     whitelist $WHITE_LIST_PATH/default.whitelist, \
     blacklist $BLACK_LIST_PATH/default.blacklist

Once you have saved your edits to snort.conf, you should test that snort can load this configuration file without any errors. You do this by running snort with the -T flag to tell snort to test the file, the -c flag to identify the path of the snort.conf file, and the -i flag for a network interface that Snort will listen on. This is shown below. Output has been truncated to the final few lines to show success:

user@snortserver:~$ sudo snort -T -i eth0 -c /etc/snort/snort.conf
   (...)
   Snort successfully validated the configuration!
   Snort exiting
user@snortserver:~$

Download and Extract the Application Detector Package

Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but will probably have changed, as the Snort team is updating regularly:

cd ~/snort_src
wget https://snort.org/downloads/openappid/3192 -O snort-openappid.tar.gz
tar -xvzf snort-openappid.tar.gz

The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:

sudo cp -r ~/snort_src/odp/ /etc/snort/rules/

and create one folder for third-party developed application detectors:

sudo mkdir /usr/local/lib/thirdparty

Editing snort.conf to enable OpenAppID

We need to enable the OpenAppID pre-processor, then we need to have Snort output the AppID data. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf), and add the following line before the commented-out section 6 (line 513 for me):

preprocessor appid: app_stats_filename appstats-u2.log, \
   app_stats_period 60, \
   app_detector_dir /etc/snort/rules

This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.

While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6, around line 526 ):

output unified2: filename snort.log, limit 128, appid_event_types

this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.

Now test the Snort configuration file to verify there are no errors:

sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0

as above, you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.

Collecting OpenAppID Data

Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:

sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0
ctrl-c

To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is visible to the interface that snort is listening on, either by passing that data directly through the Snort interface, or by ensuring that your network infrastructure copies network traffic to the Snort server (span port, port mirroring, or promiscuous mode, for example).

Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.

A simple example of this processing:

noah@snort:~$ sudo u2openappid /var/log/snort/appstats-u2.log.1449426302 
statTime="1449426240",appName="HTTP",txBytes="0",rxBytes="8152"
statTime="1449426300",appName="HTTP",txBytes="0",rxBytes="9542"
statTime="1449426240",appName="DNS",txBytes="301",rxBytes="0"
statTime="1449426240",appName="__unknown",txBytes="12376",rxBytes="1118"
statTime="1449426300",appName="DNS",txBytes="761",rxBytes="0"

In the above example, I used curl over the same interface snort was listening on to request www.xkcd.com. The various application detectors show the amount of traffic for each detector, DNS, HTTP, and the like.

An more complex example of this processing (from an older version of OpenAppID, but still valid):

noah@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 
statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894"
statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384"
statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490"
statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490"
statTime="1428300780",appName="https",txBytes="777",rxBytes="1444"
statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116"
statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684"
statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294"
statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282"
statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912"
statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907"
statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327"
statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694"
statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912"
statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625"
statTime="1428300720",appName="dns",txBytes="380",rxBytes="538"
statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943"
statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0"
statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0"
statTime="1428300780",appName="dns",txBytes="482",rxBytes="936"
statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591"
statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974"
statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0"
statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372"
statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"

If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.

If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my in-depth series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.

Installing OpenAppID with Snort 2.9.7.x on Ubuntu 14

UPDATE: Snort 2.9.9.x has been released. Please see the updated of article here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install OpenAppId in Snort 2.9.7.2 on Ubuntu 14. If you want a more in-depth explanation of the install steps for Ubuntu (without OpenAppID), as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort on Ubuntu. If you want to test the new 3.0 alpha version of Snort, please see my article: Installing Snort 3 Alpha in Ubuntu.

Preparing to Install Snort

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev

Next we want to install the pre-requisites that are specific to OpenAppID:

sudo apt-get install -y libluajit-5.1-dev pkg-config libssl-dev

Disable LRO and GRO (notes on this in the The Snort Manual), to ensure that packets do not get truncated by the network card:

sudo ethtool -K eth0 gro off
sudo ethtool -K eth0 lro off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.4.tar.gz
tar -xvzf daq-2.0.4.tar.gz	
cd daq-2.0.4
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Installing Snort

Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/snort-2.9.7.2.tar.gz
tar -xvzf snort-2.9.7.2.tar.gz
cd snort-2.9.7.2
./configure --enable-sourcefire --enable-open-appid
make
sudo make install

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

We need to a few configuration things to prepare Snort for use. We create a snort user and group, and copy some files from the Snort source. More detailed information on the steps below can be found here.

First create the user and group:

sudo groupadd snort
sudo useradd snort -r -s /sbin/nologin -c SNORT_IDS -g snort

Then create the necessary directories and empty files:

sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo touch /etc/snort/rules/white_list.rules
sudo touch /etc/snort/rules/black_list.rules
sudo touch /etc/snort/rules/local.rules
sudo mkdir /etc/snort/preproc_rules

sudo mkdir /var/log/snort

sudo mkdir /usr/local/lib/snort_dynamicrules

sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

sudo chown -R snort:snort /etc/snort
sudo chown -R snort:snort /var/log/snort
sudo chown -R snort:snort /usr/local/lib/snort_dynamicrules

Finally copy some files:

sudo cp ~/snort_src/snort-2.9.7.2/etc/*.conf* /etc/snort
sudo cp ~/snort_src/snort-2.9.7.2/etc/*.map /etc/snort

Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time):

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Next we need to edit the /etc/snort/snort.conf Snort configuration file.  I have included the line numbers after the hash so you can more easily find the setting:

var RULE_PATH /etc/snort/rules					# line 104
var SO_RULE_PATH /etc/snort/so_rules			# 105
var PREPROC_RULE_PATH /etc/snort/preproc_rules	# 106

var WHITE_LIST_PATH /etc/snort/rules			# 113
var BLACK_LIST_PATH /etc/snort/rules			# 114

Download and Extract the Application Detector Package

Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but may have changed:

cd ~/snort_src
wget https://snort.org/downloads/openappid/1516 -O snort-openappid.tar.gz
tar -xvzf snort-openappid.tar.gz

The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:

sudo cp -r ~/snort_src/odp/ /etc/snort/rules/

Editing snort.conf to enable OpenAppID

We need to enable the OpenAppID pre-processor, then we need to have snort output the AppID data. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf). You should add the following line before the commented-out section 6 (line 512 for me):

preprocessor appid: app_stats_filename appstats-u2.log, \
   app_stats_period 60, \
   app_detector_dir /etc/snort/rules

This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.

While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6 ):

output unified2: filename snort.log, limit 128, appid_event_types

this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.

Now test the Snort configuration file:

sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0

you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.

Collecting OpenAppID Data

Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:

sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0
ctrl-c

To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is passing through the interface you are listening on (from the same computer, for example).

Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.

An example of this processing:

user@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 
statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894"
statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384"
statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490"
statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490"
statTime="1428300780",appName="https",txBytes="777",rxBytes="1444"
statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116"
statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684"
statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294"
statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282"
statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912"
statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907"
statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327"
statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694"
statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912"
statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625"
statTime="1428300720",appName="dns",txBytes="380",rxBytes="538"
statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943"
statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0"
statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0"
statTime="1428300780",appName="dns",txBytes="482",rxBytes="936"
statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591"
statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974"
statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0"
statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372"
statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"

If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.

If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.