Posts By Noah Dietrich

Installing Snort++ Example Plugins

In this article, I will show you how to install and run the Example Plugins in Snort++. The example plugins are a series of additional plugins that the Snort team has made available for developers to use as examples. This guide assumes that you have cloned Snort++ (Snort 3.0 alpha) from github following my guide here. If you downloaded and installed Snort++ from the tarball on Snort’s website, these instructions may not work for you (since that tarball doesn’t include the snort-extras).

These instructions are tested with the x86 and x64 versions of Ubuntu 14 and 16, with Snort 3.0 Alpha Build-239. If you have installed Snort++ from the github clone on another distribution or architecture, you should be able to modify the below instructions for your specific case. You should ensure that Snort is working correctly before you continue.

Assuming you’ve followed my Snort++ installation guide, you should have a clone of the snort++ github repository under ~/snort_src/. We start by navigating to the extras directory, compile, and install:

cd ~/snort_src/snort3-master/extra
autoreconf -isvf
export PKG_CONFIG_PATH=/opt/snort/lib/pkgconfig/
./configure --prefix=/opt/snort
make
sudo make install

We export PKG_CONFIG_PATH in the above instructions because that tells Snort where the pkgconfig file is for snort when we installed it. if you installed snort to another location, you’ll need to use that path. The above instructions compile and install the plugins, but you need to tell Snort where those plugin directories are when you run it. There are two types of plugins: compiled applications and Lua scripts, and you need to tell snort where to look for both types of extra plugins. To tell snort about these new directories containing the compiled plugins and lua plugins, you pass it the plugin path and/or the script path.

For example, to load the alert_ex plugin (compiled):

snort --plugin-path /opt/snort/lib/snort_extra -A alert_ex --warn-all

I’m using the the warn-all flag to warn of any errors, since snort will not display non-fatal errors by default.

If you want to test the lualert plugin (lua script):

snort --script-path /opt/snort/lib/snort_extra -A lualert --warn-all

To see all available plugins that snort is aware of when running, use the following command:

snort --list-plugins

this will not list the snort-extras plugins, since you haven’t given snort the plugin-path or script-path information. To have snort list all the new script and compiled plugins including those it can see in the extras directory:

snort --script-path /opt/snort/lib/snort_extra --plugin-path /opt/snort/lib/snort_extra --list-plugins

See the snort3 extras readme and the included source files for more information. To see the number of new plugins the extras folder makes available, let’s show all the enabled logging modules by default, then with the additional extras enabled:

noah@snort3: ̃$ snort −−list−plugins | grep logger
logger::alert csv v0 static
logger::alert fast v0 static
logger::alert full v0 static
logger::alert sfsocket v0 static
logger::alert syslog v0 static
logger::log codecs v0 static
logger::log hext v0 static
logger::log pcap v0 static
logger::unified2 v0 static

noah@snort3: ̃$ snort −−script−path /opt/snort/lib/snort extra −−plugin−path /opt/snort/lib/snort extra −−list−plugins | grep logger
logger::alert csv v0 static
logger::alert ex v0 /opt/snort/lib/snort extra/loggers/alert ex.so
logger::alert fast v0 static
logger::alert full v0 static
logger::alert sfsocket v0 static
logger::alert syslog v0 static
logger::alert unixsock v0 /opt/snort/lib/snort extra/loggers/alert unixsock.so
logger::log codecs v0 static
logger::log hext v0 static
logger::log null v0 /opt/snort/lib/snort extra/loggers/log null.so
logger::log pcap v0 static
logger::lualert v0 static
logger::unified2 v0 static

As you can see, when we tell snort to look in the extras directory, we now have additional plugins available for use.

I hope you found this article helpful. If you have issues, you can contact the snort-developers list for assistance. I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Installing Snort++ in Ubuntu (Version 3.0 Alpha 4 build 239)

The instructions below show how to install Snort 3 alpha 4 build 239 on Ubuntu. This install has been tested on Ubuntu 14 and 16, for both the x86 and x64 architectures. For an outdated Ubuntu 12 version of these instructions, please go here. Note that Snort 3 is Alpha software, and therefore has bugs and issues, and should be installed for testing purposes only (not on production systems).

Snort 3 Alpha 4 Build 239 was released on July 28, 2017, and this guide has been tested with that version (releases after this specific release may not follow the same steps). Generic build instructions, prerequisites, and detailed notes are available in the manual.

If you want a more in-depth explanation of the install steps, which are very similar to the 2.9.9.x version of Snort, as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort 2.9.9.x on Ubuntu.

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential autotools-dev libdumbnet-dev libluajit-5.1-dev libpcap-dev libpcre3-dev zlib1g-dev pkg-config libhwloc-dev cmake

Install the optional (recommended) software:

sudo apt-get install -y liblzma-dev openssl libssl-dev cpputest libsqlite3-dev

Install tools required for compiling the source from github:

sudo apt-get install -y libtool git autoconf

Install the DAQ pre-requisites:

sudo apt-get install -y bison flex

If you want to build the documentation as well (not really needed, unless you want it, usually about 700 MB of libraries):

sudo apt-get install -y asciidoc dblatex source-highlight

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

First and install safec for runtime bounds checks on certain legacy C-library calls (this is optional but recommended):

cd ~/snort_src
wget http://downloads.sourceforge.net/project/safeclib/libsafec-10052013.tar.gz
tar -xzvf libsafec-10052013.tar.gz
cd libsafec-10052013
./configure
make
sudo make install

One of the Snort recommended prerequisites is Hyperscan 4.4.0. From their webpage: “Hyperscan is a regular expression engine designed to offer high performance, the ability to match multiple expressions simultaneously and flexibility in scanning operation.” Hyperscan needs Ragel 6.9 and the Boost header libraries.

Install Ragel 6.10 from source:

cd ~/snort_src
wget http://www.colm.net/files/ragel/ragel-6.10.tar.gz
tar -xzvf ragel-6.10.tar.gz
cd ragel-6.10
./configure
make
sudo make install

Download the Boost 1.64 libraries, but do not install:

cd ~/snort_src
wget https://dl.bintray.com/boostorg/release/1.64.0/source/boost_1_64_0.tar.gz
tar -xvzf boost_1_64_0.tar.gz

Install Hyperscan 4.5.1 from source, referencing the location of the Boost source directory:

cd ~/snort_src
wget https://github.com/01org/hyperscan/archive/v4.5.2.tar.gz
tar -xvzf v4.5.2.tar.gz
mkdir ~/snort_src/hyperscan-4.5.2-build
cd hyperscan-4.5.2-build/

cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_64_0/ ../hyperscan-4.5.2

make
sudo make install

If you want to test that Hyperscan works, from the build directory, run:

cd ~/snort_src/hyperscan-4.5.2-build/
./bin/unit-hyperscan

The unit tests will run (this takes a few minutes).

Download and install Data AcQuisition library (DAQ) from the Snort website (note that DAQ for Snort 3 is a different DAQ than for the 2.9.9.x series of Snort):

cd ~/snort_src
wget https://www.snort.org/downloads/snortplus/daq-2.2.2.tar.gz
tar -xvzf daq-2.2.2.tar.gz
cd daq-2.2.2
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Now we are ready to install Snort from source. This command downloads and installs the latest version of Snort 3 (currently 3.0.0 Alpha 4, build 239, but as the codebase is updated, you’ll get a newer version). If you want to specifically download the version used in this guide, use this URL instead with wget below: https://github.com/snortadmin/snort3/archive/BUILD_239.tar.gz.

If you want to install all the snort directories under a single directory, see the section at the bottom of this document titled Changing the install location of Snort. Here we choose to install the entire Snort directory structure to a single folder under /opt/:

cd ~/snort_src
wget https://github.com/snortadmin/snort3/archive/master.tar.gz
tar -xvzf master.tar.gz
cd snort3-master/
autoreconf -isvf

./configure --prefix=/opt/snort

make
sudo make install

Since the Snort installation places the Snort binary at /opt/snort/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /opt/snort/bin/snort /usr/sbin/snort

Snort 3 requires a few environmental variables, we store them temporarily in the current session so we can continue working, and save them permanently to the ~/.bashrc file (you’ll need to do this for every user profile):

export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;
export SNORT_LUA_PATH=/opt/snort/etc/snort
 
sh -c "echo 'export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;' >> ~/.bashrc"
sh -c "echo 'export SNORT_LUA_PATH=/opt/snort/etc/snort' >> ~/.bashrc"

to ensure that these two environmental variables are available when using sudo, we need to add them to the /etc/sudoers file:

sudo visudo

in the editor, add the following to to the bottom of the file:

Defaults env_keep += "LUA_PATH SNORT_LUA_PATH"

use ctrl-x to exit, save when prompted by pressing y, then press enter to save the file to /etc/sudoers.tmp (which will get copied automatically to /etc/sudoers).

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snort3:~$ snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0-a4 (Build 239) from 2.9.8-383
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2017 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.2
           Using libpcap version 1.7.4
           Using LuaJIT version 2.0.4
           Using PCRE version 8.38 2015-11-23
           Using ZLIB version 1.2.8
           Using LZMA version 5.1.0alpha
           Using OpenSSL 1.0.2g  1 Mar 2016
           Using Hyperscan version 4.5.2 2017-08-05

user@snort3:~$

A note on install locations:

When you install snort to /opt/snort, you get the following folder structure:

user@snort3x86:/opt/snort$ tree /opt/snort -L 3
/opt/snort
├── bin
│   ├── snort
│   ├── snort2lua
│   ├── u2boat
│   └── u2spewfoo
├── etc
│   └── snort
│       ├── file_magic.lua
│       ├── snort_defaults.lua
│       └── snort.lua
├── include
│   └── snort
│       ├── actions
│       ├── codecs
│       ├── daqs
│       ├── decompress
│       ├── detection
│       ├── events
│       ├── file_api
│       ├── flow
│       ├── framework
│       ├── hash
│       ├── log
│       ├── lua
│       ├── main
│       ├── managers
│       ├── mime
│       ├── packet_io
│       ├── profiler
│       ├── protocols
│       ├── pub_sub
│       ├── search_engines
│       ├── sfip
│       ├── stream
│       ├── time
│       └── utils
├── lib
│   ├── pkgconfig
│   │   └── snort.pc
│   └── snort
│       └── daqs
└── share
    └── doc
        └── snort

36 directories, 8 files

The /opt/snort/bin folder contains the following Snort binaries:

  • snort : The Snort binary.
  • snort2lua : Tool to convert a Snort 2.9.8.x configuration file into a 3.x configuration file. More notes here.
  • u2boat : U2boat is a tool for converting unified2 files into different formats.
  • u2spewfoo: U2SpewFoo is a lightweight tool for dumping the contents of unified2 files to stdout.

Additionally, the following folders are created / used:

  • /opt/snort/bin : Binaries for Snort and supporting software.
  • /opt/snort/etc/snort : The configuration files for Snort.
  • /opt/snort/include/snort : All include files for Snort.
  • /opt/snort/lib/pkgconfig : The pkgconfig file for Snort (compilation details for Snort).
  • /opt/snort/share/doc/snort : The documentation for the installed version of Snort.

Changing the install location of Snort

If you would rather have all these folders install to a more normal location (/usr/local) , add ‑‑prefix=/usr/local/ to the ./configure command when preparing to build Snort. This will install all these folders under the path you choose. You also need to modify some of the other paths detailed above, so if you decide to install in that manner, you should follow the install instructions detailed in the Snort blog.

Snort Rules

Snort3 rules have more options than Snort 2 rules, and while the normal rules downloaded with PulledPork or manually will work, for testing you will probably want to download the set of community rules specificallycreated for snort3. You can manually download snort3 specific community rules from the snort website:

cd ~/snort_src/
wget https://www.snort.org/downloads/community/snort3-community-rules.tar.gz
tar -xvzf snort3-community-rules.tar.gz
cd snort3-community-rules
sudo mkdir /opt/snort/etc/snort/rules
sudo cp snort3-community.rules /opt/snort/etc/snort/rules/
sudo cp sid-msg.map /opt/snort/etc/snort/rules/

now test that snort can load these rules:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules

your output should contain something similar:

Loading rules:
Loading /opt/snort/etc/snort/rules/snort3-community.rules:
Finished /opt/snort/etc/snort/rules/snort3-community.rules.
Finished rules.
--------------------------------------------------
rule counts
total rules loaded: 3462
text rules: 3462
option chains: 3462
chain headers: 264
--------------------------------------------------
...

you may want to run Snort with the following flags to detect issues: the warn-all and pedantic flags. From the Snort3 manual:
Warnings are not emitted unless –warn-* is specified. –warn-all enables all warnings, and –pedantic makes such warnings fatal

Installing OpenAppID

OpenAppID allows for the identification of application layer traffic. The Snort team has put together a package of detectors, with assistance from the community that you can download and install, called the Application Detector Package which needs to be installed. First download the OpenAppID detector package:

cd ~/snort_src/
wget https://www.snort.org/downloads/openappid/5759 -O OpenAppId-5759
tar -xzvf OpenAppId-5759
sudo cp -R odp /opt/snort/lib/

Now we need to edit our snort configuration file to point to this odp directory:

sudo vi /opt/snort/etc/snort/snort.lua

At line 113 (yours line number may be slightly different) you will see the appid= entry. You will want to add the app detector dir option here, pointing to the parent folder of the odf folder. It should look like this:

appid =
{
    app_detector_dir = '/opt/snort/lib',
}

note that you must have four spaces (not a tab) for the indented line. Now we want to test the configuration file loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua --warn-all

you should see output similar to:

...
Snort successfully validated the configuration (with 0 warnings).
o")~ Snort exiting

Now to load Snort with the OpenAppID detectors, as well as all rules (we omit the pedantic flag, since the rules will throw flowbit warnings that are non fatal and can be ignored:)

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/snort3-community.rules --warn-all

Create a simple rule to test that OpenAppID is working correctly:

sudo touch /opt/snort/etc/snort/rules/local.rules
sudo vi /opt/snort/etc/snort/rules/local.rules

with the following content:

alert tcp any any -> any any ( msg:"Facebook trafic Seen"; appids:"Facebook";sid:10000001; )

test to make sure the rule loads correctly:

/opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules --warn-all

You should see one rule loaded successfully. Now let’s run snort in detection mode on an interface (change eth0 below to match your interface name), printing alerts to the console:

sudo /opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/rules/local.rules -i eth0 -A alert_fast -k none

the -k none flag tells Snort to ignore bad checksums. the Stream and Frag decoders will drop packets that have bad checksums, and the packets will not get processed by the OpenAppID detectors. By including this flag, we ensure that a packet with a bad checksum still gets processed. Now from another window on that computer (open a new terminal window or a second ssh session), use wget
to connect to facebook:

wget facebook.com

from the first console window you will see alerts output similar to the following:

08/05−19:13:45.451834 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 157.240.1.35:443 −> 10.0.0.104:33882
08/05−19:13:45.451842 [∗∗] [1:10000001:0] ”Facebook trafic Seen” [∗∗] [Priority: 0] [AppID: Facebook] {TCP} 10.0.0.104:33882 −> 157.240.1.35:443

use ctrl-c to stop Snort.

Note: if you are collecting packets with a larger MTU that the standard MTU for your adapter (VLAN tagged packets, MPLS Packets, packets from a different network type with a larger MTU), you may need to use the –snaplen flag to adjust snort to process larger packets)

Where to go from here

If you want to learn more about how to run the 2.9.9.x version of Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you want to develop Snort plugins, please see my guide: Installing Snort++ Example Plugins.

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Installing Snort++ in Ubuntu (Version 3.0 Alpha 4 build 237)

NOTE: this article is out of date and has been replaced with a newer article: Installing Snort++ (Snort 3 Alpha 4 build 239) in Ubuntu.

The instructions below show how to install Snort 3 alpha 4 build 237 on Ubuntu. This install has been tested on Ubuntu 14 and 16, for both the x86 and x64 architectures. For an outdated Ubuntu 12 version of these instructions, please go here. Note that Snort 3 is Alpha software, and therefore has bugs and issues, and should be installed for testing purposes only (not on production systems).

Snort 3 Alpha 4 Build 237 was released on July 13, 2017, and this guide has been tested with that version (releases after this specific release may not follow the same steps). Generic build instructions, prerequisites, and detailed notes are available in the manual.

If you want a more in-depth explanation of the install steps, which are very similar to the 2.9.9.x version of Snort, as well as instructions on how to configure and enhance Snort’s functionality, see my series on installing Snort 2.9.9.x on Ubuntu.

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential autotools-dev libdumbnet-dev libluajit-5.1-dev libpcap-dev libpcre3-dev zlib1g-dev pkg-config libhwloc-dev cmake

Install the optional (recommended) software:

sudo apt-get install -y liblzma-dev openssl libssl-dev cpputest cmake libsqlite3-dev

Install tools required for compiling the source from github:

sudo apt-get install -y libtool git autoconf

Install the DAQ pre-requisites:

sudo apt-get install -y bison flex

If you want to build the documentation as well (not really needed, unless you want it, usually about 700 MB of libraries):

sudo apt-get install -y asciidoc dblatex source-highlight

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

First and install safec for runtime bounds checks on certain legacy C-library calls (this is optional but recommended):

cd ~/snort_src
wget http://downloads.sourceforge.net/project/safeclib/libsafec-10052013.tar.gz
tar -xzvf libsafec-10052013.tar.gz
cd libsafec-10052013
./configure
make
sudo make install

One of the Snort recommended prerequisites is Hyperscan 4.4.0. From their webpage: “Hyperscan is a regular expression engine designed to offer high performance, the ability to match multiple expressions simultaneously and flexibility in scanning operation.” Hyperscan needs Ragel 6.9 and the Boost header libraries.

Install Ragel 6.10 from source:

cd ~/snort_src
wget http://www.colm.net/files/ragel/ragel-6.10.tar.gz
tar -xzvf ragel-6.10.tar.gz
cd ragel-6.10
./configure
make
sudo make install

Download the Boost 1.64 libraries, but do not install:

cd ~/snort_src
wget https://dl.bintray.com/boostorg/release/1.64.0/source/boost_1_64_0.tar.gz
tar -xvzf boost_1_64_0.tar.gz

Install Hyperscan 4.5.1 from source, referencing the location of the Boost source directory:

cd ~/snort_src
wget https://github.com/01org/hyperscan/archive/v4.5.1.tar.gz
tar -xvzf v4.5.1.tar.gz
mkdir ~/snort_src/hyperscan-4.5.1-build
cd hyperscan-4.5.1-build/

cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_64_0/ ../hyperscan-4.5.1

make
sudo make install

If you want to test that Hyperscan works, from the build directory, run:

cd ~/snort_src/hyperscan-4.5.1-build/
./bin/unit-hyperscan

The unit tests will run (this takes a few minutes).

Download and install Data AcQuisition library (DAQ) from the Snort website (note that DAQ for Snort 3 is a different DAQ than for the 2.9.9.x series of Snort):

cd ~/snort_src
wget https://www.snort.org/downloads/snortplus/daq-2.2.1.tar.gz
tar -xvzf daq-2.2.1.tar.gz
cd daq-2.2.1
./configure
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Now we are ready to install Snort from source. This command downloads and installs the latest version of Snort 3 (currently 3.0.0 Alpha 4, build 237, but as the codebase is updated, you’ll get a newer version). If you want to specifically download the version used in this guide, use this URL instead with wget below: https://github.com/snortadmin/snort3/archive/3376324350b3ef6228c4e30799a22779413789c2.tar.gz.

If you want to install all the snort directories under a single directory, see the section at the bottom of this document titled Changing the install location of Snort. Here we choose to install the entire Snort directory structure to a single folder under /opt/:

cd ~/snort_src
wget https://github.com/snortadmin/snort3/archive/master.tar.gz
tar -xvzf master.tar.gz
cd snort3-master/
autoreconf -isvf

./configure --prefix=/opt/snort
make
sudo make install

Since the Snort installation places the Snort binary at /opt/snort/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /opt/snort/bin/snort /usr/sbin/snort

Snort 3 requires a few environmental variables, we store them temporarily in the current session so we can continue working, and save them permanently to the ~/.bashrc file (you’ll need to do this for every user profile):

export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;
export SNORT_LUA_PATH=/opt/snort/etc/snort
 
sh -c "echo 'export LUA_PATH=/opt/snort/include/snort/lua/\?.lua\;\;' >> ~/.bashrc"
sh -c "echo 'export SNORT_LUA_PATH=/opt/snort/etc/snort' >> ~/.bashrc"

to ensure that these two environmental variables are available when using sudo, we need to add them to the /etc/sudoers file:

sudo visudo

in the editor, add the following to to the bottom of the file:

Defaults env_keep += "LUA_PATH SNORT_LUA_PATH"

use ctrl-x to exit, save when prompted by pressing y, then press enter to save the file to /etc/sudoers.tmp (which will get copied automatically to /etc/sudoers).

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snort3:~$ snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0-a4 (Build 237) from 2.9.8-383
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2017 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.1
           Using libpcap version 1.7.4
           Using LuaJIT version 2.0.4
           Using PCRE version 8.38 2015-11-23
           Using ZLIB version 1.2.8
           Using LZMA version 5.1.0alpha
           Using OpenSSL 1.0.2g  1 Mar 2016
           Using Hyperscan version 4.5.1 2017-07-18

user@snort3:~$

Now let’s test snort with the default configuration file and ruleset:

user@snort3:~$ /opt/snort/bin/snort -c /opt/snort/etc/snort/snort.lua -R /opt/snort/etc/snort/sample.rules
--------------------------------------------------
o")~   Snort++ 3.0.0-a4-237
--------------------------------------------------
Loading /opt/snort/etc/snort/snort.lua:
	ssh
	pop
	stream_tcp
	gtp_inspect
	stream_icmp
	ftp_server
	stream_udp
	ips
	http_inspect
	wizard
	file_id
	ftp_data
	smtp
	back_orifice
	port_scan
	telnet
	ssl
	sip
	rpc_decode
	reputation
	classifications
	arp_spoof
	appid
	stream_user
	stream_ip
	stream
	dnp3
	ftp_client
	references
	dns
	imap
	stream_file
Finished /opt/snort/etc/snort/snort.lua.
Loading rules:
Loading /opt/snort/etc/snort/sample.rules:
Finished /opt/snort/etc/snort/sample.rules.
Finished rules.
--------------------------------------------------
rule counts
       total rules loaded: 3974
               text rules: 3974
            option chains: 3974
            chain headers: 187
--------------------------------------------------
port rule counts
             tcp     udp    icmp      ip
     any     119      31      29      26
     src    1685       4       0       0
     dst    1927     232       0       0
    both       0       1       0       0
   total    3731     268      29      26
--------------------------------------------------
flowbits
                  defined: 124
              not checked: 9
                  not set: 2
--------------------------------------------------
service rule counts - tcp    to-srv  to-cli
                   dcerpc:        3       0
                     drda:        3       0
                      ftp:        8       2
                     http:     1161    1553
                    ident:        1       0
                     imap:       29    1044
                     ircd:        1       1
                     ldap:        5       0
                    mysql:       29       1
               netbios-ns:        4       0
              netbios-ssn:       25       7
                     pop3:       12    1043
                     rtsp:        2       0
                     smtp:      550       3
                      ssl:        5       1
                   sunrpc:        9       0
                   telnet:        5       1
               vnc-server:        1       3
                    total:     1853    3659
--------------------------------------------------
service rule counts - udp    to-srv  to-cli
                   dcerpc:        2       0
                      dns:      170       2
                 kerberos:        4       4
              netbios-dgm:        7       7
               netbios-ns:        1       1
              netbios-ssn:        1       1
                      ntp:        1       0
                     snmp:        1       1
                     ssdp:        8       0
                   sunrpc:        7       3
                     tftp:        1       0
                    total:      203      19
--------------------------------------------------
fast pattern port groups        src     dst     any
                   packet:       27     125       4
                      key:       23     107       1
                   header:       23     107       1
                     body:        0       2       0
                     file:       23     107       1
--------------------------------------------------
fast pattern service groups  to-srv  to-cli
                   packet:       29      17
                      key:        3       0
                   header:        1       3
                     body:        1       0
                     file:        1       4
--------------------------------------------------
search engine
                instances: 565
                 patterns: 24659
            pattern chars: 515569
               num states: 409078
         num match states: 23799
             memory scale: MB
             total memory: 10.6222
           pattern memory: 1.43181
        match list memory: 3.91914
        transition memory: 5.20227
--------------------------------------------------
pcap DAQ configured to passive.

Snort successfully validated the configuration.
o")~   Snort exiting

user@snort3:~$ 

If you have output similar to the above, then Snort 3.0.0 Alpha 4 is installed and works.

A note on install locations:

When you install snort to /opt/snort, you get the following folder structure:

user@snort3x86:/opt/snort$ tree /opt/snort -L 3
/opt/snort
├── bin
│   ├── snort
│   ├── snort2lua
│   ├── u2boat
│   └── u2spewfoo
├── etc
│   └── snort
│       ├── file_magic.lua
│       ├── sample.rules
│       ├── snort_defaults.lua
│       └── snort.lua
├── include
│   └── snort
│       ├── actions
│       ├── codecs
│       ├── daqs
│       ├── decompress
│       ├── detection
│       ├── events
│       ├── file_api
│       ├── flow
│       ├── framework
│       ├── hash
│       ├── log
│       ├── lua
│       ├── main
│       ├── managers
│       ├── mime
│       ├── packet_io
│       ├── profiler
│       ├── protocols
│       ├── search_engines
│       ├── sfip
│       ├── stream
│       ├── time
│       └── utils
├── lib
│   ├── pkgconfig
│   │   └── snort.pc
│   └── snort
│       └── daqs
└── share
    └── doc
        └── snort

35 directories, 9 files

The /opt/snort/bin folder contains the following Snort binaries:

  • snort : The Snort binary.
  • snort2lua : Tool to convert a Snort 2.9.8.x configuration file into a 3.x configuration file. More notes here.
  • u2boat : U2boat is a tool for converting unified2 files into different formats.
  • u2spewfoo: U2SpewFoo is a lightweight tool for dumping the contents of unified2 files to stdout.

Additionally, the following folders are created / used:

  • /opt/snort/bin : Binaries for Snort and supporting software.
  • /opt/snort/etc/snort : The configuration files for Snort.
  • /opt/snort/include/snort : All include files for Snort.
  • /opt/snort/lib/pkgconfig : The pkgconfig file for Snort (compilation details for Snort).
  • /opt/snort/share/doc/snort : The documentation for the installed version of Snort.

Changing the install location of Snort

If you would rather have all these folders install to a more normal location (/usr/local) , add ‑‑prefix=/usr/local/ to the ./configure command when preparing to build Snort. This will install all these folders under the path you choose. You also need to modify some of the other paths detailed above, so if you decide to install in that manner, you should follow the install instructions detailed in the Snort blog.

Where to go from here

If you want to learn more about how to run the 2.9.9.x version of Snort, and how to install additional software to enhance a Snort system, see my series on installing Snort on Ubuntu. If you want to work with protocol (layer 7) detection, please see my article on OpenAppID.

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Snort IPS With NFQ (nfqueue) Routing on Ubuntu

Overview

This guide will show you how to configure Snort to run inline using the NFQUEUE DAQ (referred to as NFQ). This allows your Snort server to use iptables to route traffic between any number of subnets, with Snort evaluating all traffic passing through the system. This guide will assume some knowledge of routing and IP addressing, especially as it is implemented under Linux, as well as some experience setting up Snort. Configuring Snort as an inline NIPS with NFQ is more complicated than setting snort up as a NIDS, and is more complicated than setting up Snort as a NIPS using the AFPACKET DAQ. The complexity is due to the use of iptables and the need to understand IP routing. I will assume that you have installed Snort before, so I may gloss over some concepts and explanations in this guide. If you find yourself having difficult with installing and configuring Snort, I recommend you first work through my complete guide on installing Snort which should give you a good foundation on understanding how to install and configure Snort.

This guide has been written and tested on Ubuntu 16 x64. It should work on most currently supported versions of Ubuntu and Debian derivatives, but your mileage may vary.

NFQUEUE versus AFPACKET

On Ubuntu, you can run Snort two different ways in inline mode, with AFPACKET or with NFQ. AFPACKET is simpler to setup (see my guide here), but only lets you bridge sets of paired interfaces. This means you can bridge eth0 with eth1 (pass traffic between them), and also bridge eth2 with eth3, but you can not pass traffic between eth0 and eth4. A benefit of AFPACKET is that you can install an inline Snort machine without any changes to IP addressing, routing, or networking changes. NFQ on the other hand lets you leverage the power of iptables to make routing decisions. It’s more complicated to setup (it requires you as a systems administrator to understand Linux routing), but is more powerful for network security. You can also use iptables to blottck traffic before Snort even sees it (to filter out noisy traffic sources), or use additional tools that interface with iptables to increase security (like psad or fwsnort).

IPv6 Notes

Enabling NFQ with IPv6 is very similar to IPv4, the only thing to note is that you will have to run a separate instance of Snort to process IPv6 packets, and setup another iptables rule to forward IPv6 packets to a second NFQUEUE using the ip6tables command. Essentially you follow this guide twice, once for IPv4, and once for IPv6, making changes as necessary.

Steps

This guide will go through the following steps:

  1. Network Configuration
  2. Route Configuration
  3. Enable Kernel IP forwarding on the System
  4. Software Installation
  5. Software configuration
  6. Enabling NFQUEUE
  7. Enabling additional firewall rules

One thing to note: for simplicity, we will enable routing for all traffic passing through the system before we lock down the firewall rules. Because of this, I highly recommend that you test this on a development network to understand how it works before you implement on a production network. In a real environment, you would want your firewall rules in place before you enabled routing of traffic on a device that bridged networks with different security levels. For this guide, it’s much simpler to apply firewall rules at the end for the purpose of learning. I have noted a number of excellent online resources and books at the end of this guide to help understand networking, iptables, and network security.

Network Configuration

For this guide, we will use the following network, with our Snort router having three interfaces on three different networks:

The 172.16.0.0/24 network is our internal admin or management network, and will not be routed by Snort. No traffic should pass between the 172.16.0.0 network and the other networks. This is how we will SSH into this snort system for management purposes, and if configured: Snort could send log data back to a logging server on this network. It is good practice to keep a separate network (through VLANS or separate hardware) for management purposes. We won’t lock down this network at first, but will apply firewall rules at the end of this article to prevent traffic from entering this network across our Snort router.

The 10.0.0.0/24 network is our outbound network. All traffic not destined for a locally connected network will be directed out this network to the next hop gateway (in this case the ISP router). Note that the computer on this network has the Snort router as it’s default gateway. While this works, normally you’d configure the computer to have the actual external gateway (10.0.0.1 in this case) as the gateway. It’s set this way to simplify testing.

The 192.168.0.0/24 network is our client network (screened subnet). Hosts will be configured on this network to use the Snort server as it’s gateway. Traffic passing between this network and the 10.0.0.0 network will be scanned by Snort, and will be dropped and/or logged if suspicious.

Because this server is running Ubuntu 16, the interface names no longer follow the ethX standard (eth0, eth1, …). Instead, interfaces names are assigned as Predictable Network Interface Names. This means you need to check the names of your interfaces using ifconfig -a. In my case, what was originally eth0 is now ens160.

Since you are using your Snort system as a router, you’ll want static interfaces on each address. You don’t want dynamic (DCHP) addresses for the interfaces on this system because any clients on that subnet will use the IP address of the Snort system as their gateway. For any interface that Snort will process traffic on, you need to disable LRO and GRO (there’s an explanation of this in my complete guide on installing Snort). For any interface that sends traffic to an external network, you’ll need a gateway configured.

Edit /etc/network/interfaces as an admin:

sudo vi /etc/network/interfaces

Configure your network interfaces:

# Routed subnet
auto ens160
iface ens160 inet static
address 10.0.0.2
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
post-up ethtool -K ens160 gro off
post-up ethtool -K ens160 lro off

# Host Subnet (internal network / screened subnet)
auto ens224
iface ens224 inet static
address 192.168.0.2
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
# no gateway required
post-up ethtool -K ens160 gro off
post-up ethtool -K ens160 lro off

# Management Subnet (internal network, not routed)
auto ens192
iface ens192 inet static
address 172.16.0.2
netmask 255.255.255.0
network 172.16.0.0
broadcast 172.16.0.255
# no gateway
# snort not processing traffic on this interface, so no need to disable LRO or GRO

Setting up Routes

Because we don’t have additional subnets connected to the 192.1668.0.0 or 172.16.0.0 networks, we don’t need to specify any routes. We only configured one gateway on this system (10.0.0.1), so all traffic not destined to a local subnet will be sent to that gateway. To verify, use the ip route command:

noah@snort-router:~ $ ip route
default via 10.0.0.1 dev ens160 onlink 
10.0.0.0/24 dev ens160  proto kernel  scope link  src 10.0.0.2 
172.16.0.0/24 dev ens192  proto kernel  scope link  src 172.16.0.2 
192.168.0.0/24 dev ens224  proto kernel  scope link  src 192.168.0.2 

If you have multiple subnets configured off of one of those other two subnets, you would need to add a gateway for that subnet in your interfaces file, and you would need to add a static route.

Sidebar: A more complicated example (don’t follow these steps unless you have more complicated routing needs): if you had the 192.168.99.0/24 subnet connected to your 192.168.0.0/24 subnet, reachable through 192.168.0.3, as shown highlighted in red in the image below:

you would need to add two commands to the network interface for ens224: One to specify a gateway, and one to add this route. Since you now have multiple gateways on your Snort server, you want to tell the system which gateway is the default gateway (the gateway for all traffic that it does not have an explicit route for). Interface ens224 on the snort server would now look like:

# Host Subnet (internal network / screened subnet)
# Additional routing example
auto ens224
iface ens224 inet static
address 192.168.0.2
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.3
post-up ethtool -K ens160 gro off
post-up ethtool -K ens160 lro off
up ip route add 192.168.99.0/24 via 192.168.0.3 || true

While ens160 with the default route will look like:

# Routed subnet
# Additional routing example
auto ens160
iface ens160 inet static
address 10.0.0.2
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
post-up ethtool -K ens160 gro off
post-up ethtool -K ens160 lro off
up ip route add default via ens160 || yes

At this time, you should verify that you can ping hosts in every network from your Snort server, as well as external hosts via your gateway. Use the ip route commmand as well as tracepath to verify this is setup right. Do not continue until all this is working.

Firewalls

At this point, even though the system can reach all connected networks, the system will not pass traffic between the networks. Traffic from a host on the management network can not reach any hosts on the screened network, and vice versa. This is by design. Enabling this is easy, however you may want to restrict traffic between certain networks before enabling this option, for security purposes. For simplicity, we will not enable firewall rules at this time, so if your system is a live system connected to both internal sensitive networks and public networks, you may want to read through the entire article first to understand how to add firewall rules to protect the system before enabling routing.

Enable Kernel IP forwarding

Use vi to edit /etc/sysctl.conf

Un-comment the line:

net.ipv4.ip_forward=1			

Reboot the system. At this time, you should now be able to ping between two devices connected to different networks. If your routing is setup correctly, you should be able to ping out the default route as well. If pinging between locally connected subnets work, but you can’t ping to external subnets, you probably need to look at the routing further down the line, do those devices have a path back to your locally connected subnets? If you have a large network, you may want to look at enabling a routing protocol on your Snort server to exchange routing information (probably OSPF or RIP, depending on you network configuration and level of comfort).

Installing DAQ and Snort With NFQ support

If you have a Snort system up and running, you will probably need to re-compile DAQ and Snort for this guide. The reason is that to enable NFQ, you need to install libraries prior to compiling DAQ.

Start by installing all pre-requisites for Snort:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev openssl libssl-dev

We need the development libraries for Nghttp2. On Ubuntu 16 this is simple:

# Ubuntu 16 only (not Ubuntu 14)
sudo apt-get install -y libnghttp2-dev

On Ubuntu 14, we do this from scratch:

# Ubuntu 14 only (not Ubuntu 16)
sudo apt-get install -y autoconf libtool pkg-config
cd ~/snort_src
wget https://github.com/nghttp2/nghttp2/releases/download/v1.17.0/nghttp2-1.17.0.tar.gz
tar -xzvf nghttp2-1.17.0.tar.gz
cd nghttp2-1.17.0
autoreconf -i --force
automake
autoconf
./configure --enable-lib-only
make
sudo make install

Finally we need the NFQ specific libraries:

sudo apt-get install libnetfilter-queue-dev

We will make a directory for all the source tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

after the ./configure option, you will see the DAQ modules that are enabled. You must have NFQ enabled here, as seen below:

Build AFPacket DAQ module.. : yes
Build Dump DAQ module...... : yes
Build IPFW DAQ module...... : yes
Build IPQ DAQ module....... : no
Build NFQ DAQ module....... : yes  <<<<< MUST BE YES
Build PCAP DAQ module...... : yes
Build netmap DAQ module.... : no

Now we are ready to install Snort from source:

cd ~/snort_src
wget https://snort.org/downloads/snort/snort-2.9.9.0.tar.gz
tar -xvzf snort-2.9.9.0.tar.gz
cd snort-2.9.9.0
./configure --enable-sourcefire
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

The last step of our Snort installation is to test that the Snort binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

noah@snort-router:~ $ snort -V

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.9.0 GRE (Build 56) 
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014-2016 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.7.4
           Using PCRE version: 8.38 2015-11-23
           Using ZLIB version: 1.2.8

noah@snort-router:~ $

Now we want to verify that Snort can see the NFQ DAQ. We do this by running snort with the –daq-list flag:

noah@snort-router:~ $ snort --daq-list
Available DAQ modules:
pcap(v3): readback live multi unpriv
nfq(v7): live inline multi	     <<<<<<  Need this line
ipfw(v3): live inline multi unpriv
dump(v3): readback live inline multi unpriv
afpacket(v5): live inline multi unpriv

At this point, Snort has been compiled and installed with the NFQ DAQ. Next is configuration.

Configuring Snort

Most of this configuration comes from my longer articles on installing Snort. I’ll skip over some of the details of certain steps, but more in-depth information is available in my full Snort install guide. First off, for security reasons we want Snort to run as an unprivileged user. We create a snort user and group for this purpose:

sudo groupadd snort
sudo useradd snort -r -s /sbin/nologin -c SNORT_IDS -g snort

Next, we need to create a number of files and folders that Snort expects when running in NIDS mode.  We will then change the ownership of those files to our new snort user:

# Create the Snort directories:
sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo mkdir /etc/snort/rules/iplists
sudo mkdir /etc/snort/preproc_rules
sudo mkdir /usr/local/lib/snort_dynamicrules
sudo mkdir /etc/snort/so_rules

# Create some files that stores rules and ip lists
sudo touch /etc/snort/rules/iplists/black_list.rules
sudo touch /etc/snort/rules/iplists/white_list.rules
sudo touch /etc/snort/rules/local.rules
sudo touch /etc/snort/sid-msg.map

# Create our logging directories:
sudo mkdir /var/log/snort
sudo mkdir /var/log/snort/archived_logs

# Adjust permissions:
sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /var/log/snort/archived_logs
sudo chmod -R 5775 /etc/snort/so_rules
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

# Change Ownership on folders:
sudo chown -R snort:snort /etc/snort
sudo chown -R snort:snort /var/log/snort
sudo chown -R snort:snort /usr/local/lib/snort_dynamicrules

We now need to move the following files from the extracted Snort tarball to the snort configuration folder:

cd ~/snort_src/snort-2.9.9.0/etc/
sudo cp *.conf* /etc/snort
sudo cp *.map /etc/snort
sudo cp *.dtd /etc/snort

cd ~/snort_src/snort-2.9.9.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/
sudo cp * /usr/local/lib/snort_dynamicpreprocessor/

Disable all rule files:

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Now We need to edit our snort configuration file:

sudo vi /etc/snort/snort.conf

Set your internal networks here. I am choosing to have Snort consider the 192.168.0.0 and 172.16.0.0 networks as internal networks. This maters for rule processing, where many rules look for attacks against computers in your HOME_NET subnet ranges:

# line 45 (no spaces between IP addresses)
ipvar HOME_NET [192.168.0.0/24,176.16.0.0/24]

Next we need to tell Snort about the locations of all the folders we created earlier. I have included the line numbers after the hash so you can more easily find the setting (do not write the line number, just change the path to match what is below):

var RULE_PATH /etc/snort/rules						# line 104
var SO_RULE_PATH /etc/snort/so_rules				# line 105
var PREPROC_RULE_PATH /etc/snort/preproc_rules		# line 106

var WHITE_LIST_PATH /etc/snort/rules/iplists		# line 113
var BLACK_LIST_PATH /etc/snort/rules/iplists		# line 114

Enable the Local rules file. Un-comment the following line (line 545) by deleting the hash from the beginning of the line:

include $RULE_PATH/local.rules

Now we need to configure Snort to use the NFQ DAQ an run inline mode. You can choose to pass these options from the command line, but it’s neater to do it here in the configuration file. Add the following three lines to enable the NFQ DAQ in inline mode, looking at NFQ queue number 4 for packets (we will configure iptables later to pass all routed packets to this same numbered queue):

# at line 168, add the following new lines:
config daq: nfq
config daq_mode: inline
config daq_var: queue=4

Save the file, and now we need to test the configuration. To test: we run the following command. The T flag indicates we want to test, the c flag specifies the snort.conf file, and the Q flag tells snort that we’re working in inline mode (required later for Drop rules to actually drop, instead of just alerting):

$ sudo snort -T -c /etc/snort/snort.conf -Q

You should have similar output to the following (truncated for clarity):

Enabling inline operation
Running in Test mode

...

nfq DAQ configured to inline.
Decoding Raw IP4

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.9.0 GRE (Build 56) 
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014-2016 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.7.4
           Using PCRE version: 8.38 2015-11-23
           Using ZLIB version: 1.2.8

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 3.0  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>

Snort successfully validated the configuration!
Snort exiting

If you are seeing similar output, then you’re in a good place. If not, troubleshoot the errors (usually Snort will output line number where the error was found) before continuing.

Testing a Rule

We will add a simple rule to detect ICMP traffic to verify that we’re detecting and passing traffic correctly. Add the following to our local.rules file:
/etc/snort/rules/local.rules:

alert icmp any any -> $HOME_NET any (msg:"ICMP test detected"; GID:1; sid:10000001; rev:001; classtype:icmp-event;)

Since we have made changes to the file that snort loads (local.rules), it is a good idea to test the configuration file again:

sudo snort -T -c /etc/snort/snort.conf -Q

If successful, you should be able to scroll up through the output and see that Snort has loaded our one rule:

		+++++++++++++++++++++++++++++++++++++++++++++++++++
		Initializing rule chains...
		1 Snort rules read
			1 detection rules
			0 decoder rules
			0 preprocessor rules
		1 Option Chains linked into 1 Chain Headers
		0 Dynamic rules
		+++++++++++++++++++++++++++++++++++++++++++++++++++

		+-------------------[Rule Port Counts]---------------------------------------
		|             tcp     udp    icmp      ip
		|     src       0       0       0       0
		|     dst       0       0       0       0
		|     any       0       0       1       0
		|      nc       0       0       1       0
		|     s+d       0       0       0       0
		+----------------------------------------------------------------------------

Configuring NFQUEUE

I will assume you don’t have any entries in your iptables at this time. You can verify with the following command:

$ sudo iptables -L

which should show the following output:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

We want to add a rule to our iptables that makes every packet that is going to be forwarded (not destined for local delivery) to be scanned by Snort. To do this, we add a single rule that moves all forwarded traffic (from the FORWARD queue) to NFQUEUE queue number 4, which is the same queue we specified in our snort.conf configuration above, where Snort is listening for it (I chose queue 4 arbitrarily, you could use any number, as long as it matched the snort.conf queue). To add this rule to the FORWARD chain, we run the following:

sudo iptables -I FORWARD -j NFQUEUE --queue-num=4

Since Snort is not running right now, the system will deliver traffic to that queue, but it will not be processed and will drop (we can change this later to a fail-open option if wanted). Let’s run Snort with the following flags to see traffic being processed:

-A console                    the console option prints fast mode alerts to stdout
-q                            Quiet. Don't show banner and status report.
-u snort                      run snort as the following user after startup
-g snort                      run snort as the following group after startup
-c /etc/snort/snort.conf      the path to our snort.conf file
-Q                            Run inline mode (required to drop packets)

Run Snort with the command below:

sudo /usr/local/bin/snort -A console -q -u snort -g snort -c /etc/snort/snort.conf -Q

If you ping from one machine to another machine, you should see alerts show on your snort machine. You can’t ping to or from the snort machine directly, because this rule only looks at traffic not destined for the local machine.

06/07-14:38:16.275283  [**] [1:10000001:1] ICMP test detected [**] [Classification: Generic ICMP event] [Priority: 3] {ICMP} 10.0.0.88 -> 192.168.0.10
06/07-14:38:17.274174  [**] [1:10000001:1] ICMP test detected [**] [Classification: Generic ICMP event] [Priority: 3] {ICMP} 10.0.0.88 -> 192.168.0.10
06/07-14:38:18.273215  [**] [1:10000001:1] ICMP test detected [**] [Classification: Generic ICMP event] [Priority: 3] {ICMP} 10.0.0.88 -> 192.168.0.10

You have to use ctrl-c to stop snort from running after the above output.

To test that snort will Drop rather than just alert on your rules, edit your local.rules and replace the first word “alert” with “drop”. Re-run Snort as above and try pinging again. You should see alerts, and your host’s should not be able to ping. Note that all other traffic is permitted, our rule is only blocking ICMP traffic. You should see alerts specifying the Drop action:

06/07-14:42:23.935433  [Drop] [**] [1:10000001:1] ICMP test detected [**] [Classification: Generic ICMP event] [Priority: 3] {ICMP} 10.0.0.88 -> 192.168.0.10
06/07-14:42:24.942280  [Drop] [**] [1:10000001:1] ICMP test detected [**] [Classification: Generic ICMP event] [Priority: 3] {ICMP} 10.0.0.88 -> 192.168.0.10

If you were to reboot your computer, you would loose all your iptables configurations. To make them persistent, we need to export the current configuration to a text file, and tell the system to load that text file each time the network starts up. To Export the current settings, we use the iptables-save tool as follows:

sudo sh -c "iptables-save > /etc/network/iptables.rules"

To load the settings, we use the iptables-restore tool in our network interfaces file (/etc/network/interfaces) for our primary network interface, so now the configuration for ens160 looks like this:

auto ens160
iface ens160 inet static
address 10.0.0.2
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
pre-up iptables-restore < /etc/network/iptables.rules
post-up ethtool -K ens160 gro off
post-up ethtool -K ens160 lro off

Reboot and test. You’ll need to manually re-start Snort for any traffic to be passed at this time. Remember that iptables is placing all routed traffic in NFQUEUE number 4, but if there is no application listening to that queue, traffic will not pass.

Fail open

The following is optional based on your security needs. If you want traffic to pass between networks when Snort is not running (fail-open mode), if Snort crashes or stops running for some reason, you’ll want to enable the –queue-bypass flag for the iptables rule. We will need to modify the current rule. First let’s check the state of all chains:

$ sudo iptables -L --line-numbers
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    NFQUEUE    all  --  anywhere             anywhere             NFQUEUE num 4

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination  

we want to modify rule number 1 in the FORWARD chain. To do this, we run the following command, which Replaces (the -R flag) this rule, adding the queue-bypass option:

sudo iptables -R FORWARD 1 -j NFQUEUE --queue-num=4 --queue-bypass

you’ll want to re-save the running configuration as above (overwrite the text file), reboot, and test.

Additional Firewall Rules

Rather than try to fit a small guide for iptables on production systems into this guide, I will refer you to a couple of other excellent short guides on how to use iptables to secure your system. You should at a minimum block all access to the system (the INPUT queue) that isn’t management related or from the management network. Configuring Drop by default rather than accept is recommended.
Basic iptables lock-downs for a system.
More in-depth iptables lock-downs for a server.

At a minimum, to block traffic from going to the management network, you would want to insert the following rule:

sudo iptables -I FORWARD -o ens192 -j DROP

this blocks all routed traffic out the management interface. You have to insert this above the NFQUEUE rule, because once Snort permits the traffic, it does not go through other chains.

Final Steps

You will probably also want to setup PulledPork to download rules automatically, install barnyard2 for log processing, and BASE to view alerts in a GUI. All these steps can be found in my guide on snort, here: Installing Snort

Where to go From here

If you found yourself struggling to get routing working even before you got to the Snort portion of this guide, I highly recommend that you look into taking a networking course. I personally took the Cisco CCNA course offered by the local community college (I think it was only $45 USD a unit). I learned an amazing amount about how networks work, and it made me a much better systems administrator. Don’t think that because you’re a computer administrator that Cisco training isn’t for you, it will compliment your knowledge and abilities.

Books:
The CCNA Books offered by Cisco are excellent:
The CCNA official Cert Guide is what I used to pass my CCNA years ago, and are very well written textbooks. While it’s Cisco specific, most of the basic knowledge in these books are general enough to be useful to anyone working with networking.
The CCNA Routing and Switching Portable Command Guide is one of the few hard-copy books I keep on my shelf and refer to whenever i’m working on Cisco eqipment.

I found the O’reily Network Warior book to be quite satisfying, it gave a broad overview of a number of technologies related to network technology to really broaden my knowledge.

If you’re really serious about TCP/IP, the gold standard (and really heavy) book to have is the The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols Reference.

Links:
How to Enable IP Forwarding in Linux
Understanding the ip route command and IP routing in Linux.

Iptables Tutorial 1.2.2. If you want to know how it works at the architecture level (not the user level, and not the programming level), this is the guide you want to read. I found Chapters three through nine to be most helpful, but if you need some more info on TCP/IP, the first few chapters are quite good as well.

The second flowchart at this page is awesome for understanding how packets flow through iptables. Use it in conjunction with the iptables tutorial above.

the NFQUEUE options for iptables.

This article:
A Deep Dive into Iptables and Netfilter Architecture, may get a little deep into the internal workings of the kernel IP stack, but I found it fascinating. Compliments this SANS article as well.

A few relevant books that get good reviews, but which i haven’t read yet:
Linux iptables Pocket Reference
Linux Firewalls: Attack Detection and Response with iptables, psad, and fwsnort
How Linux Works: What Every Superuser Should Know

Feedback

I hope this article has been helpful to you. Please feel free to provide feedback, both issues you experienced and recommendations that you have. The goal of this guide was not just for you to create a Snort NIPS with NFQ, but to understand how all the parts work together, and get a deeper understanding of all the components, so that you can troubleshoot and modify your Snort NIDS with confidence.

Simple Splunk Scripted Input Example

Overview

In this article, I will walk you through the process of creating a scripted input in Splunk. With a scripted input, you configure your Splunk server or Universal Forwarder (UF) to run a script and capture the output of that script as events to be indexed by Splunk. This article will assume that you have some understanding of Splunk, running python and shell scripts on your system, and understand the difference between a Universal Forwarder and a Splunk Indexer. This article has been tested on Ubuntu 14, running Splunk 6.5. With minor modifications it should work for most Linux and Unix-based systems.

Getting Started

We will assume that our initial goal is to have Splunk run a python script, capturing the output as events. We will configure Splunk to run this python script as a scripted input by creating a new add-on on the Splunk system. To do this, we will create our add-on folder in the apps directory of the Splunk system. The apps directory is located under the etc folder in the $SPLUNK_HOME directory. $SPLUNK_HOME is the location where Splunk was installed. On an indexer, this will often be /opt/splunk, while on a Universal Forwarder, this will often be /opt/splunkforwarder. This guide will assume that you are working on a Universal Forwarder, but all steps can be easily modified for a Splunk Indexer.

Following Splunk’s naming conventions for applications, we will name this application TA-SimpleApp. The TA stands for “Technology Add-on” (Which is different from a Splunk App, which has a GUI). First we will create all the necessary folder and files:

# The path below is the apps folder for a Universal Forwarder
# Change if you are on an indexer (/opt/splunk) or used a different install location

cd /opt/splunkforwarder/etc/apps/

sudo mkdir TA-SimpleApp
sudo mkdir TA-SimpleApp/bin
sudo mkdir TA-SimpleApp/default

sudo touch TA-SimpleApp/bin/TA-SimpleApp.py
sudo touch TA-SimpleApp/default/inputs.conf

Next we need to adjust permissions:

cd /opt/splunkforwarder/etc/apps/
sudo chown -R splunk:splunk TA-SimpleApp

When you install Splunk, a Splunk user is created. We want these folders to be owned by the Splunk user, to ensure that it can access these files.

you should now have the following files and folders:

noah@thor:/opt/splunkforwarder/etc/apps$ tree TA-SimpleApp/
TA-SimpleApp/
├── bin
│   └── TA-SimpleApp.py
└── default
    └── inputs.conf

Now we need to add the content of these two files. First we’ll setup our default inputs.conf (located in the default folder):

[script://./bin/TA-SimpleApp.py]
interval = 10
sourcetype = my_sourcetype
disabled = False 
index = main

A breakdown of this file:

  1. The first line tells Splunk that we are creating a scripted input. We reference the path to the file we want executed (note that we can not pass parameters to the script here, to do that you need a wrapper script, shown below). Rather than referencing the relative path to the script, you can also reference the full path of the script, relative to $SPLUNK_HOME. an example of this would be: [script://$SPLUNK_HOME/etc/apps/TA-SimpleApp/bin/TA-SimpleApp.py]. I prefer the relative path. If you want to reference a location outside of the $SPLUNK_HOME folder heirarchy, you need to use the .path option.
  2. The second line tells Splunk to run this script every 10 seconds. You can also set the script to run on a schedule.
  3. The third line is optional, and tells Splunk they sourcetype of the events (Why this matters).
  4. The fourth line tells Splunk to run this script. Set to True or 1 to prevent Splunk from running this script.
  5. The fifth line is optional, and tells Splunk which index events from this script should be written to. The index must exist on the Indexer, and the default index is main.

Next we add the following content to our python script (TA-SimpleApp.py):


# So we can run this scipt under python 2 or 3
from __future__ import print_function

import sys			            # for sys.stderr.write()
import time 			        # for strftime
from datetime import datetime	# for datetime.utcnow()
import random			        # to provide random data for this example

sys.stderr.write("TA-SimpleApp python script is starting up\n")                  

# output a single event
print (str(time.time()) + ", username=\"agent smith\", status=\"mediocre\", admin=noah, money=" + str(random.randint(1, 1000)))

# output three events, each one separated by a newline (each line will be a unique event)
for x in range(0, 3):
	strEvent = str(time.time()) + ", "
	strEvent += "username=\"" + random.choice(["Stryker", "Valkerie", "Disco Stu"]) + "\", "
	strEvent += "status=\"" + random.choice(["groovy", "hungry", "rage quit"]) + "\", "
	strEvent += "admin=" + random.choice(["lenny", "carl", "moe"]) + ", "
	strEvent += "money=" + str(random.randint(1, 1000))
	print (strEvent)

Line 9 is an example of how we write information to the Splunk event log: $SPLUNK_HOME/var/log/splunk/splunkd.log. See the section below on logging.

Line 12 is where we generate a single event for Splunk to consume. We start by printing the UNIX epoch time, followed by a comma-separated list of keys and values, ending with a newline (the print function ends each line with a newline). Epoch time is preferred because it is easy for Splunk to identify, and it is highly accurate. Splunk will automatically identify the key-value pairs in this instance. If your output is less structured, you would need to configure a props.conf and transforms.conf (a simple example of these files).

Beginning with line 15, we loop thee times to create three random events. Each event will be written to stdout (using the print function at line 21, like above), and will terminate with a newline, which Splunk interprets as the end of the event. You can have Splunk ingest multiple-line events, by configuring Line Breaking in props.conf.

Now we want to test that the python script works correctly. We can do this by simply running it, and making sure we are seeing the stderr and stdout being written correctly to the screen (this is how Splunk will ingest this information). Run the script manually, and look for similar output:

noah@thor:/opt/splunkforwarder/etc/apps/TA-SimpleApp/bin$ python ./TA-SimpleAppy 
TA-SimpleApp python script is starting up
1484997424.92, username="agent smith", status="mediocre", admin=noah, money=627
1484997424.92, username="Valkerie", status="hungry", admin=carl, money=800
1484997424.92, username="Disco Stu", status="rage quit", admin=lenny, money=663
1484997424.92, username="Stryker", status="groovy", admin=moe, money=483

If you have the following output, then your script is correct. Now you need to restart Splunk to have it load your Technology Add-on, and run your script. Reboot Splunk:

noah@thor:~$ cd /opt/splunkforwarder/bin/
noah@thor:/opt/splunkforwarder/bin$ sudo ./splunk restart

Check your log files (see the next section on error logging) and the SplunkWeb search for these events. In the SplunkWeb search app, you should see something similar to:

Congratulations, if you have similar output to above, you now have a simple scripted input for Splunk. On a UF, you have to ensure that python is available. On an indexer, Splunk will use it’s own version of python (2.7.5), and on a UF, you’ll use the system’s version of python.

Error Logging

One challenge is that you have to be an administrator to view this log file, or even browse the log folder. To view this folder, I usually use sudo bash in a separate terminal window to work with log files. Because Splunk captures all output from your script, we need to differentiate between events and log information. We do this by writing events to stdout, which is ingested by Splunk, becomes an event, and is indexed. Anything written to stderr is captured by Splunk and is written to the splunkd.log file. This event will look like the following entry in the splunkd.log:

01-21-2017 09:43:09.216 +0200 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-SimpleApp/bin/myAppLauncher.sh" TA-SimpleApp myAppLauncher.sh is starting

You will notice that Splunk marks this event as an ERROR in the log, there doesn’t seem to be a way to modify this event to change the severity.

A good way to follow these events on a Universal Forwarder is as follows:

root@thor:/opt/splunkforwarder/var/log/splunk# sudo tail -f splunkd.log | grep ExecProcessor
01-21-2017 09:49:32.684 +0200 INFO  ExecProcessor - New scheduled exec process: python /opt/splunkforwarder/etc/apps/TA-SimpleApp/bin/TA-SimpleApp.py
01-21-2017 09:49:32.684 +0200 INFO  ExecProcessor - 	interval: 10 ms
01-21-2017 09:49:35.195 +0200 ERROR ExecProcessor - message from "python /opt/splunkforwarder/etc/apps/TA-SimpleApp/bin/TA-SimpleApp.py" TA-SimpleApp python script is starting up

On an indexer (rather than a UF), this command generates too much information (there are a number of apps that are started by Splunk on an Indexer, where on a UF there is only the one we created). On an indexer, you may want to grep for the name of the app: TA-SimpleApp instead.

Wrapper Script

If you need to pass parameters to your script, or you want to execute an application, you can have Splunk call a shell script, where you have more options in launching your script. Let’s create this script:

cd /opt/splunk/etc/apps
sudo touch TA-SimpleApp/bin/myAppLauncher.sh
sudo chmod a+x TA-SimpleApp/bin/myAppLauncher.sh
sudo chown splunk:splunk TA-SimpleApp/bin/myAppLauncher.sh

We need to modify our inputs.conf to call this new shell script, rather than the python script directly. Modify the first line of inputs.conf to look like this (everything else is the same):

[script://./bin/myAppLauncher.sh]

And enter the following content for this shell script (myAppLauncher.sh):

#!/bin/bash

# Write a line to the splunk log file
echo TA-SimpleApp myAppLauncher.sh is starting >&2

# Check if we can run a python interpreter, exit otherwise
command -v python >/dev/null 2>&1 || { echo No python interpreter found. >&2; exit 1; }

# execute our python script, located in the same folder as this script
cd $( dirname "${BASH_SOURCE[0]}" )
python ./TA-SimpleApp.py

On line 4, we are writing to stderr, which adds the output to an event to the splunkd.log event log (this would probably be removed in a final release, but it is good for testing).
On line 7 we are checking that we can find a python interpreter to run our script. If we can’t find one, we write an error to the splunkd.log log file and quit.
on line 10 we are setting the current directory to the directory of this shell script.
on line 11 we are calling our python script, which will execute and generate output just like the example above.

This example is very similar to launching a python script directly, however we now have the ability to pass parameters, do additional setup, change environmental variables (if needed), test for a python interpreter, and other housekeeping. You could also call a binary executable, output to your own log files, and other required setup or testing for your script. Basically, if you can do it from a shell script, and it generates output to stdout and stderr, you can use it as a scripted input. For any scripts or applications you execute here (including all child applications), all data written to stdout will become an event on your Splunk indexer. All errors written to stderr will become entries in the splunkd.log log file.

Testing your script as Splunk will run it

When Splunk runs your script, it is doing it within it’s own environment, with it’s own environmental variables. To test your script within that environment (without setting it up as an App), you can run ./splunk cmd from the bin folder:

noah@thor:/opt/splunkforwarder/bin$ ./splunk cmd ../etc/apps/TA-SimpleApp/bin/myAppLauncher.sh 

you’ll see the following output:

TA-SimpleApp myAppLauncher.sh is starting
TA-SimpleApp python script is starting up
1484994520.21, username="agent smith", status="mediocre", admin=noah, money=736
1484994520.21, username="Valkerie", status="hungry", admin=carl, money=141
1484994520.21, username="Stryker", status="rage quit", admin=carl, money=941
1484994520.21, username="Disco Stu", status="groovy", admin=moe, money=168

All data written to stdout and sdterr will show on the screen as your application writes it. Splunk is not running your app and processing the output, this tool merely lets you test your script to see if it would run in the environment that Splunk would run it. If you want to see the environmental variables for Splunk’s environment, just add the printevn command to the end of your shell script and run the ./splunk cmd command again. A simple example of the difference in environmental variables:

noah@thor:/opt/splunkforwarder/bin$ printenv | grep "splunk"
PWD=/opt/splunkforwarder/bin

noah@thor:/opt/splunkforwarder/bin$ ./splunk cmd /usr/bin/printenv | grep "splunk"
PATH=/opt/splunkforwarder/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin
PWD=/opt/splunkforwarder/bin
_=./splunk
SPLUNK_HOME=/opt/splunkforwarder
SPLUNK_DB=/opt/splunkforwarder/var/lib/splunk
SPLUNK_ETC=/opt/splunkforwarder/etc
SPLUNK_WEB_NAME=splunkweb
LD_LIBRARY_PATH=/opt/splunkforwarder/lib
OPENSSL_CONF=/opt/splunkforwarder/openssl/openssl.cnf
LDAPCONF=/opt/splunkforwarder/etc/openldap/ldap.conf

Where To Go From Here

If you want to see some examples of Splunk Tecnology Add-ons, just download them from the Splunkbase and browse through their files, or look at the links below.

Feedback is welcomed, especially if there are errors in this guide or recommendations you have from your own experience. Please contact me here.

Resources

Setting up a scripted input
Add a scripted input with inputs.conf
Writing reliable scripts
Anatomy of an app
Advanced Python Script Testing
A good Scripted Inputs Tutorial
Building Splunk Technology Add-ons From the Splunk blog.
Package and publish a Splunk app
Apps and add-ons: an introduction

Creating eBooks from AsciiDoc

Overview

This guide will show you how to convert AsciiDoc files into EPUB and Kindle’s .mobi format using open source software on Linux. This guide will assume you have some experience with Linux, and a general understanding of html, css, and XML will be very helpful. I will walk you through all the steps required to create these files, explaining how each tool works so that you can troubleshoot and adapt the workflow to make it work for you. My goal is for you to really understand how this process works, rather than just following rote steps without really understanding them. I will include links to related resources as we go through that will help round out your understanding. At the end of this article, you will have a Makefile that will allow you to easily build and modify eBooks.

This guide is tested on Ubuntu 14 x64, although it should work on most Debian-based systems without much effort. It could easily be used on any Linux-based system with a minimum amount of modification.

Overview of the Workflow

AsciiDoc is a simple document format that allows you to easily mark up text in an easy to read format, which can also be easily converted to a specific type of XML that is used for EPUBs. From Wikipedia: “AsciiDoc is a human-readable document format, semantically equivalent to DocBook XML, but using plain-text mark-up conventions.” A good overview of AsciiDoc can be found on the AsciiDoctor website.

Documents written in AsciiDoc will be converted to DocBook XML, a semantic language made for technical documentation. This DocBook XML is converted to EPUB3 html files using DocBook XSL stylesheets. XSL Stylesheets are used to convert XML to another format, in this case html files (eBooks, including .mobi and EPUB are merely an archive of html files and cascading style sheets, essentially a web page wrapped into a single archive). We will then turn those html files into an eBook using a few different tools.

Tools

All tools and files here are open source or free. We will use the following tools:

  1. asciidoctor: Used to convert AsciiDoc to DocBook XML.
  2. jing: An XML docbook schema validator.
  3. xsltproc: Used to convert Docbook XML to epub files and folders
  4. epubcheck: Validate and convert epub files & folders into single file.
  5. calibre: Calibre is a eBook viewer.
  6. KindleGen: Amazon tool to convert epub to mobi (kf8 and the older .mobi file).

We will also use the following files (download instructions are below):
The DocBook schema (in Relax NG schema Language).
The DocBook XSL Stylesheets, used to convert DocBook XML to HTML files from the DocBook Project.

Enviornment

This guide will have you setup a folder to hold the files for your eBook and all required files. It will assume that the filename of your eBook in adoc format is myBook.adoc. In this guide, we will setup all the required files, as well provide sample content for each file, so you will be able to create a full eBook from the example, and can then modify it after everything is working to match your workflow.

First, let’s install the required software:

sudo apt-get install -y asciidoctor jing xsltproc epubcheck calibre

Next let’s create the folder that will hold all the files for this eBook. We’re using a folder called ebook on the Desktop. We’ll create a number of necessary folders here as well:

mkdir ~/Desktop/ebook/
mkdir ~/Desktop/ebook/build-resources/
mkdir ~/Desktop/ebook/ebook-resources/
mkdir ~/Desktop/ebook/ebook-resources/graphics/
mkdir ~/Desktop/ebook/output/ 
cd ~/Desktop/ebook/

The build-resources folder will hold the DocBook XSL Stylesheets and the DocBook Schema file that we will download. This folder holds files that can be used for any ebook. The ebook-resources folder will store files that are specific to this one eBook, including css files, graphics (like the cover of the ebook), and any other files you want to include in your ebook. The output file is where our final products will be stored (the .mobi and EPUB files).

Now let’s create a simple AsciiDoc file. This will be the source material for our ebook. This format is text-based, and is simple to read and create. Since it’s text, it can also be added to your favorite version control tool (SVN, subversion, or the like). Here we will edit the adoc file using your favorite editor:

touch ~/Desktop/ebook/myBook.adoc
xdg-open ~/Desktop/ebook/myBook.adoc

with the following content:

= Witty Book Title
:doctype: book
:backend: docbook
:docinfo:
:!numbered:
:imagesdir: graphics

[dedication]
== My Dedications
This book is dedicated to.....

I'd also like to thank....

== This is the First Chapter

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas vehicula congue dolor, vel commodo magna viverra ac. Morbi ullamcorper, est eu egestas semper, velit elit bibendum orci, ut tristique tellus nulla sit amet ipsum. Sed fringilla, lacus sed viverra dictum, nulla augue placerat lectus, in efficitur magna risus non nibh. Ut laoreet, tortor at tempus mollis, magna risus ullamcorper dolor, quis rutrum ex augue a risus. Vivamus pellentesque accumsan est aliquam fringilla. Quisque eleifend ac eros in volutpat. Quisque eu euismod metus, at blandit diam. Phasellus in magna eget erat finibus lacinia quis at metus. Cras ut hendrerit sem.

Vivamus ligula est, volutpat nec convallis eget, efficitur at orci. Proin non aliquet nunc. Mauris dui odio, bibendum consectetur ligula at, faucibus dapibus est. Praesent porttitor, nisi sit amet accumsan euismod, sem felis semper turpis, ut fermentum leo orci sit amet tortor. Nulla eros leo, eleifend vitae ornare quis, mollis tristique eros. In quis accumsan arcu. Ut hendrerit vitae sem ut consectetur. Nunc enim massa, tempus id orci vitae, rhoncus laoreet nulla. Pellentesque elementum purus rutrum, condimentum elit vitae, sagittis magna. Maecenas ornare justo et arcu consequat, nec volutpat risus fermentum.

== This is the Second Chapter

Duis cursus ac augue id blandit. Nulla varius accumsan odio, sed vestibulum odio lobortis quis. Nunc vitae ipsum tortor. Ut ut eros dignissim est luctus finibus ac quis nulla. Etiam consequat, neque sit amet laoreet laoreet, magna odio ornare justo, et ornare sapien nunc quis dolor. Praesent felis metus, facilisis a quam id, euismod faucibus nibh. Mauris venenatis dui erat, vel auctor felis tempus eget. Pellentesque tellus metus, pretium aliquam tristique eget, bibendum ut sapien. Curabitur magna augue, feugiat id enim congue, ullamcorper iaculis arcu. Integer pulvinar elit nulla, at gravida velit sodales eget. In quis leo ac mauris mollis facilisis fermentum non ex. Ut a lorem lacinia, egestas sem eu, tincidunt risus. Proin non ornare lacus, vitae imperdiet erat.

== This is the Third Chapter

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas vehicula congue dolor, vel commodo magna viverra ac. Morbi ullamcorper, est eu egestas semper, velit elit bibendum orci, ut tristique tellus nulla sit amet ipsum. Sed fringilla, lacus sed viverra dictum, nulla augue placerat lectus, in efficitur magna risus non nibh. Ut laoreet, tortor at tempus mollis, magna risus ullamcorper dolor, quis rutrum ex augue a risus. Vivamus pellentesque accumsan est aliquam fringilla. Quisque eleifend ac eros in volutpat. Quisque eu euismod metus, at blandit diam. Phasellus in magna eget erat finibus lacinia quis at metus. Cras ut hendrerit sem.

Vivamus ut feugiat neque, sed varius tortor. Phasellus sit amet ante ut tortor pulvinar efficitur non non massa. Curabitur imperdiet justo nec urna cursus, sit amet dapibus lectus posuere. Aliquam hendrerit nisi eget nunc aliquet, gravida aliquam elit volutpat. Donec semper tincidunt neque in aliquet. Curabitur lobortis rutrum felis quis tempus. In bibendum neque vitae ipsum tempus aliquet. Maecenas euismod consequat pellentesque.

Maecenas in tincidunt nibh. Aliquam tempus libero non augue finibus fermentum. Sed massa leo, tempus sollicitudin consequat vel, ornare nec ligula. Donec et tellus bibendum, blandit nunc sed, porttitor magna. Integer eget pulvinar lorem. Sed in bibendum quam. Donec eu molestie ipsum, at maximus ipsum. Nunc vel mauris vulputate, faucibus dui quis, imperdiet enim. Phasellus sodales turpis quis velit egestas, in rhoncus diam pellentesque.

There are a number of interesting things happening in this file. The text we are using is Lorem ipsum, common filler text used by typesetters to allow you see how text looks on the screen or in print without getting caught up in the content of the text. The header of the adoc file begins with the Title of the book on the first line, underlined on the next line with a number of equals signs. The third line says that when we convert this file, we want the final document type to be a book (more on different document types here.) The backend actually gets overwridden at the command line when we convert this file later, but is good to have. The docinfo file command indicates that there is a docinfo.xml file (the one we will create below) that contains further header information. The Numbered line indicates that we don’t want page numbers. The first section we have is the dedication. The two equals signs indicate a chapter heading.

Now create the css file. This css file tells our ebook (EPUB or .mobi) how to be displayed, including font size, color, any anyting else that can be configured with css).

touch ~/Desktop/ebook/ebook-resources/master.css
xdg-open ~/Desktop/ebook/ebook-resources/master.css

and enter the following information:

html, body { height: 100%; margin: 0; padding:0; border-width: 0; }
 @page { margin: 5pt; }

/* indent paragraph */
 h2 + p {
 text-indent:0;
 }
 p {
 text-indent:1em;
 margin: 0;
 }
/* Set the minimum amount of lines to show up on a separate page. (There is not much support for this at the moment.)
https://github.com/reitermarkus/epub3-boilerplate/blob/master/Ebook/OPS/css/main.css*/
p,
blockquote {
  orphans: 2;
  widows: 2;
}
/* page break for dedication (xslst keeps on same page as the copyright */

div.dedication {
	page-break-before:always;
}

/* Move the legal notice from the title page to its own page */
div.legalnotice{
	page-break-before:always;
}

/* Tile Page formatting */
div.book div.titlepage h1{
	font-family: Helvetica,Arial,sans-serif;
	text-align: center;
	color: blue;
}

The docinfo file is an xml file that holds information about the author and the copyright information. This file needs to have the same name as your AsciiDoc file with -docinfo.xml appended, and be in the same folder as your adoc file. In this example, our AsciiDoc book is named myBook.adoc, so the docinfo file is named myBook-docinfo.xml. Create this file:

touch ~/Desktop/ebook/myBook-docinfo.xml 

with the following content (note that there can’t be any blank space at the beginning of this file):

Important note: use a text editor that will automatically recognize that you’re working in an XML file, so that it will format this file correctly (replacing the spaces with tabs). It is important that this file be formatted correctly, or you’ll get errors.

<author>
	<personname>
		<honorific>Mr</honorific>
		<firstname>Noah</firstname>
		<surname>Dietrich</surname>
	</personname>
</author>
<copyright>
  <year>2017</year>
  <holder>SublimeRobots Intl.</holder>
</copyright>
<legalnotice>
  <para>
    Copyright &#169; 2017 by Noah Dietrich
  </para>
  <para>
    All rights reserved. This book or any portion thereof may not be reproduced or used in any manner whatsoever without the express written permission of the publisher except for the use of brief quotations in a book review.
  </para>
  <para>
    Printed in the United States of America
  </para>
  <para>
    First Printing, 2017
  </para>
  <para>
    ISBN 0-9000000-0-0
  </para>
  <para>
    Jim &amp; Joe Publishers, LLC
  </para>
  <para>
    www.SublimeRobots.com
  </para>
</legalnotice>
<cover>
	<mediaobject>
		<imageobject>
			<imagedata fileref="graphics/cover.jpg">
			</imagedata>
		</imageobject>
	</mediaobject>
 </cover>

This information will be added to your ebook when it is processed, but is not stored in the adoc file. A good example of a docinfo file can be found here.

Next, we need to get the DocBook XML Schema file (docbookxi.rng) and the DocBook XSLT stylesheets. We will store them in our build-resources folder:

cd ~/Desktop/ebook/build-resources
wget http://docbook.org/xml/5.2b01/rng/docbookxi.rng

wget http://downloads.sourceforge.net/project/docbook/docbook-xsl-ns/1.79.1/docbook-xsl-ns-1.79.1.tar.bz2
tar -xvjf docbook-xsl-ns-1.79.1.tar.bz2

Finally, we need to download the KindleGen application from Amazon. Navigate to the KindleGen homepage, download the linux version, and extract the kindleGen binary to the build-resources folder.

cd ~/Desktop/ebook/build-resources
wget http://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz
tar -xzvf kindlegen_linux_2.6_i386_v2_9.tar.gz kindlegen

The Book Cover: You’ll need to put a jpeg for the cover into the graphics folder named cover.jpg. If you don’t do this, you’ll need to remove the cover section in your myBook-docinfo.xml file. You can get information on recommended jpeg sizing here.

Begin Processing

The first step is converting the adoc file into DocBook XML format. This part can be a little frustrating sometimes, as many small semantic issues can cause errors to show up at this stage. Some issues I have encountered (and quick fixes for them if I have an answer):

  • Double Spaces. This can be fixed with: sed -i.bak ‘/^$/d;G’ myBook.adoc
  • Spaces at the begining of lines: This can be fixed with sed -i.bak ‘s/^[ \t]*//’ myBook.adoc
  • Empty Chapters.

from your ebooks directory, assuming you have all the above files setup correctly, run the following command:

cd ~/Desktop/ebook/
asciidoctor --backend docbook5 --doctype book --verbose --destination-dir ./output/ myBook.adoc

here we are using the asciidoctor application to convert the adoc file into an AsciiDoc XML file. This XML file will have all the same content as our original adoc file, only formatted (and marked up semantically) to meet the DocBook schema (docbookxi.rng). The schema describes the legal layout of all valid files. We are using these options:

  • --backend docbook5: Remember when I said above we’d override the backend command in the header of the adoc file? this is the line that does that. This line says that we want to use the docbook5 backend, which will generate DocBook 5.0 XML.
  • --doctype book: We also have this information in the header, but it’s better to be sure. The command line overrides the setting in the header as well, if there is a difference.
  • --verbose: We want as much output as possible, good for troubleshooting.
  • --destination-dir ./output/: we want the xml file that is created to be put into the output folder
  • myBook.adoc: this is the filename of the adoc file we want converted to DocBook XML.

you should see output similar to:

noah@thor:~/Desktop/ebook$ asciidoctor --backend docbook5 --doctype book --verbose --destination-dir ./output/ myBook.adoc
Input file: myBook.adoc
  Time to read and parse source: 0.00527
  Time to render document: 0.00944
  Total time to read, parse and render: 0.01476
noah@thor:~/Desktop/ebook$ 

If you you have errors, you may see output similar to:

asciidoctor: WARNING: myBook.adoc: line 9: invalid style for paragraph: dedication

This usually means that there is either an error in your header, or you docinfo xml file has an error (spaces instead of tabs, extra spaces between elements, and similar issues). You must fix these issues before continuing.

Now that we have our ebook in the DocBook XML file format, we want to validate that it is semantically correct. We want to check to see that its format matches the schema defined in the docbookxi.rng file (the schema in Relax NG schema language). For this, we use a tool called jing. There is another tool called xmllint, that also does validation, but I encountered issues with it, and found jing to be much more reliable. An excellent resource for understanding the details can be found in the Processing DocBook5 section in the DocBook XSL: The Complete Guide (you’ll be referencing this online ebook a lot if you want to do any configuration of your ebook).

So the content of our adoc file and the docinfo file have been combined into a single xml file in the output directory (you can open it to see what it looks like), and we need to validate it to make sure it’s formatted correctly (sometimes asciidoctor makes mistakes). To do this, we run the following command from the same directory as before (not the output directory):

cd ~/Desktop/ebook/
jing -i ./build-resources/docbookxi.rng output/myBook.xml

This command is simple, it takes the docbookxi.rng schema file as the first input (-i), and our book in xml format as our second input, and will tell us if it’s valid (properly formatted) DocBook XML. If you have issues, try to figure out what line of the xml file is causing the issue, and try to track it back to the original asciidoc or docinfo file. This can be a challenge to do, sometimes searching the internet for your error can help.

If you see no output, then there are no errors.

Next, we are going to use xsltproc to convert our DocBook XML file into a series of html files (HTML 5 files actually), copy in our css and images to create a folder that represents our entire ebook, including all required resources.

It helps here to understand how ebook file systems are laid out before they are zipped into an archive we consider an EPUB or mobi file. A basic EPUB has the following files and folder heirarchy stored in a zipped container:

mimetype
META-INF/
  container.xml
OEBPS/
  content.opf
  chapter1.xhtml
  chapter2.xhtml
  css/
    style.css
  toc.ncx
  graphics/
    cover.jpg

Good explanations of these files can be found on Wikipedia, as well as here and here.

We need to convert our valid DocBook XML into the above folder structure. Do do this, we use xsltproc, whic applies XSLT stylesheets to XML documents. XSLT stylesheets are a language for converting XML into other formats (in our case, HTML documents). The XSLT stylesheets are provided by the DocBook project. The following command converts our DocBook XML into an EPUB folder hierarchy:

cd ~/Desktop/ebook/
xsltproc --stringparam base.dir ./output/epub3-book/OEBPS/ --stringparam chapter.autolabel 0 --stringparam chunker.output.indent  yes ./build-resources/docbook-xsl-ns-1.79.1/epub3/chunk.xsl ./output/myBook.xml

Let’s break this down:

  1. xsltproc: The application that applies the XSLT stylesheets.
  2. –stringparam base.dir ./output/epub3-book/OEBPS/: Where will the EPUB folders be output. The epub3-book folder is the location of the EBOOK. this option requires the location of the OEBPS folder.
  3. –stringparam chapter.autolabel 0: Do not number chapter headings (I feel it looks better without numbers on chapters).
  4. –stringparam chunker.output.indent yes : Make the html output pretty (helps for troubleshooting)
  5. ./build-resources/docbook-xsl-ns-1.79.1/epub3/chunk.xsl: These are the XSLT stylesheets.
  6. ./output/myBook.xml: The source DocBook XML file.

the stringparam options above are specific to the XSLT files that we are working with. To find other options that are available, read through Chapter 7. HTML output options of DocBook XSL: The Complete Guide.

You should see output similar to:

noah@thor:~/Desktop/ebook$ xsltproc --stringparam base.dir ./output/epub3-book/OEBPS/ --stringparam chapter.autolabel 0 --stringparam chunker.output.indent  yes ./build-resources/docbook-xsl-ns-1.79.1/epub3/chunk.xsl ./output/myBook.xml
Writing ./output/epub3-book/OEBPS/bk01-toc.xhtml for book
Writing ./output/epub3-book/OEBPS/ch01.xhtml for chapter(_this_is_the_first_chapter)
Writing ./output/epub3-book/OEBPS/ch02.xhtml for chapter(_this_is_the_second_chapter)
Writing ./output/epub3-book/OEBPS/ch03.xhtml for chapter(_this_is_the_third_chapter)
Writing ./output/epub3-book/OEBPS/index.xhtml for book
Writing ./output/epub3-book/OEBPS/docbook-epub.css for book
Generating EPUB package files.
Writing ./output/epub3-book/OEBPS/cover.xhtml for mediaobject
Generating image list ...
Writing ./output/epub3-book/OEBPS/package.opf for book
Writing ./output/epub3-book/OEBPS/../META-INF/container.xml for book
Writing ./output/epub3-book/OEBPS/../mimetype for book
Generating NCX file ...
Writing ./output/epub3-book/OEBPS/toc.ncx for book
noah@thor:~/Desktop/ebook$ 

We also need to manually move our css file and images into the EPUB folder hierarchy (add any additional graphics you need at this stage):

cd ~/Desktop/ebook/
cp ./ebook-resources/master.css ./output/epub3-book/OEBPS/docbook-epub.css
cp -r ./ebook-resources/graphics/ ./output/epub3-book/OEBPS/

The next step is to convert our EPUB folders into a single file (our actual EPUB). To do this we use epubcheck, then rename the file:

epubcheck  ./output/epub3-book/ -mode exp -v 3.0 -save
mv ./output/epub3-book.epub ./output/myBook.epub

Here we are using -mode exp to have epubcheck validate the expanded EPUB archives, version 3.0, and save it to a single file: ./output/myBook.epub.
This epub file is the first final product. You can view this epub on any epub compatible reader (including calibre, which we installed earlier).

The final step is to convert out epub into the mobi format, for use on Amazon Kindle devices. This is done with KindleGen. This tool is simple, it takes the name of the epub folder to convert, and the name of the .mobi to create:

cd ~/Desktop/ebook/build-resources
./kindlegen ../output/myBook.epub

if you look in your ./output folder, you will now see your epub and .mobi files. If you have an amazon device, you can email the .mobi file to yourself and have it automatically download to your device. All kindle devices support this .mobi format. More information can be found here and here.

Automating the Build Process

You will quickly find that as you are modifying your files, it becomes a hassle to constantly run these commands. The solution to this is to use a Makefile. This tool was originally designed to compile software, but can be easily modified to simplify your ebook workflow.
in your ebook folder, create a new file called Makefile:

cd ~/Desktop/ebook/
touch Makefile

enter the following text (as with the docbook file above, replace spaces at the beginning of lines with tabs if needed):

mobi : epub
	#ebook-convert ./output/myBook.epub ./output/myBook.mobi
	./build-resources/kindlegen ./output/myBook.epub

epub : ebook
	epubcheck  ./output/epub3-book/ -mode exp -v 3.0 -save
	mv ./output/epub3-book.epub ./output/myBook.epub

ebook : docbook
	xsltproc --stringparam base.dir ./output/epub3-book/OEBPS/ \
		--stringparam chapter.autolabel 0 \
		--stringparam chunker.output.indent  yes \
		./build-resources/docbook-xsl-ns-1.79.1/epub3/chunk.xsl ./output/myBook.xml
	cp ./ebook-resources/master.css ./output/epub3-book/OEBPS/docbook-epub.css
	cp -r ./ebook-resources/graphics/ ./output/epub3-book/OEBPS/

docbook : 
	asciidoctor --backend docbook5 --doctype book --verbose --destination-dir ./output/ myBook.adoc
	jing -i ./build-resources/docbookxi.rng output/myBook.xml

.PHONY: clean
clean : 
	-rm -rf ./output/*

Open a command prompt, navigate to the ebook folder, and you can now build your ebook by issuing the command make complete. If you want to delete all old versions of the ebook, you can run make clean. if you get an error: Makefile:3: *** missing separator. Stop, then you need to replace all spaces with tabs at the beginning of lines (there are issues pasting tabs from a website into a document).

Some of the options you have here:

  • make docbook: Convert your adoc to DocBook XML and validate.
  • make ebook: This runs make docbook, and then creates the ebook folders.
  • make epub: This runs the above two commands, then converts the ebook folders into a single epub
  • make mobi: This runs the above commands, and finally converts the epub into a mobi
  • make clean: delete all files in the output folder.

You can modify this makefile to match your workflow, such as adding options to xsltproc (new lines are broken up with a backslash to improve readability), or having more files added to your ebook directory.

Conclusion

This guide has given you a simple framework for creating an ebook workflow. There are a number of things that can be improved or modified in this process to suit your needs, but hopeful you have learned enough to make these modifications yourself. You’ll probably want to improve the css files for your ebook (there are a number of websites that can better discuss epub css options, some of them are linked below). You may want to look at embedding images into your book, using specific fonts, adding different parameters to the XSL transforms, and many other options.

Feedback is welcomed, especially if there are errors in this guide or recommendations you have from your own experience: please contact me here.

Helpful Links

DocInfo.xml example.
Oreily Publications docinfo.xml example for erlang book.
publishing with iBooks example docinfo.xml.
another Oreily docbook.xml example.

Amazon Kindle Publishing Guidelines

CSS Boilerplate for eBooks.
Basic css styles for Kindle html.

The eBook Design and Development Guide on Amazon.

These two guides below use a2x from the asciidoc package, rather than asciidoctor to generate the xml from docbook. I prefer asciidoctor, as i find that it worked better for my workflow.
A good guide on converting docbook to epub and mobi.
Another good guide.

Snort 2.9.9.x on Ubuntu – Part 8: Conclusion

  1. Installing Snort
  2. Configuring Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort on Ubuntu 14
  7. Creating systemD Scripts for Snort on Ubuntu 16
  8. Installing BASE
  9. Conclusion

Where to Go From Here

I hope this series of articles has been helpful to you. Please feel free to provide feedback, both issues you experienced and recommendations that you have. The goal of this guide was not just for you to create a Snort NIDS, but to understand how all the parts work together, and get a deeper understanding of all the components, so that you can troubleshoot and modify your Snort NIDS with confidence.

Capturing More Traffic With Snort

You will probably want to configure your network infrastructure to mirror traffic meant for other hosts to your Snort sensor. This configuration is dependent on what network equipment you are using. If you are running Snort as a Virtual Machine on a VMware ESXi server, you can configure promiscuous mode for ESXi by following my instructions in this article: configure promiscuous mode for ESXi.

For different network infrastrucutre, you will need to do a little research to configure network mirroring for your Snort server. Cisco calls this a span port, but most other vendors call this Port Mirroring. Instructions for Mikrotik (a linux based switch and router product that I like).  If you run DD-WRT, it can be configured with iptables, like any linux based system. If you have network equipment not listed above, any search engine should point you towards a solution, if one exists. Note that many consumer switches will not have the ability to mirror ports.

You can also purchase devices specifically made to mirror data (called taps). Some products that have been recommended on the Snort-Users list are:

More Advanced Snort Configuration

Snort has the ability to do much more than we’ve covered in this set of articles. Hopefully you’ve learned enough through this setup that you will be able to implement more advanced configurations and make Snort work for you. Some things that Snort is capable of:

Some other related articles I have written:

Recommended Reading

Feedback

I would love to get feedback from you about this guide. Recommendations, issues, or ideas, please contact me here.

Snort 2.9.9.x on Ubuntu – Part 7: Installing BASE

  1. Installing Snort
  2. Configuring Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort on Ubuntu 14
  7. Creating systemD Scripts for Snort on Ubuntu 16
  8. Installing BASE
  9. Conclusion

Installing BASE On Ubuntu

BASE is a simple web GUI for Snort. Alternate products include Snorby, Splunk, Sguil, AlienVault OSSIM, and any syslog server.

Splunk is a fantastic product, great for ingesting, collating, and parsing large data sets. Splunk is free to use (limited to 500 MB of data per day, which is a lot for a small shop). Sguil client is an application written in tcl/tk. Snorby is abandoned, and relies on old versions of many Ruby packages that makes documenting the installation difficult, and a constantly changing target.

I’ve chosen to use BASE in this guide because it’s simple to setup, simple to use, and works well for what it does. Both BASE and Snorby are abandoned projects, and while Snorby gives a nice web-2.0 interface, since it is written in Ruby-on-Rails, the Ruby packages it relies on are constantly upgrading, which causes compatibility issues with other required Snorby packages, which causes too many installation problems. If you want to try installing Snorby, please see these unsupported out of date guides for Ubuntu 14 or Ubuntu 16.

There is a slight difference between BASE on Ubuntu 14 versus 16: BASE requires PHP 5, which isn’t available in the Ubuntu 16 archives (Ubuntu has moved on to PHP 7 in this release), so we have to use a PPA on Ubuntu 16 to install the php 5 packages:

# Ubuntu 16 only:
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install -y apache2 libapache2-mod-php5.6 php5.6-mysql php5.6-cli php5.6 php5.6-common php5.6-gd php5.6-cli php-pear php5.6-xml

in Ubuntu 14, we can just install the necessary libraries:

# Ubuntu 14 only:
sudo apt-get install -y apache2 libapache2-mod-php5 php5 php5-mysql php5-common php5-gd php5-cli php-pear

next install Pear image Graph:

sudo pear install -f --alldeps Image_Graph

Download and install ADODB:

cd ~/snort_src
wget https://sourceforge.net/projects/adodb/files/adodb-php5-only/adodb-520-for-php5/adodb-5.20.8.tar.gz
tar -xvzf adodb-5.20.8.tar.gz
sudo mv adodb5 /var/adodb
sudo chmod -R 755 /var/adodb

Download BASE and copy to apache root

cd ~/snort_src
wget http://sourceforge.net/projects/secureideas/files/BASE/base-1.4.5/base-1.4.5.tar.gz
tar xzvf base-1.4.5.tar.gz
sudo mv base-1.4.5 /var/www/html/base/

Create the BASE configuration file:

cd /var/www/html/base
sudo cp base_conf.php.dist base_conf.php

Now edit the config file:

sudo vi /var/www/html/base/base_conf.php

with the following settings (note that the trailing slash on line 80 is required, despite the instructions in the configuration file):

$BASE_urlpath = '/base';                   # line 50
$DBlib_path = '/var/adodb/';               #line 80
$alert_dbname     = 'snort';               # line 102
$alert_host       = 'localhost';
$alert_port       = '';
$alert_user       = 'snort';
$alert_password   = 'MySqlSNORTpassword';   # line 106

While in the base conf.php file, you will also want to comment out line 457 (we don’t want the DejaVuSans font), and un-comment (remove the two backslashes) from line 459, enabling a blank font. The section for fonts (begining at line 456) should look like this:

//$graph_font_name = "Verdana";
//$graph_font_name = "DejaVuSans";
//$graph_font_name = "Image_Graph_Font";
$graph_font_name = "";

Set permissions on the BASE folder, and since the password is in the base conf.php file, we should prevent other users from reading it:

sudo chown -R www-data:www-data /var/www/html/base
sudo chmod o-r /var/www/html/base/base_conf.php

restart Apache:

sudo service apache2 restart

The last step to configure BASE is done via http:

  1. Browse to http://ServerIP/base/index.php and click on the setup page link (replace ServerIP with the IP of your Snort Server).
  2. Click on the Create BASE AG button on the upper right of the page.
  3. Click on the Main page link.

Note: If you read through the BASE configuration file, there are a number of other options you can implement if you like. A few options are SMTP Email alerts, IP Address to Country Support, and user authentication.

Congratulations, if you’ve made it this far, you have a fully-functioning Snort system. Please continue on to the Conclusion for more things you can do with Snort.

Snort 2.9.9.x on Ubuntu – Part 6b: Creating systemD Scripts for Snort on Ubuntu 16

  1. Installing Snort
  2. Configuring Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort on Ubuntu 14
  7. Creating systemD Scripts for Snort on Ubuntu 16
  8. Installing BASE
  9. Conclusion

Overview

In the previous articles in this series, we have created a complete Snort NIDS with a web interface and rulesets that automatically update.  In this article, we will finalize the configuration of our Snort service by creating systemD scripts for the Snort and Barnyard2 daemons. If you are running Ubuntu 14, you should go see my Upstart article instead of this article.

Creating a systemD startup script in Ubuntu 16

Ubuntu 16 has moved to systemD for services / daemons. For more information about creating and managing systemD servcies, please see this excellent article.

To create the Snort systemD service, use an editor to create a service file:

sudo vi /lib/systemd/system/snort.service

with the following content (change ens160 if different on your system):

[Unit]
Description=Snort NIDS Daemon
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i ens160

[Install]
WantedBy=multi-user.target

Now we tell systemD that the service should be started at boot:

sudo systemctl enable snort

And start the Snort service:

sudo systemctl start snort

Verify the service is running

systemctl status snort

Next, create the Barnyard2 systemd service. We will add two flags here: -D to run as a daemon, and -a /var/log/snort/archived logs, this will move logs that Barnyard2 has processed to the /var/log/snort/archived/ folder. Use an editor to create a service file:

sudo vi /lib/systemd/system/barnyard2.service

With the following content:

[Unit]
Description=Barnyard2 Daemon
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -q -w /var/log/snort/barnyard2.waldo -g snort -u snort -D -a /var/log/snort/archived_logs

[Install]
WantedBy=multi-user.target

Now we tell systemD that the service should be started at boot:

sudo systemctl enable barnyard2

And start the barnyard2 service:

sudo systemctl start barnyard2

Verify the service is running

systemctl status barnyard2

Reboot the computer and check that both services are started

user@snortserver:~$ service snort status
snort start/running, process 1116
user@snortserver:~$ service barnyard2 status
barnyard2 start/running, process 1109
user@snortserver:~$

If both services are running, you are ready to move to the next section, where you will install BASE, a web-based GUI to view and profile alert data: Installing BASE

Snort 2.9.9.x on Ubuntu – Part 6a: Creating Upstart Scripts for Snort on Ubuntu 14

  1. Installing Snort
  2. Configuring Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort on Ubuntu 14
  7. Creating systemD Scripts for Snort on Ubuntu 16
  8. Installing BASE
  9. Conclusion

Overview

Creating the Upstart Scripts for Ubuntu 14

In the previous articles in this series, we have created a complete Snort NIDS with a web interface and rulesets that automatically update.  In this article, we will finalize the configuration of our Snort service by creating Upstart scripts for the Snort and Barnyard2 daemons. If you are running Ubuntu 16, you should go see my systemD article instead of this article.

First create the Snort Upstart script:

sudo vi /etc/init/snort.conf

We will insert the below content into this Upstart script.  Note that we are using the same flags that we used in earlier articles, so if Snort ran correctly for you earlier, then you shouldn’t need to change any of these flags:

description "Snort NIDS service"
stop on runlevel [!2345]
start on runlevel [2345]
script
    exec /usr/sbin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D
end script

Now make the script executable, and tell Upstart that the script exists:

sudo chmod +x /etc/init/snort.conf
initctl list | grep snort
	snort stop/waiting

do the same for our Barnyard2 script:

sudo vi /etc/init/barnyard2.conf

with the following content:

description "barnyard2 service"
stop on runlevel [!2345]
start on runlevel [2345]
script
    exec /usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -g snort -u snort -D -a /var/log/snort/archived_logs
end script

Note that we have added a new flag here that we didn’t use before: -a /var/log/snort/archived_logs, this will move logs that Barnyard2 has processed to the /var/log/snort/archived_logs/ folder.

Now make the script executable, and tell Upstart that the script exists:

sudo chmod +x /etc/init/barnyard2.conf
initctl list | grep barnyard
	barnyard2 stop/waiting

Reboot the computer and check that both services are started:

user@snortserver:~$ service snort status
snort start/running, process 1116
user@snortserver:~$ service barnyard2 status
barnyard2 start/running, process 1109
user@snortserver:~$

If both services are running, you are ready to move to the next section, where you will install BASE, a web-based GUI to view and profile alert data: Installing BASE